Why this AI project is really about enforcement
This build doesn’t begin with a shiny demo prompt or a breathless promise about replacing spreadsheets. It starts with public money, the kind councils are already required to publish and with a very ordinary problem: the records exist, but they’re scattered, inconsistent and annoying to work with. England’s councils have to disclose payments above a fairly low threshold, yet the material is spread across dozens of local websites, tucked into PDFs, CSVs and other file formats that don’t play nicely together. Technically open, yes. Practically searchable? Not remotely.
That gap is where the project gets interesting. In tech news terms, it’s less “look what the model can do” and more “look what happens when someone asks AI to do the boring enforcement work.” The point is not to generate a slick answer from thin air. It is to take a pile of publicly available council payment data and make it legible enough that a human can actually inspect where local money goes, who gets paid, and what patterns show up when you stop treating every council website like a separate archaeological dig.
The real test of AI here is whether it can turn public records into something a person can use without a weekend, three browser tabs, and a mild headache.
That’s a very different product from the usual AI policy spectacle. The system chewed through billions of tokens, used a few hundred direct prompts from the builder and handed off work to hundreds of subagents. That scale sounds absurd until you remember what it was doing: hunting down official payment pages, pulling files from awkward corners of council sites and normalizing them into something searchable. A flashy chatbot can talk about accountability. This one had to help produce it.
The end goal’s pretty plain, which is part of the charm. Instead of a grand theory of digital culture, you get a civic tool that works a bit like Apple Maps for local spending. Zoom out and you can scan a map. Zoom in and you can see the councils, the categories, the suppliers, and the flows of public cash. That’s a more modest ambition than “changing government,” but it’s also more believable. If ai policy is going to mean anything beyond conference panels and nervous headlines, it probably has to pass tests like this one first: can the machine make public data easier to find, compare and question? The next problem’s whether it can keep doing that when the files get messy, which is where the real grind starts.
Finding the money across dozens of council sites
Once the project moved from idea to actual data collection, the hardest part turned out not to be the model work at all. It was finding where the files lived. England’s councils do publish payment records, often because they’re supposed to, but the documents are scattered across official sites, tucked behind awkward download pages and saved in formats that seem designed by committee after a long lunch. So the builder sent subagents off in parallel to search the web, map the field, and assemble a catalogue of about three dozen official sources before any serious downloading began.
That first pass mattered because it kept the build from assuming the obvious answer was the right one. A council might have a page titled “spend over £500,” but the actual file could sit in a PDF one month, an Excel workbook the next and a broken archive link after that. Every download was logged with a checksum-style fingerprint, so the system could tell whether a file had changed after it was fetched. Or corrected a row without announcing it, the build had a way to notice, if a council quietly swapped out a spreadsheet. That sounds dull. It’s also the difference between a searchable record and a pile of floating numbers.
A public file that can change without warning needs a fingerprint, not a shrug.
Along the same lines, the first clean batch started small. It covered five councils, then expanded to eight after the earlier files had been checked, normalized and made comparable. By that point the dataset was already large enough to stop feeling like a sample and start feeling like a proper national scrape. The total came to nearly two million rows and about nine billion pounds in spending. That’s a lot of council coffee, road repairs, consultants and whatever else gets booked into public ledgers under names that only make sense to finance teams and exhausted procurement staff. For tech news readers, the interesting bit is that the scale came from ordinary records, not some heroic burst of proprietary data access.
The messy part, naturally, was the parts that didn’t cooperate. A few authorities made scraping awkward enough that they left holes in the early dataset, including a couple of large counties whose sites resisted clean extraction. The records were still public, but the path to them was hostile to automation, which is a polite way of saying the files were technically available and practically annoying. Later passes filled in some of those gaps, though not all of them at once. That left the dataset uneven for a while, which is exactly the sort of thing that happens when public administration meets code and neither side agrees to be graceful about it.
That friction is where power and politics show up in this kind of work. If a council publishes tidy spreadsheets, its spending is easy to inspect. If it buries the same data in tangled PDFs or clumsy portals, the public can still have access in theory while getting very different results in practice. The same basic logic runs through the EU’s AI Act enforcement rules and its guidance on high-risk AI systems: before anyone trusts an automated system, someone has to check what it is actually seeing. In this case, that meant treating every file as evidence, not as a promise.
The payoff was less glamorous than a flashy demo, but far more useful. Instead of a vague promise about making public spending visible, the project had a living dataset with known gaps, known checks and enough rows to ask real questions. It also gave the builder a map of where the collection effort still had to go, which is where the next stage started to get interesting.
The agent workflow: nonstop collection, auditing, and storage
Once the council pages had been mapped out, the job stopped looking like a neat data project and started looking like what it was: a grind. The first pass was meant to cover England and Wales, but Wales was dropped when it became obvious the disclosure rules don’t line up cleanly with England’s. That sounds like a footnote until you try to mix two reporting systems that use different habits, different file types and different levels of cooperation from the people publishing them. At that point, one sloppy merge can ruin the whole batch.
The workflow that took over after that ran almost continuously for close to two days. A set of AI agents kept cycling through the same sequence: find the page, grab the file, flatten the format, audit what came out, load it into storage, then go back and do it again. In plain English, it was a machine built to do the kind of repetitive clerical work that a person could do once or twice before muttering into a coffee cup. At scale, though, that repetition matters and council sites change. Downloads break. A PDF shows up where last week there was a spreadsheet. The loop had to catch all of it.
The boring loop is where the real system gets built.
That loop also had to keep its own receipts. Files were not just pulled in and trusted. They were checked, normalized, and then checked again before they were written down as usable data. That is the sort of unglamorous discipline that separates a flashy demo from something that can survive contact with real public records. In the language of ai policy, it is the part people talk around when they discuss enforcement: what gets recorded, what gets verified, and what happens when the evidence does not match the claim. The same logic shows up in official planning documents too, including the EU AI Act governance and enforcement framework and the White House’s 2025 AI action plan, both of which care less about applause and more about what can actually be checked.
A normal laptop would’ve choked on the volume. So the heavier lifting moved onto a Mac Mini, which is about as glamorous as a municipal records room, and that was the point. The data was stored in columnar form for the big tables, then pushed into a single-file database so the whole thing stayed searchable without spinning up a full backend. That setup sounds modest because it is. It also makes a lot of sense when you want to query millions of rows without turning the whole exercise into a systems administration hobby.
The scale got silly fast. By the time the run had settled, the system had either collected or parked data for most English councils, which is the kind of phrase that makes data people nod and everyone else reach for a biscuit. More than five hundred threads had been involved along the way. Some were digging through sites for new pages. It downloading files. Others were flattening stubborn formats or checking whether the same council had changed its publication layout since the last pass. A few simply sat there waiting for a page to stop timing out, which is apparently a very modern form of patience.
What made the process work wasn’t raw speed. It was the refusal to move on too quickly. The agent didn’t pretend it had succeeded, when one download failed. The pipeline handled the cleanup instead of leaving it for later, which is how later turns into never, when a council published in a format that needed cleaning before it could be queried. When the data looked off, the audit step caught it and sent the system back around the loop.
That’s the part that matters for the rest of the project. Search box, or polished interface could exist, the underlying pile had to be made searchable and trustworthy enough to survive public scrutiny, before any map. The work was messy, repetitive and a little obsessive. Which is exactly what you want when the source material is a patchwork of local government files and the tools are AI agents doing administrative labor at a scale that’d make a standard desktop sigh audibly.
From raw spreadsheets to a map people can use
Once the council files were finally gathered, the harder question showed up: what, exactly, should people do with a mountain of UK council spending data that mostly lives in spreadsheets nobody wants to open before lunch?
A few directions were tried at the same time. One prototype looked like a receipt. Another borrowed the feel of a bank statement, line by line, as if local government had decided to become a particularly dull fintech app. There was also a county grid, which made the data feel orderly but a bit too admin-heavy, and a retailer-style mini-site that treated suppliers almost like products on a shelf. Those ideas were useful. None of them quite solved the main problem, which was scale. People needed to scan the whole country, not a single ledger strip or a neat little dashboard corner.
The best interface wasn’t the one that looked clever. It was the one that let a person spot patterns in seconds.
The map won because it answered the first question fast. Where’s the money going? With council boundaries drawn in, users could jump from one area to another without digging through filing-cabinet levels of data. Category filters gave the page some shape, so someone could isolate spending on software, consultants, travel, or whatever else had turned up in the records. Ranked lists helped too, because maps are lovely, but they don’t always tell you which supplier is swallowing the biggest slice of cash. Search rounded it out. If a vendor name was known, you could look it up instead of scrolling like a sleep-deprived auditor.
That choice also changed the tone of the project. The interface stopped feeling like a dump of public records and started feeling like something a normal person might actually use after work, or on a train, or in the middle of a mildly suspicious budget discussion. For public data transparency, that matters more than it sounds. A dataset can be technically open and still be functionally hidden if the structure’s miserable enough.
The visual direction moved too. But a bit flat, given the first plain map was serviceable. After comparing screenshots and testing the layout on desktop and mobile, the design shifted toward something closer to Apple Maps: cleaner panels, easier scanning, less clutter competing for attention. That kind of polish sounds cosmetic until you see it on a phone. On a small screen, every extra label becomes a tiny act of sabotage. The cleaner version gave the country room to breathe while keeping the controls close enough to use without cursing at your thumb.
Then came the less glamorous part, which is usually where these projects either survive or turn into a mess with a nice colour palette. Thousands of unknown vendors had to be classified. Duplicate supplier names needed merging, because public procurement data loves to spell the same company three different ways. Odd naming quirks were flattened out too, so a council payment to one slightly mangled version of a vendor wouldn’t sit beside another payment to what was clearly the same outfit wearing a different hat. None of that’s flashy. It is, however, the difference between a map that looks smart and one that lies by accident.
At the same time, there was also a bit of machine assistance in the background. Codex helped push through some of the layout and cleanup work, but the real labour was still human judgment: deciding what counted as the same supplier, what belonged in which category and what made the interface easier to read without sanding off the useful detail. That mix of speed and scrutiny’s where the project starts to feel less like a demo and more like a working tool.
By the time the design settled, the point had become fairly clear. The data didn’t need another dramatic treatment. It needed a shape ordinary people could understand in a few clicks, with enough search, filtering and visual order to make the spending visible without making users do archaeology first.
What the last stretch says about AI policy
The final phase of the build was mostly a long, slightly nerdy loop of review, fix and check again. Code got edited, data got cleaned, design details got nudged and the builder kept feeding notes and screenshots back into the process so the agents could revise what they had missed. Quick aside. That part may sound dull compared with the first rush of scraping, parsing and mapping. It wasn’t dull in practice. What stands out: it was the bit where the whole thing either held together or quietly fell apart.
The flashy part is the map; the real work is deciding what counts, what loads, and what gets checked twice.
That’s where AI policy starts to look less like a demo and more like enforcement. Anyone can build a system that appears to work when a small sample’s loaded. The harder job is making sure the numbers are right, the categories make sense, and the interface isn’t quietly lying by omission. The project had already pushed data into DuckDB and Parquet, which made it easier to query huge tables without a giant backend. Even so, there were still plenty of places for a bad assumption to sneak in and sit there looking official.
One of those assumptions showed up as a hidden spending cap during testing. The app was static, so it had been limiting how much data it pulled in. In plain English, the front end was only showing part of the picture. That kind of bug isn’t glamorous, but it’s exactly the sort of thing that matters when you’re dealing with public money and power and politics. If the app stops at the wrong threshold, users don’t see a subtle flaw. They see a version of reality with a chunk missing.
The same problem came up in the category work. Some spending could sit comfortably inside broad buckets, but not everything behaved that neatly. Pension-related spending, for example, raised the question of whether it deserved its own category rather than being tucked into a larger catch-all. That sort of judgment call can feel annoyingly small until you remember that categories shape what people notice first. You bury the pattern, if you bury the item. Fair enough. If you split it out, you may surface something useful. There isn’t always a perfect answer, which is probably why the review stage took so much back-and-forth.
The plan is to open-source the project after the last tweaks are done, which should make the plumbing visible instead of magical. That seems like the right ending. The work here hasn’t been about proving that AI can make a map look polished. It’s been about making public data accurate enough to trust, searchable enough to use and honest enough to survive a second glance. The glamour shot is the interface. And the real story is the audit trail.
