Official Data Sucks

experience thus far with big data sets

This article is an update on Gumshoe. Gumshoe is an investigation platform I started building because I wasn’t pleased with answers I got writing my congressmen. The question first posed was: “if they don’t listen to me, who do they listen to?”.

I got it to the point where I was mapping geographical positions of campaign donations over temporality. It was originally built with Neo4J because it felt like the right fit, but it wasn’t a good experience for 100’s of millions of enteries. Bolt doesn’t handle that volume well. So I eventually ditched it. I needed a lot of data, and I needed the analytical database to handle it which was clickhouse. The focus on columnar data in clickhouse helps me more than the row based focus of other databases. But I grew and realized I needed to query/search all the disparate data I had.

pywb was way too tedious and arcane. It didn’t work for what I needed, and to add to that it is python to which I’m not a fan.

I started on gumpla (play on gunpla but with gumshoe). It started as a config driven system to build entities, aliases, topics, topic groups, and facts. I build out a “pad” to join different data sets, then I built out functionality to create an insertable csv artifact that had all the components needed to have graph like functionality within clickhouse. I established an onnx ner pipeline to derive entities from unstructured data and feed them into the same pipeline.

Before my realization I was able to work with about 56M rows on my desktop. Not enterprise, but its a lot to me.

I was happy to start playing with the data until I realized how much straight up bullshit exists in official data. Not even plausible bullshit. Straight up absurd bullshit.

My first encounter was finding a $100M political donation from a committee that was null/bunk/didn’t really exist. This committee and subsequent donation reports were from an eccentric man in a Carolina. To give context, Elon Musk donated $30M. This man reported he donated $70M more than Elon musk from his residential address in a middle class neighborhood. So it turns out you can just file whatever number you want. With that I was intrigued by the guy and his story. He evidently not only had a penchant for making bogus committee filings and subsequent absurd donations, but he was also really interested in geothermal properties. And yes, he filed bogus bids to the bureau of land management. Its harmless, and I’m happy he has a hobby. But it really screws my numbers and faith.

So, my thoughts then were “surely the SEC has way more intergrity!”

Nope.

After ingesting the bulk num, pre, tag, sub files for the 4 quarters in 2025 I ran across another anomaly: MDB Capital Holdings apparently earning three times Apple's quarterly income. In fact, in the SEC data MDB earn 4X what Apple did. Wow, this is interesting! Kinda..

I traced it a $419 billion dimensional XBRL fact attached to a 14-person biotech startup that raised $17 million in its IPO. The problem started to really form. I needed to stop fighting bad data and derive context that explains its meaning. MDB Capital Holdings, the numbers needed a lot of context to figure out if they are meaningless or actively misleading. This ran me down a huge rabbit hole of XBRL. I guess its an active problem. But XBRL is too niche and I’m not an economist or whatever.

Every data set I touch turned out to have absurd anomalies. Anomalies you would think would be caught and corrected. Bad assumption on my part. For all my faith that the SEC makes the suits tremble, they’re completely fine being like “Your 14 person company earned $300B more than apple? Sounds legit.”

Why fight to clean it? Cleaning it is a losing battle in not only the sense that its a fools errand to catch it all, but also that I’m losing data in the cleaning process.

All of these absurdities tell me different things about the data I’m ingesting. In the case of the FEC data it tells me I should give committee context. The SEC data, I realized I needed a lot of XBRL context that can be enriched with the additions of quarters and 10-K filings.

I spent a LOT of time writing my pipelines to ingest datasets.

That is the wrong goal.

Datasets exist in a vacuum. Numbers have meaning. Topology has meaning outside of where it is seen.

I started writing my ingestion pipelines around facts, better yet, questions. The data needed to answer those questions.

Take MDB Capital Holdings, instead of building the database around the data set it exists in, I built out what makes MDB Capital Holdings make sense. What filings make any of this make sense. What contextualizes their absurd reporting to a commission who evidently cares so little that they just lean into the bogus data.

The FEC data surrounding the $100M hotshot: what is this committee, what filings does it appear in? Easier question in this context, but it illustrates a point.

These are multi-domain questions and facts. I could tell you that Winklevoss Captial Management donated $21M over 3 days for a crypto initiative, dwarfing anyone in Delaware. But it is meaningless without the context of motivation and topology. What was the initiative? What was a temporal cousin? Who did they donate to? The best question behind it comes to what payoff that may have come from it. I’m not a rich man but I have a hard time believing anyone just burns that.

With these absurdities, provenance became first class. If I see these numbers, and I report on these numbers, people will point out the absurdity. I cannot afford to not establish provenance. Provenance establishes integrity, design, and reason. It also establishes baselines of where good data actually comes from.

Most important I started with methodology. Build around how I intend to work. I wrote down 10 questions, somewhat open ended and at face value ambitious. Building out schema and configurations around the data to derive facts to answer these questions was far easier than it was to think abstractly. A tangible path forward was birthed from understanding that my system needs to focus on making things interpretable. Giving context over whatever idea of truth I was trying to cultivate.

Once I establish the platform around interpretability, I stopped the vigorous pruning and just took stock of the bush I was looking at.

Bullshit tells a story too, sometimes bullshit makes the smell of truth more noticeable.