Building the Mine
Two weeks, 1,784 commits, zero trades: building the mine
The first two weeks of a microcap research system: almost none of it is about picking stocks, and one timestamp problem could have faked every result.
Educational content, not investment advice. Disclaimer.
I have written 11,823 automated tests for a trading system that has never made a trade. That’s on purpose, and this post explains why.
I started building on September 20. Fifteen days later the project has 1,784 commits, 79 database migrations, 49 written-down design decisions, and exactly zero dollars of profit. If you’d asked me on day one how much of this would be about finding good stocks, I’d have guessed half. The real answer is closer to none.
Source: git log of the research system
Show the data
| Commits | |
|---|---|
| Sep 20 | 42 |
| Sep 21 | 113 |
| Sep 22 | 181 |
| Sep 23 | 110 |
| Sep 24 | 72 |
| Sep 25 | 10 |
| Sep 26 | 58 |
| Sep 27 | 128 |
| Sep 28 | 179 |
| Sep 29 | 166 |
| Sep 30 | 172 |
| Oct 1 | 240 |
| Oct 2 | 262 |
| Oct 3 | 50 |
| Oct 4 | 1 |
A quick note on how I build: I lean on modern tooling, including AI coding assistants, for a lot of the typing. Every design decision, and every mistake in this post, is mine.
What I’m building, in plain English
Think of it as a mine with levels. You can’t dig the lower ones until the upper ones hold weight.
- Level 1Collect BuiltDownload public company information from free official sources, on a schedule, without getting blocked.
- Level 2Remember BuiltStore every document and number exactly as it looked when it became public. Never overwrite anything.
- Level 3Analyze BuiltTurn raw documents into structured facts that can be compared across thousands of companies.
- Level 4Backtest In progressReplay history honestly and ask whether any of it would have made money after costs.
- Level 5Paper trade Not yetRun the whole thing live, with fake money, and compare it to the backtest.
- Level 6Real money Not yetThe $50,000 goes in.
The rule I set on day one: the first three levels have no trading code at all. Nothing that can place an order exists until the backtest gives it a reason to.
Level one: collecting, politely
Everything in the system comes from free, public sources, mostly official ones. That’s a deliberate constraint. Paid data is better in a lot of ways, but it costs money I’d rather keep in the stake, and if the strategy only works with a $20,000-a-year data feed, I want to find that out before I pay for one.
Free has its own costs. The SEC’s EDGAR system, which processes around 4,700 filings a day, asks automated users to stay under 10 requests per second and to identify themselves.1 So the system has a traffic controller that keeps every request under that limit, retries failures with patience, and puts urgent work (things published in the last few minutes) ahead of background work (downloading two years of history).
That last part caused one of my first real bugs. The big historical download was so large that it starved the live updates. New information was arriving and sitting in a queue behind thousands of years-old documents. The fix was a scheduler that guarantees fresh work a share of the capacity no matter how much backlog exists. It’s boring, and it’s exactly the kind of bug that would have cost real money if I’d found it with a live account.
Level two: the timestamp problem
If you read the last post, you know my biggest fear is a backtest that accidentally knows the future. The front line of that fight turned out to be a deceptively simple question: when, exactly, did this piece of information become public?
You’d think every document comes with an answer. It does. Sometimes it’s even correct.
Here’s what I found. Filings carry an acceptance time, the moment the SEC’s system accepted them. Some sources report that time with a marker claiming it’s in UTC (universal time). In the cases I checked, it wasn’t. It was New York time with the wrong label attached. Trust the label, and every filing looks like it arrived four or five hours earlier than it really did. A backtest built on that would have “traded” on news before anyone could have seen it, and it would have looked brilliant.
Then there’s the after-hours problem. Something accepted at 9:45 p.m. isn’t really in front of the market until the next morning. So the system computes a conservative “public as of” time for every document, using the most trustworthy source it can find. When the sources disagree, it picks the later time. When it can’t verify a time at all, it records the uncertainty instead of guessing.
None of this will ever appear in a chart of my returns. All of it determines whether those returns are real.
Level two, continued: never overwrite anything
The other design rule for this level is that nothing gets edited in place. Original documents are stored exactly as downloaded. If a company corrects a number, the correction is stored next to the original, with its own timestamp. Every derived number knows which inputs, which version of the code and which settings produced it.
It sounds like overkill for a personal project. But it means that a year from now, when a backtest says something surprising, I can replay exactly what the system knew on any given day and check whether it was cheating. That’s worth the disk space.
I also tested backups by actually restoring them into a fresh database and checking the result, because I’ve been an engineer long enough to know that an untested backup is a hope, not a backup.
Level three: analysis (the part I’m keeping vague)
The third level turns raw documents into structured facts that a computer can compare across thousands of companies. This is the part closest to the actual edge, so it’s where I’ll stay deliberately fuzzy. I’ll share principles, not recipes. The main one: anything that interprets messy text is allowed to suggest, but simple, deterministic, version-controlled rules make every final decision. If I can’t explain why the system scored something the way it did, it doesn’t get to trade on it.
What I got wrong
- I underestimated the data work by a factor of ten. I thought collecting and cleaning would take a few days. It took most of two weeks, and it’s still not done.
- I built the scheduler too late. The starvation bug was predictable. I just didn’t predict it.
- I should have started writing decisions down on day one. Now every meaningful choice gets a short written record: what I decided, what I rejected, and why. There are 49 of them so far.
What happens next
Level four: the backtesting engine. Its job is to replay years of history through everything above, with realistic costs, delays and the dead stocks included, and produce an honest answer to one question: would this have made money?
I’m building it now. When it answers, I’ll publish the result, with the method, though not the recipe. If it says no, I’ll publish that instead, and then figure out what to dig into next.
Footnotes
-
SEC, “About EDGAR” (2024), and “Accessing EDGAR Data,” which sets a maximum of 10 requests per second and requires automated tools to declare who they are. About EDGAR, Accessing EDGAR Data. ↩