What Six Weeks of Dogfood Taught the Machines
Our health checks found a broken DKIM record, a probe spamming us about itself, and a timer that fails on purpose. We fixed all three the same week — and the interesting part is who found them: not us, the seats that watch while we sleep.
Six weeks ago we opened the beta and pointed our own production at it. Not a pilot program — ourselves. Every alert, every ticket, every build since then has ridden the same stack you get.
The last two weeks are my favorite so far, because of what broke.
A health probe that wouldn't shut up about itself. Our edge watchdog kept filing warnings every five minutes: "gateway health check failing." A red flag — except the gateway was fine. The probe was knocking on a door that requires a key, not realizing it was holding the wrong credentials. The failure was real, just not where it pointed. The fix will make every future probe honest: measure the thing, not your access to the thing.
A DNS record that looked alive and wasn't. Email authentication for our domain was "published" — except the key in it had been pasted through something that shortened it, leaving a literal placeholder where 294 bytes of cryptography should be. It resolved. It verified as present. It would have failed every single signature. Three seats independently caught it from different angles before any mail suffered. The lesson costs nothing to share: existence is not validity. A record that answers is not a record that works.
A timer that fails on purpose. One of our scheduled jobs exits with an error every hour, on schedule. That's not a defect — it's a ratchet. It's counting how much of our work gets recorded properly, and when the number slips below baseline, it screams. Which means the scream is the system working. We're keeping it. We're also moving that whole alerting layer to a place that survives the box it runs on — because a watchdog that dies with the server isn't a watchdog.
Here's the point for you, if you're building with AI tooling or considering it: the interesting failure mode of an AI-native stack isn't the AI being wrong. It's the instrumentation lying to you — green lights on dead switches, records that exist but don't work, probes that report their own confusion as outages. The fix isn't more AI. It's adversarial habits: every seat that claims something gets checked by a different seat. Author never verifies.
That's the stack the open beta runs on — ours. The playground's live, the API keys are real, and the fleet that files these bug reports is the same fleet that ships your assets.
More of these as they happen. The honest ones are better marketing than the polished ones anyway.
— Jake
Open beta: monstergaming.ai/playground — no credit card wall, your key works.
