I Left an AI Running for 3 Days. It Found 102 Things That Could Destroy Our System.

Last week I stress tested my own system.

Codebot Workflow has two modes. The normal mode handles improvements — dependency updates, performance tweaks, cleanups. But there’s a second mode I built and rarely use: security scan. Full audit. Every vulnerability class, every attack surface.

I pointed it at korz.sh and told it to run until it was done.

Three days later: 102 issues.

But that was just the first pass. When I added judgment-day — a parallel adversarial review mode that runs two independent AI judges on the same code, simultaneously, and only accepts findings both judges agree on — the list got shorter and more serious.

That’s when things got interesting.

What is Codebot Workflow?

For those who haven’t read the earlier post: Codebot Workflow is a multi-agent system I built that runs while I sleep. It analyzes code, identifies issues, and proposes fixes. The agents are Claude-powered through an adapter I wrote.

Think of it as a security team that never gets tired, never misses an edge case, and doesn’t assume the code you wrote is the code you meant to write.

The judgment-day addition

Standard scans are thorough. But they’re also singular — one agent, one perspective, one way of looking at the code. If the agent has a blind spot, you miss whatever’s in it.

judgment-day solves this. It launches two independent judges — Judge A and Judge B — that review the exact same code simultaneously, with no communication between them. A finding only surfaces if both judges independently confirm it. It’s adversarial by design: each judge challenges the other’s assumptions, debates the findings, and won’t sign off unless they’re genuinely confident.

Think of it like having two senior security engineers who don’t know what the other found, running the audit at the same time. You only hear about the things they both agree are real.

The false positive rate drops dramatically. The confidence level goes up. And the findings that survive this process are the ones worth talking about.

The numbers

I was expecting maybe 20.

102 means roughly 1 in 137 lines of code had a critical vulnerability. That’s not noise. That’s a pattern.

Here’s what 102 vulnerabilities looked like across our codebase:

Authentication and authorization failures — 34 findings. These are the worst kind. Not because the code is malicious, but because the system trusts itself too much. Tokens that don’t verify their own signatures. Endpoints that check if a user is logged in but not if they have permission to do what they’re asking. Gaps between services where a compromised internal call looks identical to an authorized one.

Cryptographic failures — 21 findings. Broken signatures, wrong algorithms, keys used in the wrong context. The kind of thing that works in testing and fails in production, but only for an attacker.

Input validation gaps — 28 findings. Data that enters the system without being checked. The system assumes if data came from a trusted source, it must be safe. It isn’t.

Injection vectors — 12 findings. Places where untrusted input could flow into a query, a command, a shell. Some theoretical, some not.

Information disclosure — 7 findings. Error messages that leak stack traces. Responses that reveal more than they should. Logs that contain sensitive data they shouldn’t.

After judgment-day ran its parallel adversarial review, the confirmed list was shorter. But the severity of what survived went up.

What the most serious vulnerabilities had in common

The interesting part isn’t the individual findings. It’s the patterns.

Trusting client-controlled input. The system decided who you were based on values the client sent. The tenant ID came from a header. The user role came from a token the system didn’t properly verify. Once you understood which values the system trusted but shouldn’t, you could rewrite reality.

Security operations that fail silently. When something went wrong in a security check — a token revoke that failed, a hash comparison that errored — the system kept going. No error, no alert, no log. Just quiet acceptance.

Authentication without authorization. The system checked if you had credentials, but not if your credentials allowed what you were trying to do. Any authenticated user could manage any API key. The same token that let you read your own data let you read everyone’s.

Broken multi-tenancy. The architecture supported multiple customers sharing the same infrastructure. But the boundaries between tenants were enforced by parameters in requests, not by hard guarantees. Change the tenant ID, see someone else’s data.

These aren’t exotic vulnerabilities. They’re the same patterns that show up in breach reports year after year.

Two examples anyone can understand

Let me make it concrete.

The Password That Was Never Checked. One endpoint accepted a token and trusted it completely. It checked if the token looked valid, but never verified it was actually issued by us. Anyone could forge a token with any identity they wanted. Create a badge that says “I’m the superuser” and the system says “welcome, superuser.”

The Tenant ID in the URL. The system decided which customer’s data to show based on a value the browser sends. Change ?tenant=acme to ?tenant=competitor and you see the competitor’s data. Not through a bug — through design. The system trusted a value that an attacker controls.

Both fixed. But they were in production for months.

What judgment-day changed

After the initial scan, I ran the findings through judgment-day. Two judges, independent review, only what both confirmed.

The result: a shorter list. Higher confidence. The split judgments — where judges disagreed — were the most interesting part. When Judge A calls something critical that Judge B calls theoretical, it means the severity depends on context you have to understand, not just code you can scan.

This is how security review should work in practice. Not a find-everything scan that buries you in noise. A confidence-ranked output where the top is undeniable and the bottom is “maybe, investigate if you have time.”

Why manual audits miss this

I’ve done security reviews. Most engineers have. You read the code, you look for obvious problems, you check the OWASP top 10, you move on. That’s not enough.

Context fatigue. When you write code, you know what you meant. You read intent into behavior. The code says “decode this token” and your brain fills in “securely.” You don’t see the missing check because you assume it’s there.

Coverage gaps. A manual audit looks at the critical paths. It doesn’t crawl every fallback, every error path, every “this should never happen” branch.

Temporal blindness. Code changes. The validation that was correct in 2024 was silently removed in 2025 during a refactor. Nobody caught it because the security audit was already done.

Agents don’t have context fatigue. They read the code, not the intent. They don’t assume — they verify.

The adversarial setup makes it even harder to miss. When two agents are independently checking each other’s work, the coverage gaps shrink further. Each judge looks at different angles. What one misses, the other finds.

What happened after

I spent two days fixing the critical ones. Codebot had the fixes ready — it generates a patch alongside every finding. But I reviewed each one manually. Some needed adjustment for our specific architecture.

The build passed. The tests passed. We’re safer than we were.

But the question that keeps me up isn’t “how did we miss these?” It’s “how many other teams are sitting on the same patterns?”

Why this matters for every team

Every company with significant code has these issues. Not maybe. Definitely.

The question is whether you’re finding them before someone else does.

A daily automated scan would have caught the JWT bypass in the commit that introduced it. Instead, it sat in production for months.

I now run the vulnerability scan every morning. Most days, the report is empty — the system is clean.

But last week, it found 102 things. And after judgment-day finished arguing with itself, a smaller, more serious list survived.

I’d rather have the report.