The Security Gate I Run Before Any AI Code Ships

My gate is three checks that run before any branch of mine merges: a cross-tenant access test against a real database, a review pass that reads nothing except the authorization paths in the diff, and a dependency and secrets scan. Nothing ships to paying users without all three, including code I typed myself. I built it because AI-written code fails in a shape that human code rarely does. It is confident, idiomatic, well named, and wrong about who is allowed to see what.

AI code does not fail randomly, it fails on permissions

I ship SaaS products solo with an AI-native workflow, and I read a lot of generated diffs. The bugs are almost never structural. The agent names things better than I do, handles the loading state I forgot, and writes the error branch I would have skipped at midnight. Then I get to the data access line and find a query that filters by the wrong thing, or filters by nothing at all because the caller was trusted one layer up and the agent could not see that layer.

Brainers Club runs with more than 10,000 members. The Next Level is a white-label fork of the same platform. That means one codebase where every read has to answer a question the code itself does not state: which club is this person in, and is this lesson theirs. An agent reads the schema, sees a club_id column, and writes something plausible. It passes my test suite, because my test suite signs in as one user. The miss only appears when a second tenant exists, which is exactly the condition production always satisfies and local development usually does not. That is not carelessness, it is an agent optimizing for the request in front of it, with no memory of which table would end a client relationship if it leaked. The gap does not close with a better prompt. It closes with a check that runs whether or not anyone remembered to care that day.

The three checks, and when each one runs

The whole gate is wired into CI on pull requests, so it is not a habit I have to maintain. Habits break at 1am when a client is waiting. A required check does not.

• Cross-tenant access tests, against a real Postgres instance with at least two tenants and two roles seeded. Every test asserts that user A gets zero rows from tenant B, not that the request errors. Silent empty results are the correct behavior and the only one an attacker cannot read a map from.
• An authorization-only review pass. I give the agent the diff and one instruction: ignore style, ignore performance, list every line where a reader could reach data belonging to someone else, and say which check stops them. A narrow question gets a usable answer. "Please review this" gets a summary of what the code does.
• A dependency and secrets scan on the diff, because the fastest way an agent breaks a product is adding a package to solve something the standard library already did, or inlining a key into a client component that gets shipped to the browser.
• A manual read of anything touching auth, billing, or row level security. I am the only reviewer on these products, so this is where I spend my attention instead of spreading it over the whole diff.

Why I stopped letting the agent grade its own security

Asking the model that wrote the code whether the code is safe produces agreement, not evidence. The same assumption that created the hole is still in context. So the security pass runs in a separate session that gets the diff and the schema and no memory of the implementation conversation, and the test is what actually decides. Here is the shape every one of those tests takes in my repos.

// runs in CI on every pull request, against a seeded database
test('a member of club A cannot read club B lessons', async () => {
  const client = await signInAs(members.clubA)

  const { data, error } = await client
    .from('lessons')
    .select('id')
    .eq('club_id', clubs.clubB.id)

  expect(error).toBeNull()
  expect(data).toEqual([])
})

Twelve lines, and it is the single highest-value test in a multi-tenant product. It fails loudly the moment a generated query drops a filter, a new policy gets written too wide, or a refactor moves a read from the server to the client. I write one of these per table that holds customer data, before I let an agent near that table. On Arvuyot Yashir, the digital rental guarantee product, that ordering is not optional. A leak there is not a bug report, it is a phone call.

What it costs, honestly

The tests took an afternoon to seed and they run in seconds. The review pass costs one extra agent call per pull request. That is the entire price, and it buys back the thing solo shipping actually lacks, which is a second person who reads the diff with different incentives. I teach this to the 600 students I mentor at nCode and to the business owners who come through the Claude Code Workshop, and the objection is always the same: there is no time to set this up. There is less time to migrate a client off a platform that leaked their members.

Claude Code use among professional developers roughly doubled this year, and most teams added speed without adding a single check. Speed with no gate is just a faster path to the incident. If you ship AI-written code to real users, write the cross-tenant test for your most sensitive table this week, make it required on pull requests, and let everything else stay the way it is.