Method

Why VibeLock will never give you a security score

Reproducibility is not a nice property of a security claim. It is the whole product, and it is what rules out every number we could have invented.

VibeLock7 min read

The obvious way to build a security rating in 2026 is to hand a model your project and ask it for a number out of a hundred. It takes an afternoon. It demos well. It is also the one design that cannot do the job the number exists to do.

The job is narrow, and it is not “assess security”. Plenty of tools assess security. The job is to give a founder something they can put in front of an investor, an enterprise buyer or a procurement questionnaire, and have it hold up when that person pushes back on it. Everything below comes from taking that one sentence seriously for longer than is comfortable.

The question that breaks a model generated score

Imagine you send a security number to a prospective customer’s reviewer and they reply with four words: “how did you get 82?”

If a model produced it, you cannot answer. You can describe the inputs it saw, but you cannot show the step from those inputs to 82, because there was no step. You cannot show that the same project would produce 82 again tomorrow, because it might not. You cannot show which of your open items cost you the eighteen points, so you cannot tell them what you are doing about it.

Worse, the reviewer can run the experiment themselves. They change nothing, ask again, and get 79. At that point the number is not evidence of anything, and neither is the rest of your submission, because you offered it as evidence and it did not hold.

Computing it yourself does not save it

The apparent fix is to compute the number with a pure function over evidence the product observed, publish the function, and let anyone recompute it. Same inputs, same output, every time, on any machine, with no network call. When a reviewer asks how you got 82, you hand them the arithmetic.

That answer holds for about one question. The next one is the one that matters: you chose those weights. Why does an open critical cost twenty points and a medium five? Reproducing an arbitrary judgement is not the same as defending it. You have made the judgement repeatable, not sourceable, and a reviewer who wants to argue has nothing to argue with except your authority.

Reproducing an arbitrary judgement is not the same as defending it. A weight nobody outside the company can check is a claim resting on our authority, which is the one thing this product cannot afford.

A weight is a liability with no upside

Weights also carry less than the argument about them suggests. In any scheme that blends findings into one number, the number is dominated by a few facts: whether anything critical is open, whether anything is overdue, and how much of the app was tested at all. Nudge one weight and the number barely moves. Leave it alone and you still have to defend it.

A parameter that must be defended, cannot be sourced, and does not change the answer is not a feature. It is a liability with no upside, and the right thing to do with it is to not have one. Show the few facts that dominated the number instead, and the number has nothing left to add.

What the product reports instead

The checks cite the published requirements behind them, written by somebody else: OWASP ASVS 5.0 requirements with their own identifiers and levels, OWASP AISVS 1.0 requirements, articles of the GDPR, the EU AI Act and the EU ePrivacy Directive, and entries in the OWASP LLM Top 10 2026 and the OWASP Top 10 for Agentic Applications 2026. A control that satisfies no published requirement is not in the catalogue, however good a practice it might be.

In place of a number, VibeLock shows facts, each of which can be checked on its own: which of the five badge criteria hold today, how much of the app was actually tested, the open findings by severity with their deadlines, and the date of the last verified scan. Nothing is weighted, and nothing adds up into a grade.

VibeLock does set a severity for every finding, Critical, High, Medium or Low, and it is honest about why: the severity sets one thing, the deadline. Critical gets 7 days, high 30, medium 90, low none. That table is published, the same for every app, and a finding’s severity is a statement about urgency, never a deduction from a score.

The reviewer’s question now has a better answer than arithmetic. They can obtain the standard, read requirement V8.2.2, read the controls we map to it, and tell us the mapping is wrong. They can read the deadline table and disagree with it. That is a real conversation. “Why 0.30 and not 0.22” never was.

A badge, and no launch verdict

The same argument rules out the verdict everyone expects next: your application is ready to launch. It looks harmless, because it sounds like a summary of the facts above.

It is not. Declaring the moment an application may be exposed to real users is an authorial act with far more consequence than any deadline. Nothing we can observe supports it. We see the application and its code. We do not see the users, the data, the threat environment or the obligations, and readiness is a function of all four.

What VibeLock publishes instead is the VibeLock Verified badge, and it says something narrower: five stated criteria held on a stated date. Coverage of at least 80 percent, no open critical findings, no high finding past its deadline, a verified scan in the last 30 days, and an external scan plus a code connection. When one stops holding, nothing turns red and nothing is taken down: the badge shows the date the criteria last held. Those are the only two public states, Verified or Last verified on a date, and the live verification URL behind the badge is the proof, not a screenshot.

There is a structural tell that separates the two. Every criterion is falsifiable: open the verification URL, read the date, check the open findings. A readiness verdict is not, because no external document defines it. It would be the one claim in the product that could only be accepted on our authority.

Where a model does help

None of this is an argument that models are useless here. Drafting a policy from what a scan verified, explaining what a finding means to someone who cannot read a CVE report, writing the prompt that tells a coding tool how to fix one specific failing control: these are language problems, and models are good at them.

The line is between helping a human act on evidence and grading a human. VibeLock does the first everywhere and the second nowhere. Model-written text is explanation and instruction, attached to a single named finding, and it never counts as proof on its own: a fix is fixed when a rescan confirms it.

What that leaves is smaller than a number out of a hundred. It is also the only version that survives the meeting it was made for.