Security software is not one thing
Most software categories are legible. A project tool either holds your tasks in a way your team tolerates or it doesn't. An invoicing platform either produces a compliant invoice or it doesn't. You can open the product, use it for a week, and form a defensible opinion about whether it works.
Cybersecurity does not behave like that. A security product's central promise is about events that have not happened yet — an intrusion prevented, a credential not stolen, an exposed service found before somebody else found it. When the product is working, the observable outcome is silence. Silence is the same thing you get from a product that does nothing at all.
The category is also not a category. "Cybersecurity" is used commercially to describe endpoint protection, vulnerability scanning, penetration testing services, external attack surface management, bug bounty coordination, identity and access management, encrypted storage, security awareness training and compliance evidence collection. These solve different problems, are bought by different people, and fail in different ways. A page that ranks them against each other with a single score is not a review — it's a taxonomy error with a buy button attached.
So the first thing we do before writing anything is decide what the product actually is: what job it performs, who inside a business owns that job, and which products are genuine substitutes rather than adjacent purchases. Nearly every misleading security review we've read gets this wrong before it gets anything else wrong.
We separate claims from evidence
Security vendors publish confident language because their buyers are frightened. "Enterprise-grade." "Military-grade encryption." "AI-powered detection." "Zero-day protection." Some of these phrases describe something real. Most describe a marketing position built on top of something real, and a few describe nothing at all.
Our standard is simple and unglamorous: a claim in a TTML review must be attributable. When we say a platform performs authenticated web application scanning, that is because the vendor's own documentation says so and specifies what "authenticated" means in their implementation. When we say a service includes human penetration testers rather than automated scanning with a report template, that is because the scope document says so.
A claim we cannot attribute is not a finding. It is an advertisement we would be republishing for free.
This is why our security pages read more cautiously than the vendor's own site. We describe capability at the level the evidence supports, and we name the source class of that evidence on the page. Readers should always be able to see the difference between something we verified, something the vendor asserts, and something the market broadly believes.
We don't pretend that research is testing
The most common dishonesty in security reviewing is a quiet one. A page implies hands-on evaluation it never performed. Sometimes it's a stock screenshot. Sometimes it's a sentence beginning "in our testing." Sometimes it's an eight-point score in a category where meaningful independent testing would require an attack lab, a research budget and months of work.
We publish two clearly separated classes of security page. A vendor-documentation review is a structured, sourced analysis of what a product claims, how it is priced, how it is delivered, what it demonstrably includes and who it appears to be built for. A hands-on review is what we publish only when we have genuinely operated the product, and it says what we did.
Both are legitimate editorial work. Only one of them is testing. Conflating them would make our pages more persuasive and less true, which is a trade we are not willing to make. Where a review is documentation-based, that classification appears on the page rather than in a footnote nobody reads.
There is a related discipline here that matters more in security than anywhere else: we do not emit review ratings on pages that have not earned them. A number implies measurement. In a category where the measurement is genuinely hard, an invented number is worse than no number.
We distinguish absence of evidence from evidence of absence
If a vendor does not publish its SOC 2 status, that does not mean the vendor lacks one. If a platform's documentation never mentions single sign-on, it may still ship it and simply document it poorly. Security vendors are frequently uneven documenters, and the smaller ones are the worst offenders precisely because they are engineering-led.
So we write two different sentences, and we never let them collapse into one:
- "The vendor does not publish X." — a statement about the available evidence.
- "The product does not do X." — a statement about the product, which requires stronger grounds.
This sounds pedantic until you notice how much competitive security content is built on the first sentence being silently promoted to the second. It is the easiest way to damage a smaller vendor while appearing rigorous, and it is a failure of method rather than a finding.
When something material is genuinely unclear — scope boundaries, retest policy, what counts as an "asset" for billing — we say it is unclear and name the question a buyer should ask. Unresolved uncertainty belongs in the review, not in the omissions.
Pricing receives special treatment
Security pricing is where the gap between the published number and the invoice is widest. A scanning platform priced per target sounds cheap until you count subdomains. A penetration testing service priced per engagement sounds expensive until you learn a retest is included and your compliance auditor accepts the report. Contract minimums, asset definitions, seat versus endpoint counting, remediation retests, and whether the report is written for engineers or for auditors all move the real cost more than the headline figure does.
We therefore treat pricing as a structural property of the product rather than a fact to copy. We record the model, the unit, what the unit actually counts, what is excluded, and what triggers a quote. Where a vendor publishes nothing, we say so plainly instead of estimating a number that would read as authoritative and be wrong.
Pricing is also the fastest-decaying claim on any security page, which is one of the reasons our pages carry a last-reviewed date. A price without a date is not information.
Compliance requires particularly careful language
A great deal of security software is bought for one reason: somebody needs evidence for an auditor, a customer security questionnaire or an insurer. That makes compliance language commercially valuable and therefore routinely overstated — by vendors, and by the publications that summarise them.
No tool makes a business SOC 2 compliant. No scanner satisfies ISO 27001 on its own. What a product can legitimately do is produce artefacts that support a control: a scan cadence, a remediation trail, an attestation letter, a penetration test report in a form auditors accept. That is a real and useful thing to sell, and it is not the same thing as compliance.
So we describe the artefact and the control it supports, not the certification. "Produces a report commonly accepted as evidence for the annual penetration testing expectation" is longer than "SOC 2 compliant." It is also the only version of that sentence a reader can act on without being misled.
Then we ask who should actually buy it
The most useful judgement in a security review is almost never "is this good." It is "is this the right shape of thing for a business like yours, run by a team like yours, at the stage you are at."
A continuous scanning platform is excellent for a twelve-person SaaS company with an engineering-owned security function and no security hire. The same platform is a poor fit for a regulated enterprise that needs adversarial testing by named humans against a defined scope. Neither product is worse than the other. They are answers to different questions, and pretending otherwise is how buyers end up with tooling they cannot operate.
Every TTML security review therefore carries an explicit "best for" and an explicit "look elsewhere." The second one is the harder to write and the more valuable to read. A review that cannot name anybody it doesn't suit has not made a judgement — it has made a summary.
Affiliate relationships come afterwards
The Tool Money Lab is funded partly by affiliate relationships, and we disclose that on every page that carries one. The relevant question is not whether commercial links exist — it is what order the decisions are made in.
Our editorial registry and our affiliate registry are deliberately separate systems. Coverage is decided editorially: what buyers are trying to compare, where our category is thin, which products are genuine substitutes. Whether a tracked link exists is resolved afterwards, from a different registry, at render time. A product with no programme is reviewed the same way as one with a generous programme, and the second one gets no ranking advantage for existing.
Independence isn't a promise on an about page. It's an architecture decision you can inspect.
We also keep commercial furniture out of editorial essays like this one. This page has no call to action, no rating and nothing to click through to a merchant, because it isn't selling anything. That is a boundary worth defending in a category where fear converts unusually well.
Comparisons are where bad evidence multiplies
A weak review misleads one reader about one product. A weak comparison misleads them about two, and does it in a table — the most authoritative-looking format on the internet. A cell containing a tick mark asserts equivalence between two features that may share a name and nothing else.
Our comparisons are generated from the same underlying review records rather than written freehand, which has one important consequence: a comparison cannot make a claim that isn't already sourced in a review. When we don't hold comparable evidence for both sides of a row, the row says so instead of guessing.
We also try to compare things that a buyer would genuinely hold in tension — a PTaaS platform against another PTaaS platform, a bug bounty programme against an engagement-based test — rather than manufacturing a matchup because the search term exists.
Our methodology is designed to change
A methodology that never changes is either perfect or unexamined. Ours has changed several times: we added explicit evidence classes to individual claims, we stopped emitting ratings on documentation-based pages, we separated compliance artefacts from compliance status, and we tightened the language we allow around detection claims.
Because our pages are generated from typed records rather than hand-authored one at a time, a methodology improvement propagates. When we tighten a standard, it lands across the catalogue rather than on whichever pages somebody remembers to revisit. That is the main practical argument for treating a publication as a system.
It also means we can be held to it. Every security page shows when it was last reviewed, what class of evidence it rests on, and which sources were consulted. If we've got something wrong, that is enough information for a reader — or a vendor — to say so precisely.
The point of all this
Security buyers are not usually looking for a winner. They are looking for a decision they can defend six months from now, to a colleague, an auditor or themselves. That decision is easier to make from an honest, narrower page than from a confident, broader one.
So we would rather publish a review that says "this is documentation-based, here is what the vendor commits to, here is the pricing model, here is who it suits and here is who it doesn't" than one that performs certainty we haven't earned. In this category particularly, the useful thing a publication can offer is not a verdict. It is a clear account of what is known, what is claimed, and where the difference lies.
That is the standard we hold our cybersecurity coverage to, and it is the standard we think readers should hold every security publication to — including the ones that agree with us.