Why start from criteria rather than features?
Feature lists grow with every release and every vendor describes the same work in different words. Criteria stay stable. Our scoring method uses seven, with weights that favor the back half of the workflow: discovery and remediation 20% each, attribution and validation 15% each, and integrations, time to value and pricing transparency 10% each. You do not have to use our weights. The score calculator shows how the ranking changes when you move them.
What should each requirement ask?
| Criterion | Requirement | Evidence to ask for |
|---|---|---|
| Discovery | Find internet-facing assets we did not list, including subsidiaries, acquisitions and third-party dependencies, from minimal inputs. | Assets found from the parent company name alone, with the unknown ones marked. |
| Attribution | Show why each asset belongs to us and which entity and owner it belongs to; hold doubtful matches for review. | Evidence behind ten randomly chosen assets; the count of false attributions. |
| Exploitability validation | Confirm safely which exposures can be exploited on our assets. | What each test sends to production; three validated findings our team can check. |
| Remediation | Deliver each finding with an owner, guidance and a ticket; apply or hand off a compensating control; retest the fix. | One finding taken from discovery to closure in our change process. |
| Integrations | Connect natively to our ticketing, SIEM, SOAR, CMDB and edge controls. | A list of native integrations versus API-only ones. |
| Time to value | Produce useful results with little setup. | What the vendor needs from us on day one, and the date of the first report. |
| Pricing transparency | Let us model cost before and after an acquisition. | The pricing basis in writing, and what happens to price when we add an entity. |
How should you weight them?
Weight the criteria by your gap, not by the vendor's pitch. A group that already has a reliable inventory may move weight from discovery to validation. A team with a small change window may weight remediation higher still, because a compensating control saves more time than a better list. A procurement-led evaluation may raise pricing transparency. Write the weights down before the first demo so they do not shift toward whichever vendor presented last.
What makes a requirement testable?
- It names an input: the parent company name, a domain, or a list of seeds.
- It names an output you can count: assets, false attributions, validated findings, days to closure.
- It says who checks the result: your team, not the vendor.
- It has a time box, for example four weeks for the whole proof of concept.
Which requirements do vendors find hardest?
On public documentation, three stand out. Few tools document all of owner, proof, ticket and mitigation together; our findings table shows IONIX and watchTowr do. Few publish prices: in our set only Hadrian lists a per-test price, for NOVA penetration tests, and Microsoft Defender EASM publishes a billable-asset definition with Azure billing. And retesting after a fix is documented by only some tools, including CyCognito and Hadrian.
The next lesson covers what happens after you buy: owners, approvals and service levels.
Terms in this lesson: Proof of concept (POC), Exposure validation, Compensating control, Retest
Sources: CyCognito, Pricing, Hadrian, Pricing, Microsoft Learn, Billable assets, IONIX, WAF Posture Management, watchTowr, Platform