The demo always goes well. A clean sample estate, three well-behaved seed domains, a dashboard that fills in under a minute, and a count big enough to sound impressive without being frightening. Then you run the same product against your own estate, with the agency-registered campaign domain, the acquisition that still uses its old apex, and the four hostnames that all point at one load balancer, and the picture changes. Useful EASM evaluation criteria are not a list of features from the brochure. They are six tests you run on your own estate, during a proof of value, each with a pass condition you agreed beforehand and a failure mode you are deliberately hunting for.
Short version
- Check one: discovery depth, measured as a two-way diff against your own free discovery from the same seed.
- Check two: attribution, and whether the screen in front of you is a candidate list or a verified inventory.
- Check three: evidence quality, judged against what your change board will actually accept.
- Check four: change detection speed, tested with a planted asset and a deliberate dangling record.
- Check five: noise and deduplication, costed in triage hours per week, not findings per month.
- Check six: pricing units, export rights and exit terms, settled in writing before renewal exists.
Why feature lists make poor EASM evaluation criteria
Every shortlisted product does certificate transparency, passive DNS, port scanning and some form of risk scoring. On a tick list the decks differ mainly in colour. What the criteria have to decide is narrower and harder: does this product shorten the time an exposure sits open on your estate, or does it simply raise the number of things you know about and leave you with a longer queue.
Set the pilot up so it can answer that. Two or three seed domains, chosen to be awkward rather than tidy: your main apex, one acquisition, and one brand you no longer actively market. A fixed window, four to six weeks, long enough for a certificate to be issued and renewed and for a change to land. One named internal owner who runs the same comparison against every vendor, because a pilot compared by three different people is not a comparison.
One rule holds the whole thing together. Every claim the platform makes must come with evidence you can re-check yourself, with the method and the timestamp attached. If you cannot reproduce a finding from the record, you cannot defend it to a change board, and the finding is decoration.
Check one: discovery depth from your own seed domain
Breadth is usually judged on how many assets a vendor returns, which rewards padding. Ask instead which sources the platform queries and through which interface: certificate transparency logs, passive DNS, RDAP (the successor to port 43 WHOIS, specified in RFC 9082 and 9083), ASN and reverse DNS sweeps, search engines, public code and package registries.
Certificate transparency is where most platforms get their volume, so press on the detail. A product reading tiled logs through the Static CT API will see newly issued certificates sooner than one scraping a single aggregator on a schedule. Pre-certificate entries matter too: a name can appear in a log before the service behind it exists, which is a genuine early warning and also a source of hostnames that never resolve.
Then run your own discovery in parallel. Nothing exotic; the free passive methods are enough, and we have written up the approach in our guide to finding forgotten internet-facing assets for free. Diff both ways. Unique finds on their side prove source depth. Unique finds on yours prove a gap. The overlap, which is the part vendors like to show you, tells you nothing about either.
Probe the blind spots on purpose: a wildcard certificate that hides every name behind it, a host that has never appeared in any public log because it was only ever reached by IP, and an asset sitting in a supplier’s cloud account under their registrant details.
Pass condition: the platform returns at least one asset you did not have, with the source named, and the recency gap between certificate issuance and inventory appearance is measured in hours. Failure mode: shallow discovery dressed up as a large number, where their inventory is mostly your seed list expanded by brute force.
Check two: attribution, or how candidates become a verified inventory
A hostname found in a log is a candidate, not an asset. Ask directly which of the two you are looking at, and how the product marks the difference on screen. Many do not, and the distinction quietly becomes your problem at triage time. The mechanics of this are covered in our walkthrough of how discovery moves from seed to inventory.
The attribution checks that carry weight are registration data, ASN and IP ownership, the resolution path, hosting fingerprints, and corroboration across two independent sources. One source is a guess with a timestamp.
False attribution hurts in both directions and the directions are not symmetrical. Someone else’s host inside your inventory burns triage hours and, if anyone acts on it, raises a legal question you do not want. Your host sitting outside the inventory stays unmonitored, which is the whole reason you are buying.
Pass condition: confidence and supporting evidence are recorded per asset, with first seen and last seen dates, and all of it exports. Failure mode: candidates passed off as inventory, with the verification work silently transferred to your analysts.
Check three: the evidence a change board will accept
This is where most pilots quietly fail, because the finding looks fine until someone has to raise a change against it. Passive banner collection and verified reachability are different claims, and a platform that blurs them will eventually cost you credibility with the infrastructure team.
The classic case is a PHP or OpenSSH version string. Long-term support distributions back-port security fixes without changing the advertised version, so a host can report a branch that looks end of life while the specific vulnerable code has already been patched. Raise three of those and the change board starts discounting everything you send. Our comparison of likely versus definitely sets out where each method stops.
Fronted and virtual-hosted services create the same problem with ownership rather than versions. A header returned by a CDN edge, a shared host answering for a tenant who is not you, a default certificate on a multi-tenant platform: all of them can be filed against your asset if the product is not reading carefully.
A finding record needs four things before anyone raises a change: the observation, the method used to obtain it, the timestamp, and a way for someone else to reproduce it. Read the vendor’s severity language in the same spirit. “Potentially exploitable”is an inference. “Reachable on TCP 443 from an unauthenticated source at 09:14 on 4 March”is an observation.
Pass condition: every finding states which method produced it, and version inference is visibly separated from verified reachability. Failure mode: banner-only evidence that your own engineers will reject.
Check four: how fast the platform notices a change
Plant something. Register a subdomain you control, issue a certificate for it, and stand up a service on an unusual port. Record the hours or days until each shortlisted platform reports it. Then do the harder half: point a CNAME at a storage bucket, delete the bucket, and leave the record in place.
Dangling DNS is the clearest differentiator in a pilot, and it separates products faster than any feature matrix. A weak platform sees the host stop responding and drops it from the inventory, which is exactly backwards. A good one reads the resolution chain, notices the record now points at infrastructure nobody has claimed, and raises it as a takeover candidate. The mechanics are set out in our piece on the dangling DNS problem.
Ask which events override the scan calendar: a new certificate in a log, a DNS edit, a new host appearing inside your ASN, a decommission ticket from your own change system. A calendar alone means you find things on the vendor’s schedule rather than the attacker’s.
Pass condition: the canary name appears within a day, the orphaned record is flagged rather than deleted, and you control which asset tiers are checked daily versus quarterly. Failure mode: change detection that lags decommission by weeks, which is the window takeover lives in.
Check five: noise, deduplication and the weekly triage load
One service behind four names should arrive as one asset with three aliases. Plenty of products deliver four findings, and the inventory count looks healthier for it. Ask how canonical identity is decided and whether you can merge records yourself.
Then count the hours. Have the pilot owner log triage time weekly, in plain hours, and multiply across the full estate before you compare licence quotes. A product that costs less on the licence line and consumes extra analyst days every week is not cheaper. That arithmetic is the one most shortlists never do.
Prioritisation needs to be defensible on its own terms: ownership, reachability and business criticality, not a raw CVSS sort that puts an unauthenticated admin panel below a theoretical issue on a parked domain. Test it with awkward cases. Exposed telnet on port 23, or an internet-facing HMI or remote access gateway, should be recognised for what it is rather than filed as a generic open port; our guide to exposed telnet servers covers why that distinction matters operationally.
Pass condition: reporting measures exposure time from detection to confirmed closure, per asset tier. Failure mode: finding volume as the headline metric, which flatters the noisiest product on the list. A pilot that raises 400 findings and shortens nothing has failed.
Check six: pricing units, exit terms and the full cost line
Define what counts as an asset before anyone quotes a number. Domains, hosts, IP addresses, certificates, cloud resources: each unit produces a different bill from the same estate. If the unit is the host and discovery is working properly, your inventory grows during the term by design. You are paying more because the product is doing its job, which is a strange incentive to sign up to without a cap or a band.
Settle in writing what each of these counts as: a wildcard certificate, a CDN-fronted name, a short-lived container host that lives for nine minutes, and a decommissioned record you have chosen to keep monitoring. Do it now, not at renewal.
Four cost lines, compared side by side: licence, onboarding and tuning in the first quarter, triage hours per week as measured during the pilot, and remediation effort. Ask who does the tuning, vendor, partner or your own team, and whether that resource is contracted or assumed.
On portability, ask for a full export of inventory and findings, API access, and a clear statement of what you keep if you leave. Seven fields decide whether a record survives an audit: canonical identity and aliases, resolution path, named owner with date, first seen and last seen, confidence, evidence, and disposition with review date. Ask the vendor to export all seven during the pilot rather than promising it.
Regulation sets a floor here, not a design. The Cyber Security and Resilience Bill, before Parliament at the time of writing, proposes a two-stage duty: an initial notification within 24 hours and a fuller report within 72. Meeting a 24-hour clock is a question about whether you can pull owner, first seen and evidence out of your inventory at speed. Compliance frequency is not a monitoring strategy.
Scoring the pilot
Write the pass conditions down before the first vendor connects to anything, score each check as pass, partial or fail against your own estate, and keep the diff files. Six checks, one page, no weighted matrix. Sound EASM evaluation criteria give you something to point at in a year when the inventory has doubled and someone asks why you chose this one; for the longer buyer’s view, our practical checklist for choosing an EASM product is the next thing worth reading.
Frequently Asked Questions
What should EASM evaluation criteria cover at minimum?
Discovery depth from your own seeds, attribution quality, evidence standard, change detection speed, noise and triage load, and commercial terms including asset definition and export. Each needs a pass condition agreed in advance and tested against your estate rather than the vendor’s sample. Anything else is a feature list.
How long should an EASM proof of value run before you can judge it?
Four to six weeks is usually enough, because you need a certificate issuance, a DNS change and a decommission to happen inside the window. Shorter than a month and you are judging the onboarding import, not the monitoring. Run every shortlisted product over the same seeds in the same period if you can.
How do you tell a candidate asset list from a verified inventory?
Look for confidence and evidence recorded per asset, with at least two independent attribution sources and first seen and last seen dates. If the interface shows no distinction between a name pulled from a certificate log and a host confirmed to resolve into your address space, you are looking at candidates. The verification work has been handed to you without being priced.
What counts as an asset for EASM pricing, and why does it matter?
It varies by vendor: domains, hosts, IPs, certificates or cloud resources. The unit matters because working discovery grows the count during the term, so the bill rises as the product succeeds. Agree in writing how wildcards, fronted names, ephemeral container hosts and decommissioned records are counted before you sign.
Can existing vulnerability scanning cover the same criteria?
Not the discovery and attribution half. A scanner tests what you point it at, so it is only as complete as the target list someone maintained by hand. The two are complementary, and our comparison of where each one stops sets out the boundary in detail.
