Certificate Transparency Logs: Subdomain Discovery Gaps

Certificate Transparency Logs: Subdomain Discovery Gaps

Certificate transparency logs subdomain discovery is usually taught as a recon trick: query crt.sh, collect the output, feel productive. From the defender’s chair the useful point is a different one. What comes back from a CT query is a list of candidates, not a list of assets, and the gap between those two things is where most estate inventories go wrong.

CT logs are genuinely one of the best passive data sources you have. They are also incomplete in ways that are predictable, and inflated in ways that make raw counts close to meaningless. Both problems are fixable if you build a workflow rather than a one-off query.

What certificate transparency logs actually record about your domains

Certificate transparency is an append-only, publicly auditable record of certificates issued by publicly trusted certificate authorities. The mechanism was specified in RFC 6962 and has since been extended and replaced in parts. Logs are operated by a handful of organisations, including Google, Cloudflare, Let’s Encrypt and DigiCert. When a CA submits a certificate to a log, it receives a signed certificate timestamp, a cryptographic receipt proving the submission happened.

The reason your certificates are in there is browser policy, not law. Chrome has required SCTs for publicly trusted certificates issued since 30 April 2018, and Safari has a comparable requirement. A certificate without them produces a trust error, so CAs log by default.

The fields that matter for discovery

Five fields do nearly all the work: the common name, the subject alternative names, the issuer, the validity window and the serial number. The SAN list is where forgotten hostnames surface, because a single certificate issued for a load balancer or a shared ingress may carry twenty names, several of which nobody in your team has thought about since the project shipped. The issuer tells you which CA was used, which is often the first hint that a name was not provisioned by your platform team at all. The validity window tells you whether the name is still being actively renewed or was abandoned three years ago.

Why one issuance produces more than one entry

CAs typically log a pre-certificate first: an object identical to the final certificate but carrying a poison extension so it cannot be used. The final leaf certificate is then logged as well, often to more than one log. One issuance, two or more entries. Add automated renewal on a 90-day cycle and a single hostname can appear dozens of times in a year, each time looking like a fresh discovery to a script that counts records.

None of this touches the asset. You are reading a public log that a third party published about your certificates. No packets reach your servers, nothing is probed, and the same data is equally available to everyone else. That is the trade: CT is free, passive and safe, and it is also read by attackers as routine practice.

From crt.sh output to a de-duplicated hostname list

A workable pipeline has four stages: query, de-duplicate, resolve, confirm ownership. Most teams stop after the first one.

Query more than one source

The wildcard-style lookup is the starting point. On crt.sh, which Sectigo operates, that means a query in the form %.example.co.uk with JSON output, which returns every logged certificate whose names fall under the domain. Pull the same domain from at least one other source as well, whether that is a CT search API such as SSLMate’s Cert Spotter, an internet scan dataset, or your own poller against the logs directly. Crt.sh is a single aggregator with rate limits and occasional outages, and a coverage gap caused by a 502 looks identical to a clean estate.

De-duplicate on the SAN set, not the certificate

Extract the names field, split it on newlines, lowercase everything, strip leading *. prefixes, drop the email addresses that occasionally appear, then sort for unique values. Several hundred certificate records commonly collapse into a few dozen distinct hostnames. Report the de-duplicated figure. A report that says “412 certificates discovered”has described renewal churn and told the reader nothing about the size of the estate.

Keep the certificate-level data, though. Which issuer, which validity window and which SAN groupings a name appears in is exactly the evidence you need later when you are trying to work out who provisioned it.

Tiled logs and the Static CT API

Log infrastructure is changing under you. Operators have been moving towards the Static CT API, which serves logs as tiled static files that can sit behind a CDN, rather than the dynamic get-entries and get-sth endpoints of the original protocol. If you maintain your own poller, anything written against the legacy endpoints needs revisiting rather than assuming it will keep working as older logs are retired. Check the current Chrome CT log list before you rely on any specific log being alive.

Certificate lifetimes are shrinking too. Under ballot SC-081, agreed by the CA/Browser Forum in 2025, the maximum lifetime for publicly trusted TLS certificates steps down in stages towards 47 days by 2029, with the first reduction due in 2026. For discovery this means more renewal noise per hostname per year, and monitoring rules that suppress duplicates become mandatory rather than nice to have.

Historical coverage is uneven

Logs get retired, and some historical entries are only reachable through aggregators that ingested them while they were live. More importantly, certificates issued before CT enforcement may never have been logged at all. A legacy service running a certificate first issued in 2016 and renewed inside a private PKI can be completely absent from CT while sitting happily on a public IP address. Age is a blind spot, not just a data-quality quirk.

Six blind spots subdomain discovery from CT logs will never cover

  1. Wildcard certificates. This is the big one. A single *.example.co.uk certificate publishes no hostnames whatsoever. An estate standardised on wildcards for convenience can look reassuringly small in CT while dozens of live services sit behind that one entry, unnamed and unlisted. If your CT results are suspiciously tidy, check how many wildcards you issue before you congratulate anyone.
  2. Private and internal CAs. Certificates from an internal CA, a Kubernetes-internal issuer or a self-signed pair are not publicly trusted and are not logged. Plenty of internet-reachable management interfaces run exactly that.
  3. Anything without TLS. Plain HTTP services, exposed databases, RDP, SSH and vendor management ports never generate a certificate in the first place. Nor do services reachable only on a bare IP address, which have no hostname to publish.
  4. Assets covered by somebody else’s certificate. A SaaS tenant served under a provider’s own domain, a default platform hostname, or a site fronted by a CDN’s shared certificate will surface the provider’s names, not yours. The asset is yours; the certificate record is not.
  5. Domains nobody has mapped to you. Campaign domains, regional brands and the infrastructure of an acquired company are invisible until somebody adds them to your seed list. CT answers the question you ask it, and only that question, which is why it has to sit alongside DNS enumeration and registration data.
  6. The pre-enforcement era. Names carrying certificates issued and renewed outside the CT ecosystem, or issued before 30 April 2018 and never replaced, can be entirely missing.

Take those six together and the conclusion is straightforward: CT gives you a strong, cheap, high-signal slice of your external estate, and treating that slice as the whole is one of the more common errors made in external attack surface work. Pair it with DNS record enumeration, registration data and reverse lookups on the IP ranges you own before you make any statement about coverage.

Candidates, not certainties: triaging hits into inventory

A certificate in a log proves that somebody asked a CA for a name and passed a validation check. It does not prove a host was ever deployed, that it is live now, or that the thing behind it belongs to you.

Two gates before anything enters inventory. First, resolve the name: NXDOMAIN results, parked pages and staging names that were requested and never deployed are extremely common. Second, check what responds: a status code, a banner, a certificate served on the live connection you can compare against the logged one. Only then does the candidate become an asset and get an owner.

The valuable exceptions are the names that fail those checks in a particular way. Consider old-app.example.co.uk with a CNAME still pointing at a cloud storage bucket that was deleted eighteen months ago. CT shows the name existed and was certificated. DNS shows the record survives. Together those two facts describe a subdomain takeover waiting for someone to claim the resource, and the fix is deleting the DNS record, not reissuing a certificate. A CNAME pointing to a deleted bucket is the single highest-value finding a CT-plus-DNS workflow produces, and it is invisible to either data source alone. Working out who was responsible for that bucket in the first place is its own discipline, covered in the piece on third-party subdomain ownership.

When a candidate is confirmed, record more than the hostname. Resolved addresses, hosting provider, the certificate issuer and expiry, the business owner, and the technical owner responsible for renewal. The fields worth capturing in an internet-facing asset inventory are what turn a hostname list into something you can act on during a patch wave. The NCSC’s own asset management guidance makes the same point in blunter terms: you cannot secure an asset you do not know you own.

Running CT as a continuous early warning feed

Point-in-time enumeration ages badly. A developer deploys a new service. An agency stands up a campaign site. A certificate is issued for a name nobody registered internally. Run CT as a change-detection feed instead and it becomes one of the fastest shadow IT alarms available.

The pattern to watch for is a certificate whose name appears in no internal register. A marketing agency issues a certificate for campaign.brand.co.uk on its own hosting, and the name lands in a public log within hours and in your CMDB never. That is not a breach. It is a team moving faster than procurement, and you want to know about it on day one rather than at the point the campaign ends and the DNS record outlives the hosting contract.

ACME automation on internal tooling has a related side effect: it quietly publishes your naming conventions. Hostnames like jenkins, uat, vpn-test and sso-staging enter a public log the moment a publicly trusted certificate is issued for them. Useful to you when auditing your own estate. Equally useful to anyone else reading the same logs, which is an argument for internal PKI or wildcards on genuinely internal tooling, not an argument against CT.

On cadence, a daily poll for new names is proportionate for most mid-sized estates, with a weekly pass over expiring certificates. Noise control is the part people underestimate. Suppress any issuance where the SAN set and issuer are unchanged from the previous certificate, group findings by parent domain so one wildcard-fronted platform does not generate forty tickets, and route certificate hygiene items to a normal work queue rather than the incident channel. Expiry warnings that page an analyst at 2am train people to ignore the feed. The same logic that drives continuous discovery of the assets you have forgotten applies here: the value is in the delta, not the dump.

A secondary benefit worth enabling: monitoring issuance for lookalike and typosquat names against your brand. Expect a high false positive rate, since plenty of unrelated organisations legitimately hold similar strings, so treat it as an intelligence feed for the brand protection or fraud team rather than a security alert.

Where CT-based discovery fits alongside scanning and internal tooling

Passive observation tells you a host is likely exposed or likely affected. Active testing is what proves it definitely is. A CT entry plus a resolving DNS record plus a banner suggesting an unpatched version is an inference, not a confirmed fact, and the distinction between likely and definitely decides how you escalate it.

Plenty of what CT points you towards is not a CVE at all. Mismatched SANs where a certificate covers hostnames that belong to a decommissioned project. Admin panels on names nobody knew existed. Certificates issued by a CA your policy does not permit. Expiry clusters that will take down four services on the same Tuesday. Those are hygiene and configuration findings, and they are usually cheaper to fix than a patch cycle.

If you are assessing tooling, ask three concrete questions. How does the product acquire CT data, and does it poll logs directly or depend on a single aggregator? How fresh is the data, measured in hours from issuance? And what validation happens before a candidate becomes an inventory record, specifically whether resolution and a response check are enforced or whether every SAN string is simply imported as an asset. The practical checklist for choosing an EASM product covers the wider evaluation, but those three answers separate serious discovery from a crt.sh wrapper with a dashboard.

Two myths worth killing. CT-driven discovery does not exploit anything; it reads a public log and, at most, resolves a name. And publishing your certificates does not make you a target, because the logs are public by design and attackers have been reading them since before your monitoring did. The only thing withholding your own attention achieves is that they see your estate more clearly than you do.

The Short Version

  • CT output is a candidate list. Resolve every unique name and check what responds before anything enters inventory.
  • De-duplicate on the unique SAN set, not on certificate records, or your count reflects renewal churn rather than discovery.
  • Wildcards are the biggest under-reporting cause, alongside private CAs, non-TLS services, provider-owned certificates, unmapped domains and pre-2018 issuance.
  • A logged name plus a surviving CNAME to a deleted cloud resource is a takeover risk. Remove the DNS record.
  • Run it as a daily change feed with renewal suppression, and expect shrinking certificate lifetimes and tiled logs to break older scripts.

Used properly, certificate transparency logs subdomain discovery is the cheapest early warning you will ever deploy, and the least reliable single source of truth you could choose. Both statements are true at once, which is the whole point.

Frequently Asked Questions

Can certificate transparency logs find every subdomain a domain has?

No. CT only records certificates issued by publicly trusted CAs, so wildcard certificates, internal or self-signed certificates, services with no TLS and assets served under a provider’s own certificate all stay hidden. Certificates issued before Chrome’s SCT requirement took effect on 30 April 2018 may never have been logged either. Treat CT as one input alongside DNS enumeration and registration data.

Why do CT logs show subdomains that no longer resolve?

Because logs are append-only by design: once a certificate is submitted it stays there permanently, whether or not the host still exists. A name requested for a staging environment in 2019 and torn down a month later remains searchable indefinitely. That is why resolution is a mandatory triage step rather than an optional one.

Do wildcard certificates hide subdomains from certificate transparency logs?

Yes, effectively. A certificate for *.example.co.uk publishes the wildcard and nothing about the individual hostnames behind it, so an estate standardised on wildcards appears far smaller in CT than it really is. Use DNS record enumeration and your own zone files to fill that gap.

How often should you check CT logs for new certificates on your domains?

Daily is proportionate for most organisations, with renewal suppression so unchanged SAN sets do not generate alerts, plus a weekly review of upcoming expiries. As certificate lifetimes shorten under the CA/Browser Forum schedule agreed in 2025, the volume of renewal events per hostname rises, so the filtering matters more than the polling frequency.

Is querying certificate transparency logs for subdomain discovery legal and safe?

Querying the logs is reading published public data and involves no contact with the asset at all, which is what makes CT-based subdomain discovery a safe passive technique. The caveat is what you do next: resolving a name is harmless, but active probing or testing of a host you do not own or have permission to assess is a different activity with different rules. Keep the passive and active stages clearly separated, and confirm ownership before you test anything.