x509: no valid chains built: Go 1.27.3 blocks 8 keys

A Go service that verified a certificate chain fine yesterday can fail today with tls: failed to verify certificate: x509: no valid chains built, and that line will not tell you why. No CA name, no hash, no hint that your binary now carries a blocklist.

Short version: Go 1.27.3 and 1.26.10 (CVE-2026-111943) compile eight SHA-256 SPKI hashes of eMudhra intermediates into crypto/x509 and drop any chain containing them. The distrust branch bumps no counter, so the error arrives with an empty detail string, which means one bare error line now has four distinct causes. Fixing it is a toolchain upgrade plus a rebuild plus a redeploy for every Go binary you run, with no config-level escape.

What landed, and how fast

Someone filed Go issue #82345 on 19 October 2026 under a title that deliberately says nothing: "crypto/x509: mitigation for security concerns with a specific Certificate Authority." Roughly twelve hours later the fix was on master as 7f86f95, "crypto/x509: distrust a handful of intermediates," cherry-picked to release-branch.go1.27 and release-branch.go1.26 as CLs 851047, 851145 and 851165, assigned a CVE, and shipped. The golang-announce release went out with no pre-announcement window, unlike the go1.27.2 drop thirteen days earlier.

Commenters supplied what the issue title withheld. The eight are eMudhra intermediates: EM DV TLS CA, EM OV TLS CA and EM EV TLS CA in G3B (eMudhra BV, NL) and G3C (eMudhra Inc, US) flavours under emSign Root TLS CA - G3, plus two code-signing intermediates, EM OV CS CA - G3B and EM OV CS CA - G3C, under emSign Root CS CA - G3. All eight were revoked on 18 and 19 October with reason code cACompromise. Commenter x0wllaar pointed out that the revocation entries appear in CCADB with no public announcement attached anywhere.

The interesting part is not the CA story. Certificate.Verify carries a doc comment that reads, verbatim, "WARNING: this function doesn't do any revocation checking." A cACompromise revocation is invisible to every Go program ever compiled. The ecosystem's emergency channel does not reach Go, so the only lever available was the compiler.

Why one error string now has four causes

Reading Verify in src/crypto/x509/verify.go on master: it builds candidate chains, then prunes them with candidateChains = slices.DeleteFunc(candidateChains, func(chain []*Certificate) bool {...}). Three branches inside that closure already dropped chains for invalid certificate policies, incompatible key usage, and name-constraint failures. The new fourth branch is one line: if containsDistrustedSPKIHash(chain) { return true }.

The three original branches increment invalidPoliciesChains or incompatibleKeyUsageChains, or set constraintsHintErr. Those counters are what build the details string in the returned error. The distrust branch increments nothing. So when every candidate chain is filtered for distrust alone, details stays empty and Verify returns CertificateInvalidError{c, NoValidChains, ""}, whose Error() is exactly "x509: no valid chains built" with no suffix.

The last row is a one-minute check, so run it first. For a live endpoint, pull the chain with openssl s_client -connect host:443 -showcerts </dev/null, split it, and hash each certificate:

openssl x509 -in cert.pem -noout -pubkey \
  | openssl pkey -pubin -outform DER \
  | openssl dgst -sha256

openssl pkey -outform DER emits precisely the SubjectPublicKeyInfo DER that RawSubjectPublicKeyInfo holds, which is what containsDistrustedSPKIHash feeds to sha256.Sum256 before hex-encoding and looking up distrustedSPKIHashes. The hex matches the map entries byte for byte. Two known-bad values to compare against: ebed6475…5105 (EM DV TLS CA - G3B) and 49dd353d…fe5f (EM DV TLS CA - G3C). The full eight are in the commit diff and the issue thread, and CCADB's "All Certificate Information" CSV on the resources page is the machine-readable place to confirm the revocations. If no hash matches, you are back on policy or key usage, and the detail suffix normally says which. That split matters for the same reason it did when pnpm 12.6 treated NODE_EXTRA_CA_CERTS as a trust-store replacement: one TLS error text, several unrelated defects behind it.

The list is keyed on the key, not the certificate

Because the lookup hashes the public key info, re-issuance buys the CA nothing. A new serial, new validity window and new signature over the same key pair produce the same SPKI hash and get dropped identically. A new key pair is required. That design also keeps the blocklist cheap: eight 32-byte values cover every certificate ever issued over those keys, past and future, which is why Go could afford to compile it in rather than ship a fetch mechanism it does not have.

Your toolchain version is now part of your trust configuration

Go is the implementation language of the infrastructure control plane: kubectl, kubelet, Terraform, Vault, etcd, Prometheus, Docker, Helm, the GitHub Actions runner shims, most operators in a cluster. Every one of those binaries carries its own statically linked copy of crypto/x509, so every one carries its own copy of this blocklist, or no copy at all if it was built before yesterday.

That inverts the normal patch model. A CA distrust is usually a trust-store operation. Firefox pushes CRLite. Chrome pushes a CRLSet, which the Chromium documentation describes as "the primary means by which Chrome quickly blocks certificates in emergency situations," covering intermediates and leaves, delivered by the component updater, visible at chrome://components, no browser upgrade needed. Debian and RHEL push ca-certificates. None of that moves a Go binary. update-ca-certificates cannot insert an entry into a map that was compiled into /usr/local/bin/kubectl three releases ago.

So remediation is a rebuild, gated on each upstream vendor's release train. The two checks that find your exposure:

go version -m /usr/local/bin/kubectl        # works on binaries you did not build
grep -E '^(go|toolchain) ' go.mod           # the pin that quietly undoes the upgrade

The trap is on the build side. A toolchain go1.27.1 line in go.mod, or GOTOOLCHAIN=local baked into a CI image, keeps producing unpatched binaries on a fully patched machine, and nothing in the build output complains. CI stays green while the estate stays unpatched, which is the same failure pattern as an npm 12 rollout that only breaks after CI goes green. Add go env GOTOOLCHAIN to the same audit, and treat a floating pin with the same suspicion you would give a node:lts tag that moves without a commit.

Three code paths, one list, and the forks that get nothing

The commit also touches root_darwin.go and root_windows.go, because macOS and Windows delegate chain building to the OS verifier and never reach the Go pruning pass. Darwin returns CertificateInvalidError with NoValidChains. Windows applies slices.DeleteFunc(chains, containsDistrustedSPKIHash) and errors only once every chain is gone. Same eight hashes, three enforcement points, slightly different error construction in each.

There is a fourth path with no enforcement at all: any third-party x509 fork. zcrypto, the vendored copy inside certificate-transparency-go, anything else that lifted verify.go and diverged. Those get the distrust when their maintainers port it, if they do. One list reimplemented across several codebases goes the way the four core.fsmonitor agent patches went, each with a different key list, and a scanner-style fix that leaves a sibling untouched is the same shape as Starlette 1.0.1 missing its sibling package.

One more consequence worth naming: containsDistrustedSPKIHash reads no key usage. Two of the eight are code-signing intermediates, so a Go program verifying a chain with ExtKeyUsageCodeSigning rejects them too. Chrome's CRLSet does not cover code signing, and Microsoft tracks code-signing trust through a separate CCADB report. Go is enforcing across purposes ahead of the purpose-specific root programs.

The counterpoint

The blast radius is probably tiny. eMudhra is a minor issuer and these are new-generation intermediates, so the number of production chains that break is likely small enough that most teams will never see the error. Fair. The operational lesson survives anyway, because the mechanism is now established: a CA trust decision propagates through the Go ecosystem at the speed of your slowest dependency's release train, and the next distrust might hit an issuer you actually use.

Also note what is absent. No GODEBUG, no exported knob, no documented escape hatch, which for a Go behaviour change of this class reads as intentional. And on detection, a search of golang/vulndb on 20 October returns nothing referencing CVE-2026-111943. The shape of the fix suggests a report may never help: govulncheck matches vulnerable symbols, and this release changed a data table. No symbol, no finding. Test that assumption before you rely on it to tell you which services are stale.

What to run this week, in order

1. Inventory toolchains, not package versions. Loop go version -m over every Go binary you ship or vendor and flag anything at go1.27.2 or earlier in the 1.27 line, or go1.26.9 or earlier in 1.26. Those have no blocklist. This includes binaries you pulled from a vendor.
2. Grep your build inputs for pins. grep -E '^(go|toolchain) ' go.mod across every repo, plus go env GOTOOLCHAIN in each CI image. A toolchain directive below 1.27.3 or a GOTOOLCHAIN=local on an unpatched image is how a fully patched machine keeps emitting unpatched binaries.
3. Add the SPKI hash check to your TLS runbook. When x509: no valid chains built appears with an empty suffix, hash the served chain's SPKIs with the openssl pkey -pubin -outform DER | openssl dgst -sha256 pipeline above before touching code or certificates. A match against the eight ends the investigation; no match sends you to policy and key-usage validation, which is a different fix.
4. Check whether you hold anything from these issuers. If any of the eight sits in a chain you serve or verify, a re-issue over the same key pair will not clear it. Plan for a new key pair, and check your code-signing chains too, since the filter ignores EKUs.
5. Audit your x509 forks. If anything in your stack verifies certificates through zcrypto or a vendored verify.go, it is not enforcing this list and will not start on a toolchain bump.

And stop assuming revocation reaches your software at all. Mozilla built CRLite and Google built CRLSets because CRL and OCSP checking does not work at scale. Go answered the same question by not checking revocation, then shipping a compile-time list when it had to. Any service whose threat model assumes a revoked intermediate stops working should be able to name the mechanism that makes that true, and if the answer is "the OS trust store," there is a statically linked Go binary in the same cluster proving otherwise.

What I read while writing this: Go issue #82345 and its comment thread, commit 7f86f95, the current src/crypto/x509/verify.go on master, the Chromium CRLSets documentation, and the CCADB resources page for the revocation records.