crypto/x509: reject SHA-1 signatures in Verify

[FiloSottile]

SHA-1 is weak: a SHA-1 collision was demonstrated and estimated to cost around $50k. https://shattered.io

Accepting SHA-1 signed certificates is a security issue, and lets attackers mount collision attacks if the CA is still signing SHA-1 certificates. crypto/x509 already rejects outright any MD5 signatures for the same reason.

The WebPKI has banned SHA-1 certificates for years now, and crypto/x509 targets a profile compatible with the WebPKI.

I propose we announce in Go 1.17 that we'll remove support in Go 1.18, and provide a GODEBUG opt-out until Go 1.19.

[FiloSottile]

It doesn't look like the opt-in GODEBUG phase has been useful in the past, and SHA-1 is already severely broken. Let's go straight to the GODEBUG opt-out phase for Go 1.17, assuming people test the new release in canaries or similar anyway.

[rsc]

This proposal has been added to the active column of the proposals project and will now be reviewed at the weekly proposal review meetings. — rsc for the proposal review group

[ianlancetaylor]

@Prajwal-Koirala See https://golang.org/wiki/NoPlusOne. Thanks.

[rsc]

How many of the ancient servers being discussed in #45428 are serving SHA-1 signatures?

[FiloSottile]

SHA-1 in crypto/x509 is unrelated to crypto/tls, except to the extent that if you're running a legacy stack you're more likely to have both components be out of date. You can serve a SHA-1 certificate over TLS 1.3, if you felt like it.

There are no publicly trusted SHA-1 certificates anymore, so we pretty much have no numbers about them. (Well, we do, and they say zero, but they don't capture internal deployments.) Anyone using them is doing it with their own managed CA.

[rsc]

Based on the discussion above, this proposal seems like a likely accept. — rsc for the proposal review group

[rsc]

No change in consensus, so accepted. 🎉 This issue now tracks the work of implementing the proposal. — rsc for the proposal review group

[FiloSottile]

https://golang.org/cl/327811 has the pre-announcement, moving to Go 1.18 for implementation.

[gopherbot]

Change https://golang.org/cl/327811 mentions this issue: doc/go1.17: add Go 1.18 pre-announcements

[gopherbot]

Change https://golang.org/cl/359777 mentions this issue: crypto/x509: disable SHA-1 signature verification

[ianlancetaylor]

@FiloSottile Is there anything more to do here for 1.18? Thanks.

[andig]

I'm wondering if this is maybe premature. Trying to connect to my server I'm seeing

authentication handshake failed: x509: certificate signed by unknown authority (possibly because of \"x509: cannot verify signature: insecure algorithm SHA1-RSA (temporarily override with GODEBUG=x509sha1=1)\" while trying to verify candidate authority certificate \"serial:...\")

which seems related to this change. The fun part is that the ca and server certificates were only generated a year ago using OpenSSL with default setting afaikt. It seems as if this would hit a broader audience in a bad way if OpenSSL default certificated are rejected?

[odeke-em]

@andig did you see the opt-in using GODEBUG=x509sha1=1 go run main.go? That might resolve your problem as this issue is to intentionally opt out until opted in. I am also going to punt this to Go1.19 since we've fulfilled the goals per the original issue.

[andig]

@odeke-em I'm aware of the switch. Nevertheless I would expect Go to successfully connect to something that was setup using default OpenSSL config.

I am also going to punt this to Go1.19 since we've fulfilled the goals per the original issue.

What does that mean? Right now gotip still requires opt-out, so this is part of 1.18. Should I open new issue to discuss moving it to 1.19?

[FiloSottile]

@andig this kind of change is a balancing act. You're right that a big part is what other important players in the ecosystem are doing. Regrettably, OpenSSL CLIs move too slowly to be a benchmark. Apparently, the OpenSSL project considers them testing, not production tools. If we were to wait until years have passed since the OpenSSL CLI defaults caught up with the world, we'd be always 10 years behind, which is an unacceptable cost. Moreover, the situation is made worse by long-cycle Linux distributions such as Debian, which will ship to users versions from several years ago. At the end of the day, the OpenSSL CLI made you a certificate that even at the time was not going to work with any browser. Letting it work with Go is a disservice, as if it didn't you'd probably have noticed that OpenSSL was not doing the right thing.

[mnordhoff]

As an example, I think Debian unstable changed the default to SHA-256 in like 2013, inherited by later Debian and Ubuntu releases at some point. Many environments should be generating acceptable certificates by default, I think.

[andig]

I realise this is now absolutely my problem and not related to Go, but I'm totally failing to sign a CSR with subjectAlternateNames on OSX using SHA256. If anyone here has a pointer to a guide that would be greatly appreciated. Please excuse the OT response.

[gopherbot]

Change https://golang.org/cl/382454 mentions this issue: doc/go1.18: add crypto/tls, crypto/x509, and crypto/elliptic release notes

[AGWA]

The WebPKI has banned SHA-1 certificates for years now, and crypto/x509 targets a profile compatible with the WebPKI.

Although the WebPKI has banned SHA-1 certificates, it has not banned SHA-1-signed OCSP responses or CRLs. Consequentially, Go 1.18 is unable to verify some WebPKI OCSP responses (e.g. https://httpd.sslmate.com/ocspwatch/response/3530439198) and CRLs. Although the BRs were just amended to ban SHA-1-signed OCSP responses as of 2022-06-01, there is no sunset planned for CRLs.

If crypto/x509 aims to be compatible with the WebPKI, then I'm afraid this change has to be rolled back or limited to just certificate signatures.

Incidentally, the Go 1.18 release notes are imprecise. They say "crypto/x509 will now reject certificates signed with the SHA-1 hash function" when in reality all SHA-1 signatures are being rejected.

[gopherbot]

Change https://go.dev/cl/394294 mentions this issue: crypto/x509: only disable SHA-1 verification for certificates

[FiloSottile]

If crypto/x509 aims to be compatible with the WebPKI, then I'm afraid this change has to be rolled back or limited to just certificate signatures.

You are, as always, correct :) we'll limit it to just certificates in Go 1.18.1.

Change https://go.dev/cl/394294 mentions this issue: crypto/x509: only disable SHA-1 verification for certificates

@gopherbot please open a backport issue for Go 1.18, we need to fix what's effectively a regression in WebPKI compatibility.

[gopherbot]

Backport issue(s) opened: #51852 (for 1.18).

Remember to create the cherry-pick CL(s) as soon as the patch is submitted to master, according to https://go.dev/wiki/MinorReleases.

[mikewiacek]

FWIW we're using the x509 package to help parse and validate signed PE files from past and current Windows systems. There are tens of millions such files in existence and these signatures don't expire as time moves forward. Whether or not SHA-1 is viewed as insecure today, doesn't mean it's dangerous to determine if a file may have been validly signed in the past.

In our case, there are two things to think about "Does the mathematics say the file was signed correctly?" and separately "Is the valid signature reliable enough to trust?". I suppose you could technically sign a file with an 8-bit RSA key, and the answers to these questions should be "yes" and "definitely not". But as proposed I wouldn't be able to disambiguate the two conditions.

I think https://go.dev/cl/394294 will address this for us but I wanted to call it out here as a valid use of crypto/x509.

[simo5]

@mikewiacek eventually it does not matter if the file you are testing now "pretends" to have been signed in the past, once you distrust SHA1 signatures, you just do not get any value in checking old ones, they can all be faked, therefore the fact they "check out" is effectively meaningless.

[mikewiacek]

@simo5 not true. We're actually building a platform for large scale malware analysis. I'm very much interested in knowing they were signed at all because it's a useful feature to help classify trends. Looking for certificates that were used to sign legitimate software AND then used to sign malware is important in large scale deep retrospective analysis. Even if they were faked, the existence of such fakes is still useful data.

[simo5]

@mikewiacek you may have a legitimate niche use case, that in no way implies what I said about trust and general usefulness is not true.

[liggitt]

is there an ETA for https://go.dev/cl/394294 and go1.18.1?

[gopherbot]

Change https://go.dev/cl/398074 mentions this issue: [release-branch.go1.18] crypto/x509: only disable SHA-1 verification for certificates

[gopherbot]

Change https://go.dev/cl/396774 mentions this issue: crypto/x509: move sha1 removal to unspecified future release

[FiloSottile]

The removal of the x509sha1=1 GODEBUG switch has been moved back. SHA-1 certificates are not secure, and applications still relying on them should migrate ASAP and not rely on the ongoing presence of the GODEBUG opt-in.

[ianlancetaylor]

CC @golang/security

This issue is still in the 1.19 milestone. What is left to do here? What is the target release? Thanks.

[jglick]

What is left to do here?

Presumably to delete https://github.com/golang/go/blob/fdb640b7a1324c2a4fc579389c4bc287ea90f1db/src/crypto/x509/x509.go#L728-L729

Did anyone consider retaining the ability to honor SHA-1 for a while but as an option in https://github.com/golang/go/blob/b9c4d94fdbe0c9e11a1e604cf321614b90a1d882/src/crypto/x509/verify.go#L175-L176 (with suitably stern warnings) rather than an environment variable?

[rolandshoemaker]

Per Filippo's comment we moved back the removal to an unspecified future release due to persistent ongoing ecosystem reliance on SHA-1 that cannot be worked around. I've removed this from the 1.19 milestone.

Rather than making further API changes to retain support, continuing to use a GODEBUG flag makes it hard to use, which is the correct choice since you should really only be using it if you absolutely must.

[gopherbot]

Change https://go.dev/cl/445496 mentions this issue: crypto/x509: respect GODEBUG changes for allowing SHA1 certificates

[joedian]

@gopherbot please backport this to previous releases.

[gopherbot]

Change https://go.dev/cl/445655 mentions this issue: crypto/x509: respect GODEBUG changes for allowing SHA1 certificates

[gopherbot]

Change https://go.dev/cl/445656 mentions this issue: crypto/x509: respect GODEBUG changes for allowing SHA1 certificates

[gopherbot]

Change https://go.dev/cl/629676 mentions this issue: crypto/x509: remove x509sha1 GODEBUG