search

golang / go

The Go programming language
https://go.dev
BSD 3-Clause "New" or "Revised" License
123.69k stars 17.62k forks source link

crypto/x509: Trust setting not inherited on darwin #30471

Closed vdobler closed 5 years ago

vdobler commented 5 years ago

What version of Go are you using (go version)?

$ go version
go version go1.12 darwin/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/m/Library/Caches/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOOS="darwin"
GOPATH="/Users/m/go"
GOPROXY=""
GORACE=""
GOROOT="/usr/local/go"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/darwin_amd64"
GCCGO="gccgo"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD="/Users/m/go/src/git.intern.migros.net/devops-api-mobile/ch.migros.lottery/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/f6/vr81k40j5y35bpj039w1hwfdrvg106/T/go-build545405925=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?

$ go build

What did you expect to see?

Nothing (a successful build).

What did you see instead?

go: git.intern.migros.net/package7path@v1.3.0: unrecognized import path "git.intern.migros.net/package/path" (https fetch: Get https://git.intern.migros.net/package/path?go-get=1: x509: certificate signed by unknown authority) go: error loading module requirements

More details

This is basically just issue #24652 but with Go 1.12

The same fix (mark root ca as "Allways trust") as described in https://github.com/golang/go/issues/24652#issuecomment-448035853 makes the problem go away.

Curl and Browsers do accept the certificate. E.g

$ curl -v "https://git.intern.migros.net/package/path?go-get=1"
*   Trying 164.xxx.xxx.xxx...
* TCP_NODELAY set
* Connected to git.intern.migros.net (164.xxx.xxx.xxx) port 443 (#0)
* TLS 1.2 connection using TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
* Server certificate: git.intern.migros.net
* Server certificate: Migros System CA 2
* Server certificate: Migros Root CA 2
> GET /package/path?go-get=1 HTTP/1.1
> Host: git.intern.migros.net
> User-Agent: curl/7.54.0
> Accept: */*

But running TestSystemRoots yields:

$ GODEBUG=x509roots=1 ../bin/go test -v -run TestSystemRoots crypto/x509
=== RUN   TestSystemRoots
crypto/x509: 6 certs have a trust policy
crypto/x509: verify-cert approved CN=Migros User CA 2,O=Migros,C=CH
crypto/x509: verify-cert approved CN=QuoVadis Swiss Advanced CA G2,O=QuoVadis Trustlink Switzerland Ltd.,C=CH
crypto/x509: verify-cert approved CN=webproxy.dc.migros.ch,OU=Informatik,O=Migros,L=Zuerich,ST=Zuerich,C=CH
crypto/x509: verify-cert approved CN=Migros Root CA 2,O=Migros,C=CH
crypto/x509: verify-cert approved CN=Migros Root CA 2,O=Migros,C=CH
crypto/x509: verify-cert approved CN=Migros User CA 2,O=Migros,C=CH
crypto/x509: verify-cert rejected CN=dlv-cert: "Cert Verify Result: Invalid Extended Key Usage for policy"
crypto/x509: verify-cert approved CN=QuoVadis Swiss Advanced CA G2,O=QuoVadis Trustlink Switzerland Ltd.,C=CH
crypto/x509: verify-cert approved CN=Migros Root CA 2,O=Migros,C=CH
crypto/x509: verify-cert approved CN=macman.migros.net,OU=Migros Genossenschafts Bund,O=Migros Genossenschafts Bund,L=Zuerich,ST=Zueich,C=CH
crypto/x509: verify-cert approved CN=Migros User CA 2,O=Migros,C=CH
crypto/x509: verify-cert approved CN=QuoVadis Swiss Advanced CA G2,O=QuoVadis Trustlink Switzerland Ltd.,C=CH
crypto/x509: verify-cert approved CN=macman.migros.net,OU=Migros Genossenschafts Bund,O=Migros Genossenschafts Bund,L=Zuerich,ST=Zueich,C=CH
crypto/x509: ran security verify-cert 13 times
Number of trusted certs = 2
Cert 0: Migros Root CA 2
   Number of trust settings : 4
   Trust Setting 0:
      Policy OID            : SSL
      Policy String         : autodiscover.migros.net
      Allowed Error         : CSSMERR_TP_CERT_EXPIRED
      Result Type           : kSecTrustSettingsResultTrustRoot
   Trust Setting 1:
      Policy OID            : SSL
      Policy String         : autodiscover.migros.net
      Allowed Error         : Host name mismatch
      Result Type           : kSecTrustSettingsResultTrustRoot
   Trust Setting 2:
      Policy OID            : Apple X509 Basic
      Policy String         : autodiscover.migros.net
      Allowed Error         : CSSMERR_TP_CERT_EXPIRED
      Result Type           : kSecTrustSettingsResultTrustRoot
   Trust Setting 3:
      Allowed Error         : CSSMERR_TP_CERT_EXPIRED
      Result Type           : kSecTrustSettingsResultTrustRoot
Cert 1: macman.migros.net
   Number of trust settings : 2
   Trust Setting 0:
      Policy OID            : EAP
      Allowed Error         : CSSMERR_TP_CERT_EXPIRED
      Result Type           : kSecTrustSettingsResultTrustAsRoot
   Trust Setting 1:
      Policy OID            : Apple X509 Basic
      Allowed Error         : CSSMERR_TP_CERT_EXPIRED
      Result Type           : kSecTrustSettingsResultTrustAsRoot
Number of trusted certs = 5
Cert 0: Migros User CA 2
   Number of trust settings : 1
   Trust Setting 0:
      Result Type           : kSecTrustSettingsResultTrustAsRoot
Cert 1: QuoVadis Swiss Advanced CA G2
   Number of trust settings : 1
   Trust Setting 0:
      Result Type           : kSecTrustSettingsResultTrustAsRoot
Cert 2: webproxy.dc.migros.ch
   Number of trust settings : 0
Cert 3: dlv-cert
   Number of trust settings : 0
Cert 4: Migros Root CA 2
   Number of trust settings : 0
--- FAIL: TestSystemRoots (0.31s)
    root_darwin_test.go:35:     cgo sys roots: 56.90889ms
    root_darwin_test.go:36: non-cgo sys roots: 150.793582ms
    root_darwin_test.go:77: certificate only present in non-cgo pool: CN=Migros User CA 2,O=Migros,C=CH (verify error: x509: certificate signed by unknown authority)
    root_darwin_test.go:79: signed certificate only present in non-cgo pool (acceptable): CN=QuoVadis Swiss Advanced CA G2,O=QuoVadis Trustlink Switzerland Ltd.,C=CH
    root_darwin_test.go:77: certificate only present in non-cgo pool: CN=Migros Root CA 2,O=Migros,C=CH (verify error: x509: certificate signed by unknown authority)
    root_darwin_test.go:79: signed certificate only present in non-cgo pool (acceptable): CN=macman.migros.net,OU=Migros Genossenschafts Bund,O=Migros Genossenschafts Bund,L=Zuerich,ST=Zueich,C=CH
    root_darwin_test.go:79: signed certificate only present in non-cgo pool (acceptable): CN=Developer ID Certification Authority,OU=Apple Certification Authority,O=Apple Inc.,C=US
    root_darwin_test.go:99: off-EKU certificate only present in cgo pool (acceptable): CN=dlv-cert
FAIL
crypto/x509: kSecTrustSettingsResultInvalid = 0
crypto/x509: kSecTrustSettingsResultTrustRoot = 1
crypto/x509: kSecTrustSettingsResultTrustAsRoot = 2
crypto/x509: kSecTrustSettingsResultDeny = 3
crypto/x509: kSecTrustSettingsResultUnspecified = 4
crypto/x509: Migros User CA 2 returned 4
crypto/x509: QuoVadis Swiss Advanced CA G2 returned 4
crypto/x509: webproxy.dc.migros.ch returned 1
crypto/x509: dlv-cert returned 1
crypto/x509: Migros Root CA 2 returned 4
crypto/x509: Migros Root CA 2 returned 4
crypto/x509: macman.migros.net returned 4
FAIL    crypto/x509 0.317s
vdobler commented 5 years ago

@FiloSottile In #24652 you requested to be tagged if the problem is not completely solved in Go 1.12.

SteelPhase commented 5 years ago

I've run into the same issue, and while I can't share the output of the test in question. I can confirm that marking the intermediate ca as Always Trusted resolves the issue for me. I would hope that this would be fixed in a Go 1.12.x release

FiloSottile commented 5 years ago

Thank you for the report. Is this a regression or was it broken in Go 1.11 as well?

SteelPhase commented 5 years ago

This was working for me in Go 1.11. I installed my companies root CA, and two intermediate CAs in the system keychain. At the time when I did this, it was required because Go didn't check the login keychain. That specific details appears to have changed, but I'm not 100% sure that was introduced in Go 1.12. I only had the root CA marked as Always Trusted.

After running into this issue today, I cleaned up the certs in my system keychain, and moved them to the login keychain. I tested this in both locations. When only the root CA is marked Always Trusted, go doesn't trust either of the intermediate certs. When all 3 certs are marked Always Trusted, go has no trust issues with the certificates.

As an additional note, The ouput of GODEBUG=x509roots=1 go test -v -run TestSystemRoots crypto/x509 didn't change based on Always Trusted settings, and I assume that is a red herring.

I can confirm that a script that relies on the certificate being trusted works correctly when I change the intermediate CA it depends on to Always Trusted

SteelPhase commented 5 years ago

I'm going to include this tidbit just because it's slightly related

I'm on macOS Mojave 10.14.3, and it appears security handles pointing login.keychain to login.keychain-db internally. I shasum'd the output of the commands with the different paths below it it's the same value.

https://github.com/golang/go/blob/13d24b685a6d7b05a249f85be91c390f5595f745/src/crypto/x509/root_darwin.go#L74-L79

steel@computer:src$ls -la ~/Library/Keychains/
drwxr-xr-x  11 steel  000000000     352 Feb 28 19:36 .
drwx------@ 75 steel  000000000    2400 Feb 11 10:21 ..
-rw-r--r--@  1 steel  000000000    6148 Jan  7 15:33 .DS_Store
-rw-r--r--@  1 steel  000000000  736696 Feb 28 19:36 login.keychain-db
-rw-------   1 steel  000000000   30740 Feb 12 11:13 metadata.keychain-db
steel@computer:src$security find-certificate -a ~/Library/Keychains/login.keychain | shasum
a2b9a124e87200b0663f8b45df45c39584663bd9  -
steel@computer:src$security find-certificate -a ~/Library/Keychains/login.keychain-db | shasum
a2b9a124e87200b0663f8b45df45c39584663bd9  -
kewilson commented 5 years ago

@FiloSottile .. I sent you an email yesterday regarding this same issue as mentioned in 24652 still occurring. I'm trying to get past the issue now so if you want me to try some stuff to help out please let me know.

golang 1.12.1. macos 10.13.6 intellij ultimate 2018.3.5

kewilson commented 5 years ago

I employed the All Trust work around on three certs using system defaults, all the others in play were already trusted and still receive "x509: certificate signed by unknown authority" when calling urls from golang project. As mentioned by others pasting the url in chrome and it returns data as expected.

kewilson commented 5 years ago

Anyone? Bueller? Bueller?

adamdecaf commented 5 years ago

Could your issue be fixed by https://github.com/golang/go/issues/30672 ? There's an open CL, could you try it?

https://go-review.googlesource.com/c/go/+/166219

kewilson commented 5 years ago

@adamdecaf .. how might I go about trying this since it hasn't been merged as far as I can tell?

$>brew upgrade golang golang 1.12.1 already installed

andlabs commented 5 years ago

I'll try it, because I'm having a similar problem with the same error message (in this case, it's an IRC server's certificate; I don't remember why it's misconfigured). Patching 33e5da48d5d22a722f2363b15e2d53061fb71cf4, running on 10.12.6, using the Cherry-Pick option in Gerrit.

Before:

=== RUN   TestSystemRoots
--- FAIL: TestSystemRoots (0.30s)
    root_darwin_test.go:35:     cgo sys roots: 81.121343ms
    root_darwin_test.go:36: non-cgo sys roots: 172.672874ms
    root_darwin_test.go:77: certificate only present in non-cgo pool: CN=irc.rustedlogic.net,OU=Server Admins,O=Rusted Logic IRC Network,L=Example City,ST=Example State,C=US (verify error: x509: certificate signed by unknown authority)
    root_darwin_test.go:79: signed certificate only present in non-cgo pool (acceptable): CN=irc.gnome.org,OU=Domain Control Validated+OU=PositiveSSL Multi-Domain
    root_darwin_test.go:79: signed certificate only present in non-cgo pool (acceptable): CN=Developer ID Certification Authority,OU=Apple Certification Authority,O=Apple Inc.,C=US

After:

=== RUN   TestSystemRoots
--- PASS: TestSystemRoots (0.30s)
    root_darwin_test.go:35:     cgo sys roots: 88.204333ms
    root_darwin_test.go:36: non-cgo sys roots: 168.548068ms
    root_darwin_test.go:79: signed certificate only present in non-cgo pool (acceptable): CN=Developer ID Certification Authority,OU=Apple Certification Authority,O=Apple Inc.,C=US

@kewilson You will need to build Go from source, downloading the repository via git and keeping it at HEAD. In Gerrit, there will be a button at the bottom right of the description that says DOWNLOAD; click it to get instructions on how to apply the Gerrit patch locally.

gopherbot commented 5 years ago

Change https://golang.org/cl/178539 mentions this issue: crypto/x509: include roots with empty or multiple policies on macOS

FiloSottile commented 5 years ago

This should be now fixed at tip. Please test it with https://golang.org/dl/gotip and report back.

$ go get golang.org/dl/gotip
$ gotip download
$ GODEBUG=x509roots=1 gotip test -v -run TestSystemRoots crypto/x509
$ gotip run [YOUR_PROGRAM]
andlabs commented 5 years ago

Can confirm the normal ./all.bash from HEAD works for me; thanks. (Not sure about gotip; I was waiting for this so I could set up my Go development environment to begin with =P )

tommyknows commented 5 years ago

cannot confirm this to be working. I cloned the repository and ran ./all.bash. It works if running as root / sudo, but I guess the certificates are not present there.

Running it as my normal user, I get:

--- FAIL: TestSystemRoots (0.82s)
    root_darwin_test.go:35:     cgo sys roots: 464.908686ms
    root_darwin_test.go:36: non-cgo sys roots: 293.10636ms
    root_darwin_test.go:77: certificate only present in non-cgo pool: CN=[redacted] (verify error: x509: certificate signed by unknown authority)
    root_darwin_test.go:79: signed certificate only present in non-cgo pool (acceptable): CN=[redacted]
    root_darwin_test.go:77: certificate only present in non-cgo pool: CN=[redacted](verify error: x509: certificate signed by unknown authority)
    root_darwin_test.go:79: signed certificate only present in non-cgo pool (acceptable): CN=Developer ID Certification Authority,OU=Apple Certification Authority,O=Apple Inc.,C=US

Am I missing something?

FiloSottile commented 5 years ago

@tommyknows Please post the full (redacted) output of https://github.com/golang/go/issues/30471#issuecomment-494875696 in a new issue and tag me.

gopherbot commented 4 years ago

Change https://golang.org/cl/227037 mentions this issue: crypto/x509: use Security.framework without cgo for roots on macOS