Certificate has unknown CA on Windows Subsystem for Linux

[crockeea]

General summary/comments (optional)

Stack works fine with Powershell and mingw64, but when I try to use it through "Bash on Ubuntu on Windows", I get

root@src# stack build --verbose
Version 1.9.3, Git revision 40cf7b37526b86d1676da82167ea8758a854953b (6211 commits) x86_64 hpack-0.31.1
2019-02-04 12:27:29.076297: [debug] Checking for project config at: /mnt/c/Users/ericcro/Desktop/VPC-Key-Distribution/src/stack.yaml
2019-02-04 12:27:29.079121: [debug] Loading project config file stack.yaml
2019-02-04 12:27:29.086691: [debug] Decoding build plan from: /root/.stack/build-plan/lts-13.3.yaml
2019-02-04 12:27:29.086885: [debug] Trying to decode /root/.stack/build-plan-cache/lts-13.3.cache
2019-02-04 12:27:29.087097: [debug] Exception ignored when attempting to load /root/.stack/build-plan-cache/lts-13.3.cache: /root/.stack/build-plan-cache/lts-13.3.cache: openBinaryFile: does not exist (No such file or directory)
2019-02-04 12:27:29.087483: [debug] Failure decoding /root/.stack/build-plan-cache/lts-13.3.cache
2019-02-04 12:27:29.087695: [debug] Decoding Stackage snapshot definition from file failed: InvalidYaml (Just (YamlException "Yaml file not found: /root/.stack/build-plan/lts-13.3.yaml"))
2019-02-04 12:27:29.088748: [debug] Downloading build plan from: https://raw.githubusercontent.com/fpco/lts-haskell/master//lts-13.3.yaml
2019-02-04 12:27:29.089178: [debug] Downloading /fpco/lts-haskell/master//lts-13.3.yaml
Downloading lts-13.3 build plan ...
RedownloadHttpError (HttpExceptionRequest Request {
  host                 = "raw.githubusercontent.com"
  port                 = 443
  secure               = True
  requestHeaders       = [("User-Agent","The Haskell Stack")]
  path                 = "/fpco/lts-haskell/master//lts-13.3.yaml"
  queryString          = ""
  method               = "GET"
  proxy                = Nothing
  rawBody              = False
  redirectCount        = 10
  responseTimeout      = ResponseTimeoutDefault
  requestVersion       = HTTP/1.1
}
 (InternalException (HandshakeFailed (Error_Protocol ("certificate has unknown CA",True,UnknownCa)))))

Steps to reproduce

To reproduce:

1. Open Bash on Ubuntu on Windows
2. go to any stack-enabled project and run stack build

Expected

I expect that stack will begin downloading and installed package dependencies.

Actual

Instead, I get a certificate error for raw.githubusercontent.com.

Stack version

$ stack --version
Version 1.9.3, Git revision 40cf7b37526b86d1676da82167ea8758a854953b (6211 commits) x86_64 hpack-0.31.1

What I've tried: Prior tickets with similar errors (#2241, a stackoverflow question, another issue, and yet another issue), suggested going to several sites in IE/Edge. This doesn't help. These tickets also suggest that the problem may be related to hs-tls somehow, but the troubleshooting section there doesn't reveal anything:

root@src# tls-retrievecertificate.exe raw.githubusercontent.com 443 --chain --verify
connecting to raw.githubusercontent.com on port 443 ...
###### Certificate 1 ######
serial:   10937661528139297494475781313019169126
issuer:   DistinguishedName {getDistinguishedElements = [([2,5,4,6],ASN1CharacterString {characterEncoding = Printable, getCharacterStringRawData = "US"}),([2,5,4,10],ASN1CharacterString {characterEncoding = Printable, getCharacterStringRawData = "DigiCert Inc"}),([2,5,4,11],ASN1CharacterString {characterEncoding = Printable, getCharacterStringRawData = "www.digicert.com"}),([2,5,4,3],ASN1CharacterString {characterEncoding = Printable, getCharacterStringRawData = "DigiCert SHA2 High Assurance Server CA"})]}
subject:  DistinguishedName {getDistinguishedElements = [([2,5,4,6],ASN1CharacterString {characterEncoding = Printable, getCharacterStringRawData = "US"}),([2,5,4,8],ASN1CharacterString {characterEncoding = Printable, getCharacterStringRawData = "California"}),([2,5,4,7],ASN1CharacterString {characterEncoding = Printable, getCharacterStringRawData = "San Francisco"}),([2,5,4,10],ASN1CharacterString {characterEncoding = Printable, getCharacterStringRawData = "GitHub, Inc."}),([2,5,4,3],ASN1CharacterString {characterEncoding = Printable, getCharacterStringRawData = "www.github.com"})]}
validity: DateTime {dtDate = Date {dateYear = 2017, dateMonth = March, dateDay = 23}, dtTime = TimeOfDay {todHour = 0h, todMin = 0m, todSec = 0s, todNSec = 0ns}} to DateTime {dtDate = Date {dateYear = 2020, dateMonth = May, dateDay = 13}, dtTime = TimeOfDay {todHour = 12h, todMin = 0m, todSec = 0s, todNSec = 0ns}}
###### Certificate 2 ######
serial:   6489877074546166222510380951761917343
issuer:   DistinguishedName {getDistinguishedElements = [([2,5,4,6],ASN1CharacterString {characterEncoding = Printable, getCharacterStringRawData = "US"}),([2,5,4,10],ASN1CharacterString {characterEncoding = Printable, getCharacterStringRawData = "DigiCert Inc"}),([2,5,4,11],ASN1CharacterString {characterEncoding = Printable, getCharacterStringRawData = "www.digicert.com"}),([2,5,4,3],ASN1CharacterString {characterEncoding = Printable, getCharacterStringRawData = "DigiCert High Assurance EV Root CA"})]}
subject:  DistinguishedName {getDistinguishedElements = [([2,5,4,6],ASN1CharacterString {characterEncoding = Printable, getCharacterStringRawData = "US"}),([2,5,4,10],ASN1CharacterString {characterEncoding = Printable, getCharacterStringRawData = "DigiCert Inc"}),([2,5,4,11],ASN1CharacterString {characterEncoding = Printable, getCharacterStringRawData = "www.digicert.com"}),([2,5,4,3],ASN1CharacterString {characterEncoding = Printable, getCharacterStringRawData = "DigiCert SHA2 High Assurance Server CA"})]}
validity: DateTime {dtDate = Date {dateYear = 2013, dateMonth = October, dateDay = 22}, dtTime = TimeOfDay {todHour = 12h, todMin = 0m, todSec = 0s, todNSec = 0ns}} to DateTime {dtDate = Date {dateYear = 2028, dateMonth = October, dateDay = 22}, dtTime = TimeOfDay {todHour = 12h, todMin = 0m, todSec = 0s, todNSec = 0ns}}
### certificate chain trust

and x509-util system shows a boatload of certificates.

[dbaynard]

I wonder if there's some checking against the subject on there, rather than the list of alternative names? I'll get somebody to investigate.

[dbaynard]

@borsboom has suggested: Could you be missing ca-certificates?

[borsboom]

Also, how did you install Stack?

[crockeea]

@borsboom I was indeed missing ca-certificates, but installing it didn't fix the problem.

I've long since forgotten how stack was installed, but just to verify, I uninstalled stack using your comment and installed it again with curl -sSL https://get.haskellstack.org/ | sh. Same error.

[hurlebouc]

Hello,

I'm also on WSL and I have the same problem.

[crockeea]

@dbaynard @borsboom Any updates on this? I'm not the only one impacted here.

[ketzacoatl]

@crockeea can you confirm, does curl return an error if you poke at github as well? What version of the ca-certificates package was installed? Can you inspect the CA Certs in that package to compare to a default Ubuntu install?

If you can prove that the host OS has the CA certs and can talk to github, stack will work too. This does not sound like an issue specific to Stack.

[crockeea]

~# curl raw.githubusercontent.com
~# curl https://github.com
<tons of html code>
~# apt-cache policy ca-certificates
  Installed: 20170717~16.04.2
  Candidate: 20170717~16.04.2
  Version table:
 *** 20170717~16.04.2 500
        500 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64 Packages
        100 /var/lib/dpkg/status
     20170717~16.04.1 500
        500 http://security.ubuntu.com/ubuntu xenial-security/main amd64 Packages
     20160104ubuntu1 500
        500 http://archive.ubuntu.com/ubuntu xenial/main amd64 Packages

The first curl command output nothing.

A stackoverflow answer suggests the following for listing CAs:

awk -v cmd='openssl x509 -noout -subject' '
    /BEGIN/{close(cmd)};{print | cmd}' < /etc/ssl/certs/ca-certificates.crt

On real Ubuntu, this command outputs 148 lines, the first few of which are:

subject=CN = ACCVRAIZ1, OU = PKIACCV, O = ACCV, C = ES
subject=C = ES, O = FNMT-RCM, OU = AC RAIZ FNMT-RCM
subject=C = IT, L = Milan, O = Actalis S.p.A./03358520967, CN = Actalis Authentication Root CA
subject=C = SE, O = AddTrust AB, OU = AddTrust External TTP Network, CN = AddTrust External CA Root
subject=C = US, O = AffirmTrust, CN = AffirmTrust Commercial
subject=C = US, O = AffirmTrust, CN = AffirmTrust Networking
subject=C = US, O = AffirmTrust, CN = AffirmTrust Premium
subject=C = US, O = AffirmTrust, CN = AffirmTrust Premium ECC
subject=C = US, O = Amazon, CN = Amazon Root CA 1
subject=C = US, O = Amazon, CN = Amazon Root CA 2

On Bash on Ubuntu on WIndows, the same command also outputs 148 lines, the first few of which are:

subject= /CN=ACCVRAIZ1/OU=PKIACCV/O=ACCV/C=ES
subject= /CN=ACEDICOM Root/OU=PKI/O=EDICOM/C=ES
subject= /C=IT/L=Milan/O=Actalis S.p.A./03358520967/CN=Actalis Authentication Root CA
subject= /C=SE/O=AddTrust AB/OU=AddTrust External TTP Network/CN=AddTrust External CA Root
subject= /C=SE/O=AddTrust AB/OU=AddTrust TTP Network/CN=AddTrust Class 1 CA Root
subject= /C=SE/O=AddTrust AB/OU=AddTrust TTP Network/CN=AddTrust Public CA Root
subject= /C=SE/O=AddTrust AB/OU=AddTrust TTP Network/CN=AddTrust Qualified CA Root
subject= /C=US/O=AffirmTrust/CN=AffirmTrust Commercial
subject= /C=US/O=AffirmTrust/CN=AffirmTrust Networking
subject= /C=US/O=AffirmTrust/CN=AffirmTrust Premium

[crockeea]

@ketzacoatl What are your thoughts? It looks like WSL can talk to Github and has CAs available, yet stack still does not work.

[snoyberg]

@crockeea Can you follow the debug steps in the tls package documentation? https://github.com/vincenthz/hs-tls#common-issues

[crockeea]

In Bash on Ubuntu on Windows:

$ cabal install x509-util
$ ./cabal/bin/x509-util system
<prints tons of certificates>

$ cabal install tls-debug
...
Configuring tls-debug-0.4.5...
Building tls-debug-0.4.5...
Preprocessing executable 'tls-stunnel' for tls-debug-0.4.5...

src/Stunnel.hs:4:8:
    Could not find module ‘Network.BSD’
    Perhaps you meant
      Network.TLS (from tls-1.4.1@tls_IPYqPctMYvpANuBu2lf3kO)

(and after modifying the cabal file, I get the same error for tls-retrievecertificate, which was the next thing I was trying to run).

I'm happy to try anything else...

[snoyberg]

I think you need to use a version of network less than 3

On Tue, Mar 26, 2019, 7:47 AM crockeea notifications@github.com wrote:

In Bash on Ubuntu on Windows:

$ cabal install x509-util

$ ./cabal/bin/x509-util system

$ cabal install tls-debug ... Configuring tls-debug-0.4.5... Building tls-debug-0.4.5... Preprocessing executable 'tls-stunnel' for tls-debug-0.4.5... src/Stunnel.hs:4:8: Could not find module ‘Network.BSD’ Perhaps you meant Network.TLS (from tls-1.4.1@tls_IPYqPctMYvpANuBu2lf3kO) (and after modifying the cabal file, I get the same error for tls-retrievecertificate, which was the next thing I was trying to run). I'm happy to try anything else... — You are receiving this because you commented. Reply to this email directly, view it on GitHub , or mute the thread .

[crockeea]

$ cabal sandbox init
$ cabal install network-2.8.0.0
$ cabal install tls-debug
$ cabal -v sandbox hc-pkg list
<confirmed that network-2.8.0.0 is installed>
$ .cabal-sandbox/bin/tls-retrievecertificate github.com 443 --chain --verify
connecting to github.com on port 443 ...
###### Certificate 1 ######
serial:   13324412563135569597699362973539517727
issuer:   DistinguishedName {getDistinguishedElements = [([2,5,4,6],ASN1CharacterString {characterEncoding = Printable, getCharacterStringRawData = "US"}),([2,5,4,10],ASN1CharacterString {characterEncoding = Printable, getCharacterStringRawData = "DigiCert Inc"}),([2,5,4,11],ASN1CharacterString {characterEncoding = Printable, getCharacterStringRawData = "www.digicert.com"}),([2,5,4,3],ASN1CharacterString {characterEncoding = Printable, getCharacterStringRawData = "DigiCert SHA2 Extended Validation Server CA"})]}
subject:  DistinguishedName {getDistinguishedElements = [([2,5,4,15],ASN1CharacterString {characterEncoding = UTF8, getCharacterStringRawData = "Private Organization"}),([1,3,6,1,4,1,311,60,2,1,3],ASN1CharacterString {characterEncoding = Printable, getCharacterStringRawData = "US"}),([1,3,6,1,4,1,311,60,2,1,2],ASN1CharacterString {characterEncoding = Printable, getCharacterStringRawData = "Delaware"}),([2,5,4,5],ASN1CharacterString {characterEncoding = Printable, getCharacterStringRawData = "5157550"}),([2,5,4,6],ASN1CharacterString {characterEncoding = Printable, getCharacterStringRawData = "US"}),([2,5,4,8],ASN1CharacterString {characterEncoding = Printable, getCharacterStringRawData = "California"}),([2,5,4,7],ASN1CharacterString {characterEncoding = Printable, getCharacterStringRawData = "San Francisco"}),([2,5,4,10],ASN1CharacterString {characterEncoding = Printable, getCharacterStringRawData = "GitHub, Inc."}),([2,5,4,3],ASN1CharacterString {characterEncoding = Printable, getCharacterStringRawData = "github.com"})]}
validity: DateTime {dtDate = Date {dateYear = 2018, dateMonth = May, dateDay = 8}, dtTime = TimeOfDay {todHour = 0h, todMin = 0m, todSec = 0s, todNSec = 0ns}} to DateTime {dtDate = Date {dateYear = 2020, dateMonth = June, dateDay = 3}, dtTime = TimeOfDay {todHour = 12h, todMin = 0m, todSec = 0s, todNSec = 0ns}}
###### Certificate 2 ######
serial:   16582437038678467094619379592629788035
issuer:   DistinguishedName {getDistinguishedElements = [([2,5,4,6],ASN1CharacterString {characterEncoding = Printable, getCharacterStringRawData = "US"}),([2,5,4,10],ASN1CharacterString {characterEncoding = Printable, getCharacterStringRawData = "DigiCert Inc"}),([2,5,4,11],ASN1CharacterString {characterEncoding = Printable, getCharacterStringRawData = "www.digicert.com"}),([2,5,4,3],ASN1CharacterString {characterEncoding = Printable, getCharacterStringRawData = "DigiCert High Assurance EV Root CA"})]}
subject:  DistinguishedName {getDistinguishedElements = [([2,5,4,6],ASN1CharacterString {characterEncoding = Printable, getCharacterStringRawData = "US"}),([2,5,4,10],ASN1CharacterString {characterEncoding = Printable, getCharacterStringRawData = "DigiCert Inc"}),([2,5,4,11],ASN1CharacterString {characterEncoding = Printable, getCharacterStringRawData = "www.digicert.com"}),([2,5,4,3],ASN1CharacterString {characterEncoding = Printable, getCharacterStringRawData = "DigiCert SHA2 Extended Validation Server CA"})]}
validity: DateTime {dtDate = Date {dateYear = 2013, dateMonth = October, dateDay = 22}, dtTime = TimeOfDay {todHour = 12h, todMin = 0m, todSec = 0s, todNSec = 0ns}} to DateTime {dtDate = Date {dateYear = 2028, dateMonth = October, dateDay = 22}, dtTime = TimeOfDay {todHour = 12h, todMin = 0m, todSec = 0s, todNSec = 0ns}}
### certificate chain trust

This all appears to be working to me, but I confirmed that stack still fails. Using tls-retrievecertificate on raw.githubusercontent.com port 443 (which is what stack complains about) also shows certificates as above.

[snoyberg]

As a short-term workaround, you can try manually downloading the snapshot file to /root/.stack/build-plan/lts-13.3.yaml (see the log messages at the very top).

It's possible that a new version of one of the dependencies fixed a bug affecting WSL. One way to test this would be to build tls-debug against lts-11.22 (the snapshot Stack 1.9.3 is built against), using stack --resolver lts-11.22 build tls-debug. I realize that may be difficult to pull off due to the bug you're encountering.

[snoyberg]

FYI, I spun up WSL on my Windows machine, and I'm unable to reproduce the issue here.

[crockeea]

@snoyberg About the time you said you couldn't reproduce, I was also unable to reproduce. I have no idea what I did to fix the issue. When I went back in my terminal history, I see that the error I reported in https://github.com/commercialhaskell/stack/issues/4560#issuecomment-476695954 is in fact not a certificate error:

root@SEA-1800195595:~# stack install lol
HttpExceptionRequest Request {
  host                 = "raw.githubusercontent.com"
  port                 = 443
  secure               = True
  requestHeaders       = [("User-Agent","The Haskell Stack")]
  path                 = "/fpco/stackage-content/master/stack/stack-setup-2.yaml"
  queryString          = ""
  method               = "GET"
  proxy                = Nothing
  rawBody              = False
  redirectCount        = 10
  responseTimeout      = ResponseTimeoutDefault
  requestVersion       = HTTP/1.1
}
 (ConnectionFailure Network.Socket.getAddrInfo (called with preferred socket type/protocol: AddrInfo {addrFlags = [AI_ADDRCONFIG], addrFamily = AF_UNSPEC, addrSocketType = Stream, addrProtocol = 6, addrAddress = <assumed to be undefined>, addrCanonName = <assumed to be undefined>}, host name: Just "raw.githubusercontent.com", service name: Just "443"): does not exist (Name does not resolve))

My best guess as to what fixed the original certificate problem is an upgrade to Windows 10 1809 (from 1703). The error about "Name does not resolve" appears to have been a coincidental DNS issue(?).

[snoyberg]

Huh, weird. Glad it's working now!

[kozross]

I'm still on 1703 (employed-enforced decision which I can't do anything about), and am hitting this issue as well. I have tried the various things attempted by crockeea as above, to the same effect.

[supermario]

For what it's worth to anyone stumbling on this – the issue seemed to magically disappear for me after running a curl https://<failing hostname> and then trying the stack command again 🤷‍♂️