SSL CA certificates not found on fedora

[emmanueltouzery]

If I run this test program on fedora 18: https://dl.dropbox.com/u/22600720/Test.hs

ghc Test.hs -threaded

The output for me is: Test: ProtocolError "error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed"

curl and http-conduit on the same computer have no problems therefore I think the CA certificates are present on the computer.

$ ls -lh /etc/ssl/certs/ total 1.5M -rw-r--r-- 1 root root 697K Jan 4 19:10 ca-bundle.crt -rw-r--r-- 1 root root 778K Jan 4 19:10 ca-bundle.trust.crt -rwxr-xr-x 1 root root 610 Mar 18 22:21 make-dummy-cert -rw-r--r-- 1 root root 2.2K Mar 18 22:21 Makefile -rwxr-xr-x 1 root root 829 Mar 18 22:21 renew-dummy-cert

And if in Inconvenience.hs, I change the linux code to this:

SSL.contextSetCAFile ctx "/etc/ssl/certs/ca-bundle.crt"

Then it works.

I tracked this in one of the dependencies of http-conduit: https://github.com/vincenthz/hs-certificate/tree/master/System/Certificate/X509

the Unix.hs version seems to open by hand every file under that certs folder. I guess openSSL will only open single certificates, not bundles, hence the problem?

And finally: $ ls -l /usr/lib/ssl ls: cannot access /usr/lib/ssl: No such file or directory

$ ls -l /etc/ssl/certs lrwxrwxrwx 1 root root 16 Jan 18 21:56 /etc/ssl/certs -> ../pki/tls/certs

[istathar]

I really want this to Just Work™ regardless of what system you're on. If we have to make Fedora class distros a special case (ie, discriminate on distro not OS) then fine, but that further complicates the job of Setup.hs; probing distro accurately is hard.

I do wonder why Fedora does it this way; the openssl documentation clearly describes that individual certificates are loaded on demand whereas 1 meg of certificates in a single file all have to be parsed and loaded by every process.

@emmanueltouzery what happens if you specify a file that's not present on the system? i.e. I'm happy to add the file to be loaded conditionally, but I'm curious what happens if SSL.contextSetCAFile hits file not found.

AfC

[emmanueltouzery]

Btw this is on a CentOS machine, but on that machine I can't do anything besides running ls, I won't run any test programs on it:

$ ls -lh /etc/ssl/certs/ total 1.2M -rw-r--r--. 1 root root 559K Apr 7 2010 ca-bundle.crt -rw-r--r--. 1 root root 636K Apr 7 2010 ca-bundle.trust.crt -rwxr-xr-x. 1 root root 610 Feb 22 00:45 make-dummy-cert -rw-r--r--. 1 root root 2.2K Feb 22 00:45 Makefile

I'll do that test you mentioned, some time later.

[emmanueltouzery]

Bindly settling SSL.contextSetCAFile to a file which may not exist does nothing good:

Test: user error (error:02001002:system library:fopen:No such file or directory)

Otherwise this webpage has some hints as to where find CA certs on different OSes: http://mercurial.selenic.com/wiki/CACertificates

[istathar]

@emmanueltouzery So I guess I don't understand. You said your system has a /etc/ssl/certs directory with a rather large ca-bundle.crt in it, so I'd like to figure out why

SSL.contextSetCADirectory ctx "/etc/ssl/certs"

isn't loading the certs from that file?

[again, not saying this isn't a problem for you, just trying to understand what the right fix is. I brought up a Fedora 19 system and was able to see the exception being thrown. Untyped exceptions aren't very helpful, are they?]

AfC

[emmanueltouzery]

All I can do is confirm that with those two programs on fedora 18: https://dl.dropbox.com/u/22600720/Test1.hs https://dl.dropbox.com/u/22600720/Test2.hs

If I compile them with ghc -threaded, Test1 outputs: `"<meta http-equiv=\"content-type\" content=\"text/html;charset=utf-8\">\n

\n

301 Moved

\n The document has moved\nhere.\r\n \r\n"` And Test2 outputs: `Test2: ProtocolError "error:14090086:SSL routines: SSL3_GET_SERVER_CERTIFICATE:certificate verify failed"` To my mind, but I don't know all that much on the topic, you do not need a complete fedora installation to test this case, but only the /etc/ssl/certs/ folder from Fedora. That folder is 1.5Mb and I'm more than happy to send it to you if you are interested. I also spotted that in fedora19 they'll change some things related to certificates handling but honestly I didn't dig in the issue and I don't know whether it's related: http://fedoraproject.org/wiki/Features/SharedSystemCertificates Even if they changed though the problem would probably keep affecting current RHEL and CentOS releases, which I assume are affected.

[istathar]

@emmanueltouzery Can you test if this works satisfactorily for you now?

AfC

[emmanueltouzery]

Hello, sorry for the delay...

So I tried and I think it's fine. I just want to make sure that it's configured to actually check the certificates and not blindingly accept everything, so I'm just putting the program that I ran just to be 100% sure:

{-# LANGUAGE OverloadedStrings #-}

import OpenSSL (withOpenSSL)
import Network.Http.Client
import System.IO.Streams (InputStream(..))
import Network.URI
import qualified System.IO.Streams as Streams
import qualified Blaze.ByteString.Builder as Builder
import qualified Data.ByteString as B
import qualified OpenSSL.Session as SSL
import qualified System.Info as Sysinfo

main :: IO ()
main = withOpenSSL $ do
    let url = "https://www.gmail.com"
    response <- http' url "" concatHandler $ do
        http GET url
    print response

http' :: B.ByteString -> B.ByteString 
        -> (Response -> InputStream B.ByteString -> IO B.ByteString) 
        -> RequestBuilder a ->  IO B.ByteString
http' url contents responseProcessor requestSpec = withOpenSSL $ do
    c <- establishConnection url
    q <- buildRequest requestSpec
    sendRequest c q $ Streams.write (Just $ Builder.fromByteString contents)
    result <- receiveResponse c responseProcessor
    closeConnection c
    return result

And the output is: `<meta http-equiv=\"content-type\" content=\"text/html;charset=utf-8\">\n

\n

301 Moved

\n The document has moved\nhere.\r\n \r\n` And not the error about the invalid certificate so I think you indeed fixed the problem!

[AdamWill]

Ran across this while working on related stuff. FWIW, the best practice is probably to first try and use the SSL library's default trust store; platform builds of OpenSSL (and the MacOS X variant, and LibreSSL, and whatever) usually have the correct default trust store location compiled in, and you should just use that. The OpenSSL function to enable use of the default trust store is SSL_CTX_set_default_verify_paths() ; I know flat zip about Haskell but I'd expect its wrapper would expose this, openssl wrappers usually do. I believe the best idea ought to be to try and use the default trust store, if that does not work you can look up the common Debian/RHEL paths (why not specifically check the existence of /etc/pki/tls/certs/ca-bundle.crt rather than looking for the /etc/pki/tls directory?) as a fallback. If you do that, you delegate the job of knowing about trust store locations to the platform's OpenSSL build, and if a distro locates it somewhere else or decides to move it, you'll be OK.

[istathar]

@AdamWill Hey, thanks Adam. I didn't know about set_default_verify_paths(), and thanks for the heads up about a more reliable way to implement this logic. I'll see about this in the next week or two.

AfC

P.S. Now you know some Haskell :)

[AdamWill]

No probs :) I wound up writing a more detailed blog post on this stuff (been meaning to do that for a while): https://www.happyassassin.net/2015/01/12/a-note-about-ssltls-trusted-certificate-stores-and-platforms/