search

input-output-hk / cardano-explorer

Backend solution powering the cardano-explorer. :warning: See disclaimer below. :warning:
20 stars 11 forks source link

[Build] OpenSSL.SSL.Error: [('SSL routines', 'ssl3_get_server_certificate', 'certificate verify failed')] #275

Closed atomlab closed 4 years ago

atomlab commented 4 years ago

I try to build cardano-explorer in virtual machine. Just Ubuntu 18.04. Without docker.

$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 18.04.4 LTS
Release:    18.04
Codename:   bionic

Setup enviroment

apt-get install -y sudo bzip2 curl git xz-utils
useradd -ms /bin/bash cardano && mkdir /nix /etc/nix && chown cardano /nix
su cardano
curl https://nixos.org/nix/install | sh
export PATH=/home/cardano/.nix-profile/bin/:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin

Build

git clone https://github.com/input-output-hk/cardano-explorer/
cd cardano-explorer/
nix-build -A cardano-explorer-node -o explorer-node

output

...
shrinking RPATHs of ELF executables and libraries in /nix/store/prpz62y09m2jzc68asrrs98rzgbm8ajc-python3.6-pyOpenSSL-18.0.0
strip is /nix/store/6dknibd5sx8mjaxic0xnvxmpsn06ygc5-binutils-2.30/bin/strip
stripping (with command strip and flags -S) in /nix/store/prpz62y09m2jzc68asrrs98rzgbm8ajc-python3.6-pyOpenSSL-18.0.0/lib
patching script interpreter paths in /nix/store/prpz62y09m2jzc68asrrs98rzgbm8ajc-python3.6-pyOpenSSL-18.0.0
checking for references to /build in /nix/store/prpz62y09m2jzc68asrrs98rzgbm8ajc-python3.6-pyOpenSSL-18.0.0...
strip is /nix/store/6dknibd5sx8mjaxic0xnvxmpsn06ygc5-binutils-2.30/bin/strip
running install tests
============================= test session starts ==============================
platform linux -- Python 3.6.6, pytest-3.7.4, py-1.5.4, pluggy-0.7.1
OpenSSL: b'OpenSSL 1.0.2p  14 Aug 2018'
cryptography: 2.3.1
rootdir: /build/pyOpenSSL-18.0.0, inifile: setup.cfg
plugins: flaky-3.1.0
collected 496 items

tests/test_crypto.py ................................................... [ 10%]
........................................................................ [ 24%]
........................................................................ [ 39%]
...............................................................          [ 52%]
tests/test_debug.py .                                                    [ 52%]
tests/test_rand.py ....                                                  [ 53%]
tests/test_ssl.py .............s.........s.......s.....s..s..s..s....... [ 63%]
.............FFF...s...............s.................................... [ 78%]
..s.................................s.....s..............s.............. [ 92%]
.................................                                        [ 99%]
tests/test_tsafe.py .                                                    [ 99%]
tests/test_util.py .                                                     [100%]

=================================== FAILURES ===================================
____________________ TestContext.test_add_extra_chain_cert _____________________

self = <tests.test_ssl.TestContext object at 0x7fffe8cd91d0>
tmpdir = local('/build/pytest-of-nixbld/pytest-0/test_add_extra_chain_cert0')

    def test_add_extra_chain_cert(self, tmpdir):
        """
            `Context.add_extra_chain_cert` accepts an `X509`
            instance to add to the certificate chain.

            See `_create_certificate_chain` for the details of the
            certificate chain tested.

            The chain is tested by starting a server with scert and connecting
            to it with a client which trusts cacert and requires verification to
            succeed.
            """
        chain = _create_certificate_chain()
        [(cakey, cacert), (ikey, icert), (skey, scert)] = chain

        # Dump the CA certificate to a file because that's the only way to load
        # it as a trusted CA in the client context.
        for cert, name in [(cacert, 'ca.pem'),
                           (icert, 'i.pem'),
                           (scert, 's.pem')]:
            with tmpdir.join(name).open('w') as f:
                f.write(dump_certificate(FILETYPE_PEM, cert).decode('ascii'))

        for key, name in [(cakey, 'ca.key'),
                          (ikey, 'i.key'),
                          (skey, 's.key')]:
            with tmpdir.join(name).open('w') as f:
                f.write(dump_privatekey(FILETYPE_PEM, key).decode('ascii'))

        # Create the server context
        serverContext = Context(TLSv1_METHOD)
        serverContext.use_privatekey(skey)
        serverContext.use_certificate(scert)
        # The client already has cacert, we only need to give them icert.
        serverContext.add_extra_chain_cert(icert)

        # Create the client
        clientContext = Context(TLSv1_METHOD)
        clientContext.set_verify(
            VERIFY_PEER | VERIFY_FAIL_IF_NO_PEER_CERT, verify_cb)
        clientContext.load_verify_locations(str(tmpdir.join("ca.pem")))

        # Try it out.
>       self._handshake_test(serverContext, clientContext)

tests/test_ssl.py:1370:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
tests/test_ssl.py:1248: in _handshake_test
    s.do_handshake()
/nix/store/prpz62y09m2jzc68asrrs98rzgbm8ajc-python3.6-pyOpenSSL-18.0.0/lib/python3.6/site-packages/OpenSSL/SSL.py:1907: in do_handshake
    self._raise_ssl_error(self._ssl, result)
/nix/store/prpz62y09m2jzc68asrrs98rzgbm8ajc-python3.6-pyOpenSSL-18.0.0/lib/python3.6/site-packages/OpenSSL/SSL.py:1639: in _raise_ssl_error
    _raise_current_error()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

exception_type = <class 'OpenSSL.SSL.Error'>

    def exception_from_error_queue(exception_type):
        """
        Convert an OpenSSL library failure into a Python exception.

        When a call to the native OpenSSL library fails, this is usually signalled
        by the return value, and an error code is stored in an error queue
        associated with the current thread. The err library provides functions to
        obtain these error codes and textual error messages.
        """
        errors = []

        while True:
            error = lib.ERR_get_error()
            if error == 0:
                break
            errors.append((
                text(lib.ERR_lib_error_string(error)),
                text(lib.ERR_func_error_string(error)),
                text(lib.ERR_reason_error_string(error))))

>       raise exception_type(errors)
E       OpenSSL.SSL.Error: [('SSL routines', 'ssl3_get_server_certificate', 'certificate verify failed')]

/nix/store/prpz62y09m2jzc68asrrs98rzgbm8ajc-python3.6-pyOpenSSL-18.0.0/lib/python3.6/site-packages/OpenSSL/_util.py:54: Error
______________ TestContext.test_use_certificate_chain_file_bytes _______________

self = <tests.test_ssl.TestContext object at 0x7fffe9c354a8>
tmpfile = b'/build/pytest-of-nixbld/pytest-0/tmp9au27i8z'

    def test_use_certificate_chain_file_bytes(self, tmpfile):
        """
            ``Context.use_certificate_chain_file`` accepts the name of a file (as
            an instance of ``bytes``) to specify additional certificates to use to
            construct and verify a trust chain.
            """
        self._use_certificate_chain_file_test(
>           tmpfile + NON_ASCII.encode(getfilesystemencoding())
        )

tests/test_ssl.py:1417:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
tests/test_ssl.py:1408: in _use_certificate_chain_file_test
    self._handshake_test(serverContext, clientContext)
tests/test_ssl.py:1248: in _handshake_test
    s.do_handshake()
/nix/store/prpz62y09m2jzc68asrrs98rzgbm8ajc-python3.6-pyOpenSSL-18.0.0/lib/python3.6/site-packages/OpenSSL/SSL.py:1907: in do_handshake
    self._raise_ssl_error(self._ssl, result)
/nix/store/prpz62y09m2jzc68asrrs98rzgbm8ajc-python3.6-pyOpenSSL-18.0.0/lib/python3.6/site-packages/OpenSSL/SSL.py:1639: in _raise_ssl_error
    _raise_current_error()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

exception_type = <class 'OpenSSL.SSL.Error'>

    def exception_from_error_queue(exception_type):
        """
        Convert an OpenSSL library failure into a Python exception.

        When a call to the native OpenSSL library fails, this is usually signalled
        by the return value, and an error code is stored in an error queue
        associated with the current thread. The err library provides functions to
        obtain these error codes and textual error messages.
        """
        errors = []

        while True:
            error = lib.ERR_get_error()
            if error == 0:
                break
            errors.append((
                text(lib.ERR_lib_error_string(error)),
                text(lib.ERR_func_error_string(error)),
                text(lib.ERR_reason_error_string(error))))

>       raise exception_type(errors)
E       OpenSSL.SSL.Error: [('SSL routines', 'ssl3_get_server_certificate', 'certificate verify failed')]

/nix/store/prpz62y09m2jzc68asrrs98rzgbm8ajc-python3.6-pyOpenSSL-18.0.0/lib/python3.6/site-packages/OpenSSL/_util.py:54: Error
_____________ TestContext.test_use_certificate_chain_file_unicode ______________

self = <tests.test_ssl.TestContext object at 0x7fffe872a390>
tmpfile = b'/build/pytest-of-nixbld/pytest-0/tmpo8fynk0m'

    def test_use_certificate_chain_file_unicode(self, tmpfile):
        """
            ``Context.use_certificate_chain_file`` accepts the name of a file (as
            an instance of ``unicode``) to specify additional certificates to use
            to construct and verify a trust chain.
            """
        self._use_certificate_chain_file_test(
>           tmpfile.decode(getfilesystemencoding()) + NON_ASCII
        )

tests/test_ssl.py:1427:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
tests/test_ssl.py:1408: in _use_certificate_chain_file_test
    self._handshake_test(serverContext, clientContext)
tests/test_ssl.py:1248: in _handshake_test
    s.do_handshake()
/nix/store/prpz62y09m2jzc68asrrs98rzgbm8ajc-python3.6-pyOpenSSL-18.0.0/lib/python3.6/site-packages/OpenSSL/SSL.py:1907: in do_handshake
    self._raise_ssl_error(self._ssl, result)
/nix/store/prpz62y09m2jzc68asrrs98rzgbm8ajc-python3.6-pyOpenSSL-18.0.0/lib/python3.6/site-packages/OpenSSL/SSL.py:1639: in _raise_ssl_error
    _raise_current_error()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

exception_type = <class 'OpenSSL.SSL.Error'>

    def exception_from_error_queue(exception_type):
        """
        Convert an OpenSSL library failure into a Python exception.

        When a call to the native OpenSSL library fails, this is usually signalled
        by the return value, and an error code is stored in an error queue
        associated with the current thread. The err library provides functions to
        When a call to the native OpenSSL library fails, this is usually signalled
        obtain these error codes and textual error messages.
        """
        errors = []

        while True:
            error = lib.ERR_get_error()
            if error == 0:
                break
            errors.append((
                text(lib.ERR_lib_error_string(error)),
                text(lib.ERR_func_error_string(error)),
                text(lib.ERR_reason_error_string(error))))

>       raise exception_type(errors)
E       OpenSSL.SSL.Error: [('SSL routines', 'ssl3_get_server_certificate', 'certificate verify failed')]

/nix/store/prpz62y09m2jzc68asrrs98rzgbm8ajc-python3.6-pyOpenSSL-18.0.0/lib/python3.6/site-packages/OpenSSL/_util.py:54: Error
===Flaky Test Report===

test_gmtime_adj_notBefore passed 1 out of the required 1 times. Success!
test_gmtime_adj_notAfter passed 1 out of the required 1 times. Success!
test_export_text passed 1 out of the required 1 times. Success!

===End Flaky Test Report===
=============== 3 failed, 480 passed, 13 skipped in 5.97 seconds ===============
builder for '/nix/store/ri6llpk8jw1l2vvg0v8hwswqqfhi9yv0-python3.6-pyOpenSSL-18.0.0.drv' failed with exit code 1
cannot build derivation '/nix/store/kvpwsdgrkxy2y90mavhda6q1858cazpf-python3.6-urllib3-1.23.drv': 1 dependencies couldn't be built
building '/nix/store/afld99qn9yszhk14rf01qa5gljb4ph3b-python3.6-webencodings-0.5.1.drv'...
cannot build derivation '/nix/store/0hfdcdk5njhfpx2r6zy9nng54cnvw0d5-python3.6-requests-2.19.1.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/mi0880zwhjxzywkvk9ykqx2cm8kjcxj7-python3.6-Sphinx-1.7.8.drv': 1 dependencies couldn't be built
building '/nix/store/zli8hgams1s7rv5mafs5dilhxdj1rc4b-xproto-7.0.31.drv'...
cannot build derivation '/nix/store/28bh34ckdajvsf2hwhvb43zaakrbm93k-ghc-8.6.5.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/9wf05g9fpqrd82jdyls0wk9j52hya68x-cardano-explorer-node-1.3.0-exe-cardano-explorer-node-config.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/ylivmds5b06vry8dvad918zkq3x2xbar-cardano-explorer-node-1.3.0-exe-cardano-explorer-node-ghc-8.6.5.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/5q0d62dmiq4mm0m814hwcfa83fm1gb8r-default-Setup.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/ww68y37shw77v5dgfrx16abyw1anyza2-cardano-explorer-node-1.3.0-exe-cardano-explorer-node.drv': 1 dependencies couldn't be built
error: build of '/nix/store/ww68y37shw77v5dgfrx16abyw1anyza2-cardano-explorer-node-1.3.0-exe-cardano-explorer-node.drv' failed
craigem commented 4 years ago

Heya @atomlab - I've been unable to replicate this off the current master (and also on NixOS, not Ubuntu). So to start narrowing down our differences, can you please advise which commit you built against?

Thanks!

disassembler commented 4 years ago

Can you share what your nix.conf contains and verify is has hydra.iohk.io? We are aware of a breakage in python build tooling, but we should have that in our binary cache.

atomlab commented 4 years ago

Heya @atomlab - I've been unable to replicate this off the current master (and also on NixOS, not Ubuntu). So to start narrowing down our differences, can you please advise which commit you built against?

Thanks!

last commit

# cd ./cardano-explorer/
cardano-explorer# git log -1
commit 61c2a152e280a4c511a99b40e056a7baee7e74e1 (HEAD -> master, origin/master, origin/HEAD)
Merge: f5bec6d 7363606
Author: Erik de Castro Lopo <erikd@mega-nerd.com>
Date:   Thu Feb 20 07:16:10 2020 +1100

    Merge pull request #270 from input-output-hk/update_setup_doc

    Update building-running doc - new socket path
atomlab commented 4 years ago

Can you share what your nix.conf contains and verify is has hydra.iohk.io? We are aware of a breakage in python build tooling, but we should have that in our binary cache.

I haven't nix.conf

# ls /etc/nix/nix.conf
ls: cannot access '/etc/nix/nix.conf': No such file or directory

It my mistake. Sorry. I have just copied 

cp ./docker/nix.conf /etc/nix/nix.conf

And build have finished success.

$ nix-build -A cardano-explorer-node -o explorer-node
these paths will be fetched (2.64 MiB download, 17.08 MiB unpacked):
  /nix/store/9my3g5qx4zk20pys5kja3s8q2x8lihxi-cardano-explorer-node-1.3.0-exe-cardano-explorer-node
copying path '/nix/store/9my3g5qx4zk20pys5kja3s8q2x8lihxi-cardano-explorer-node-1.3.0-exe-cardano-explorer-node' from 'https://hydra.iohk.io'...
/nix/store/9my3g5qx4zk20pys5kja3s8q2x8lihxi-cardano-explorer-node-1.3.0-exe-cardano-explorer-node

$ ls -l explorer-node
lrwxrwxrwx 1 cardano cardano 97 feb 25 12:16 explorer-node -> /nix/store/9my3g5qx4zk20pys5kja3s8q2x8lihxi-cardano-explorer-node-1.3.0-exe-cardano-explorer-node

$ explorer-node/bin/cardano-explorer-node --help
Usage: cardano-explorer-node --config FILEPATH --genesis-file FILEPATH
                             --socket-path FILEPATH --schema-dir FILEPATH
  Cardano explorer database node.

Available options:
  --config FILEPATH        Path to the explorer node config file
  --genesis-file FILEPATH  Path to the genesis JSON file
  --socket-path FILEPATH   Path to a cardano-node socket
  --schema-dir FILEPATH    The directory containing the migrations.
  -h,--help                Show this help text