Trying to verify a hostname with a self signed cert fails

[b4handjr]

When I try to use --verify-base-url against a host that has a self signed certificate, verify times out with this error:

request = <SubRequest '_verify_url' for <Function test_owned_experiments_page_loads>>, base_url = 'https://localhost'

    @pytest.fixture(scope='session', autouse=True)
    def _verify_url(request, base_url):
        """Verifies the base URL"""
        verify = request.config.option.verify_base_url
        if base_url and verify:
            session = requests.Session()
            retries = Retry(backoff_factor=0.1,
                            status_forcelist=[500, 502, 503, 504])
            session.mount(base_url, HTTPAdapter(max_retries=retries))
>           session.get(base_url)

.tox/integration-test/lib/python3.6/site-packages/pytest_base_url/plugin.py:31: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
.tox/integration-test/lib/python3.6/site-packages/requests/sessions.py:546: in get
    return self.request('GET', url, **kwargs)
.tox/integration-test/lib/python3.6/site-packages/requests/sessions.py:533: in request
    resp = self.send(prep, **send_kwargs)
.tox/integration-test/lib/python3.6/site-packages/requests/sessions.py:646: in send
    r = adapter.send(request, **kwargs)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <requests.adapters.HTTPAdapter object at 0x7f576912eac8>, request = <PreparedRequest [GET]>, stream = False, timeout = <urllib3.util.timeout.Timeout object at 0x7f5769bc3c50>, verify = True, cert = None, proxies = OrderedDict()

    def send(self, request, stream=False, timeout=None, verify=True, cert=None, proxies=None):
        """Sends PreparedRequest object. Returns Response object.

        :param request: The :class:`PreparedRequest <PreparedRequest>` being sent.
        :param stream: (optional) Whether to stream the request content.
        :param timeout: (optional) How long to wait for the server to send
            data before giving up, as a float, or a :ref:`(connect timeout,
            read timeout) <timeouts>` tuple.
        :type timeout: float or tuple or urllib3 Timeout object
        :param verify: (optional) Either a boolean, in which case it controls whether
            we verify the server's TLS certificate, or a string, in which case it
            must be a path to a CA bundle to use
        :param cert: (optional) Any user-provided SSL certificate to be trusted.
        :param proxies: (optional) The proxies dictionary to apply to the request.
        :rtype: requests.Response
        """

        try:
            conn = self.get_connection(request.url, proxies)
        except LocationValueError as e:
            raise InvalidURL(e, request=request)

        self.cert_verify(conn, request.url, verify, cert)
        url = self.request_url(request, proxies)
        self.add_headers(request, stream=stream, timeout=timeout, verify=verify, cert=cert, proxies=proxies)

        chunked = not (request.body is None or 'Content-Length' in request.headers)

        if isinstance(timeout, tuple):
            try:
                connect, read = timeout
                timeout = TimeoutSauce(connect=connect, read=read)
            except ValueError as e:
                # this may raise a string formatting error.
                err = ("Invalid timeout {}. Pass a (connect, read) "
                       "timeout tuple, or a single float to set "
                       "both timeouts to the same value".format(timeout))
                raise ValueError(err)
        elif isinstance(timeout, TimeoutSauce):
            pass
        else:
            timeout = TimeoutSauce(connect=timeout, read=timeout)

        try:
            if not chunked:
                resp = conn.urlopen(
                    method=request.method,
                    url=url,
                    body=request.body,
                    headers=request.headers,
                    redirect=False,
                    assert_same_host=False,
                    preload_content=False,
                    decode_content=False,
                    retries=self.max_retries,
                    timeout=timeout
                )

            # Send the request.
            else:
                if hasattr(conn, 'proxy_pool'):
                    conn = conn.proxy_pool

                low_conn = conn._get_conn(timeout=DEFAULT_POOL_TIMEOUT)

                try:
                    low_conn.putrequest(request.method,
                                        url,
                                        skip_accept_encoding=True)

                    for header, value in request.headers.items():
                        low_conn.putheader(header, value)

                    low_conn.endheaders()

                    for i in request.body:
                        low_conn.send(hex(len(i))[2:].encode('utf-8'))
                        low_conn.send(b'\r\n')
                        low_conn.send(i)
                        low_conn.send(b'\r\n')
                    low_conn.send(b'0\r\n\r\n')

                    # Receive the response from the server
                    try:
                        # For Python 2.7, use buffering of HTTP responses
                        r = low_conn.getresponse(buffering=True)
                    except TypeError:
                        # For compatibility with Python 3.3+
                        r = low_conn.getresponse()

                    resp = HTTPResponse.from_httplib(
                        r,
                        pool=conn,
                        connection=low_conn,
                        preload_content=False,
                        decode_content=False
                    )
                except:
                    # If we hit any problems here, clean up the connection.
                    # Then, reraise so that we can handle the actual exception.
                    low_conn.close()
                    raise

        except (ProtocolError, socket.error) as err:
            raise ConnectionError(err, request=request)

        except MaxRetryError as e:
            if isinstance(e.reason, ConnectTimeoutError):
                # TODO: Remove this in 3.0.0: see #2811
                if not isinstance(e.reason, NewConnectionError):
                    raise ConnectTimeout(e, request=request)

            if isinstance(e.reason, ResponseError):
                raise RetryError(e, request=request)

            if isinstance(e.reason, _ProxyError):
                raise ProxyError(e, request=request)

            if isinstance(e.reason, _SSLError):
                # This branch is for urllib3 v1.22 and later.
>               raise SSLError(e, request=request)
E               requests.exceptions.SSLError: HTTPSConnectionPool(host='localhost', port=443): Max retries exceeded with url: / (Caused by SSLError(SSLError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:852)'),))

.tox/integration-test/lib/python3.6/site-packages/requests/adapters.py:514: SSLError

According to this there is a way for requests to skip this verification. Of course that can lead to other problems.

[davehunt]

So you want to verify the base URL but skip checking the certificate is valid? We could maybe add a --verify-base-url-cert option with associated environment variable to control this.

[b4handjr]

--verify-base-url-cert would require a Certificate, and --verify-base-url wouldn't?

[BeyondEvil]

Maybe an --ignore-cert or --verify-cert instead? @davehunt

[davehunt]

I see how my suggestion is ambiguous, and I didn't really think it through. How about --accept-insecure-base-url?

[BeyondEvil]

Question: What are we really checking with --verify-base-url if not that it has a valid cert? 🤔

Isn't the solution here to just skip the flag? 🤷‍♂

[davehunt]

Question: What are we really checking with --verify-base-url if not that it has a valid cert? 🤔

Isn't the solution here to just skip the flag? 🤷‍♂

We're verifying that it's a valid URL (returns a valid status code). It also (currently) verifies the cert. See https://github.com/pytest-dev/pytest-base-url#verifying-the-base-url

[BeyondEvil]

Question: What are we really checking with --verify-base-url if not that it has a valid cert? 🤔 Isn't the solution here to just skip the flag? 🤷‍♂

We're verifying that it's a valid URL (returns a valid status code). It also (currently) verifies the cert. See https://github.com/pytest-dev/pytest-base-url#verifying-the-base-url

Right, but let's say we don't and the URL isn't valid (regardless of cert) wouldn't the error from the driver tell us that? I'm trying to see the value in a use-case where we only basically "ping" the URL without validating the cert. 🤔

I guess in theory if you're not using webdriver/selenium and doing some API testing, it has some value, but again I would expect whatever library is used for the requests to give an error about URL being unreachable or such. 🤔

[davehunt]

Question: What are we really checking with --verify-base-url if not that it has a valid cert? 🤔 Isn't the solution here to just skip the flag? 🤷‍♂

We're verifying that it's a valid URL (returns a valid status code). It also (currently) verifies the cert. See https://github.com/pytest-dev/pytest-base-url#verifying-the-base-url

Right, but let's say we don't and the URL isn't valid (regardless of cert) wouldn't the error from the driver tell us that? I'm trying to see the value in a use-case where we only basically "ping" the URL without validating the cert. 🤔

Yes, but every single test will fail with the same reason. The value of this check is that it aborts the entire test session if the URL is invalid. When using Selenium, you can also accept insecure certificates.

I guess in theory if you're not using webdriver/selenium and doing some API testing, it has some value, but again I would expect whatever library is used for the requests to give an error about URL being unreachable or such. 🤔

[BeyondEvil]

Yes, but every single test will fail with the same reason. The value of this check is that it aborts the entire test session if the URL is invalid.

Ah, gotcha! Thanks! 👍

[b4handjr]

I can work on a patch for this

[BeyondEvil]

@jrbenny35 Any updates on this? :)

[b4handjr]

@jrbenny35 Any updates on this? :)

I have been a bit busy with work but I will try to get something sooner than later.

[BeyondEvil]

How about instead of another flag, just give the one we have a value?

--verify-base-url=ignore-cert

[BeyondEvil]

@jrbenny35 Is this still a problem in need of a fix?

I know urllib3 has done a couple of new releases surrounding TLS/SSL.

[BeyondEvil]

How about instead of another flag, just give the one we have a value?

--verify-base-url=ignore-cert

@jrbenny35 Is this still a problem in need of a fix?

I know urllib3 has done a couple of new releases surrounding TLS/SSL.

ping @davehunt

[davehunt]

I'll let @jrbenny35 respond. I'm not aware of any other experiencing this issue.

[b4handjr]

I haven't ran into this issue but let me try and see if I can reproduce it again.

[BeyondEvil]

Hey @jrbenny35,

Any luck repro:ing? :)

[b4handjr]

Hey! I have not had any luck. I guess lets close this and if I run into it again I will open it with another log. Thanks for your patience!