Test Failure test_adjustments.py::TestAdjustments::test_service_port for waitress version 3.0.0

[jgarte]

Hi,

I get the following test failure:

/gnu/store/bsmq338l3hm6jw9spkilja3phypg1i54-python-waitress-3.0.0/lib/python3.10/site-packages/waitress/adjustments.py:368: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

host = '127.0.0.1', port = 'http', family = <AddressFamily.AF_UNSPEC: 0>
type = <SocketKind.SOCK_STREAM: 1>, proto = 6
flags = <AddressInfo.AI_PASSIVE: 1>

    def getaddrinfo(host, port, family=0, type=0, proto=0, flags=0):
        """Resolve host and port into list of address info entries.

        Translate the host/port argument into a sequence of 5-tuples that contain
        all the necessary arguments for creating a socket connected to that service.
        host is a domain name, a string representation of an IPv4/v6 address or
        None. port is a string service name such as 'http', a numeric port number or
        None. By passing None as the value of host and port, you can pass NULL to
        the underlying C API.

        The family, type and proto arguments can be optionally specified in order to
        narrow the list of addresses returned. Passing zero as a value for each of
        these arguments selects the full range of results.
        """
        # We override this function since we want to translate the numeric family
        # and socket type values to enum constants.
        addrlist = []
>       for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
E       socket.gaierror: [Errno -8] Servname not supported for ai_socktype

/gnu/store/4r7k7ipiaqkdf4lmnxwmbz0wx2yzygzc-python-3.10.7/lib/python3.10/socket.py:955: gaierror

During handling of the above exception, another exception occurred:

self = <tests.test_adjustments.TestAdjustments testMethod=test_service_port>

    def test_service_port(self):
        if WIN:  # pragma: no cover
            # On Windows this is broken, so we raise a ValueError
            self.assertRaises(
                ValueError,
                self._makeOne,
                listen="127.0.0.1:http",
            )

            return

>       inst = self._makeOne(listen="127.0.0.1:http 0.0.0.0:https")

tests/test_adjustments.py:230: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
tests/test_adjustments.py:107: in _makeOne
    return Adjustments(**kw)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <waitress.adjustments.Adjustments object at 0x7ffff5760550>
kw = {'listen': '127.0.0.1:http 0.0.0.0:https'}, k = 'listen'
v = '127.0.0.1:http 0.0.0.0:https'
enabled_families = <AddressFamily.AF_UNSPEC: 0>, wanted_sockets = []
hp_pairs = [], i = '127.0.0.1:http', host = '127.0.0.1'

    def __init__(self, **kw):
        if "listen" in kw and ("host" in kw or "port" in kw):
            raise ValueError("host or port may not be set if listen is set.")

        if "listen" in kw and "sockets" in kw:
            raise ValueError("socket may not be set if listen is set.")

        if "sockets" in kw and ("host" in kw or "port" in kw):
            raise ValueError("host or port may not be set if sockets is set.")

        if "sockets" in kw and "unix_socket" in kw:
            raise ValueError("unix_socket may not be set if sockets is set")

        if "unix_socket" in kw and ("host" in kw or "port" in kw):
            raise ValueError("unix_socket may not be set if host or port is set")

        if "unix_socket" in kw and "listen" in kw:
            raise ValueError("unix_socket may not be set if listen is set")

        if "send_bytes" in kw:
            warnings.warn(
                "send_bytes will be removed in a future release", DeprecationWarning
            )

        for k, v in kw.items():
            if k not in self._param_map:
                raise ValueError("Unknown adjustment %r" % k)
            setattr(self, k, self._param_map[k](v))

        if not isinstance(self.host, _str_marker) or not isinstance(
            self.port, _int_marker
        ):
            self.listen = [f"{self.host}:{self.port}"]

        enabled_families = socket.AF_UNSPEC

        if not self.ipv4 and not HAS_IPV6:  # pragma: no cover
            raise ValueError(
                "IPv4 is disabled but IPv6 is not available. Cowardly refusing to start."
            )

        if self.ipv4 and not self.ipv6:
            enabled_families = socket.AF_INET

        if not self.ipv4 and self.ipv6 and HAS_IPV6:
            enabled_families = socket.AF_INET6

        wanted_sockets = []
        hp_pairs = []
        for i in self.listen:
            if ":" in i:
                (host, port) = i.rsplit(":", 1)

                # IPv6 we need to make sure that we didn't split on the address
                if "]" in port:  # pragma: nocover
                    (host, port) = (i, str(self.port))
            else:
                (host, port) = (i, str(self.port))

            if WIN:  # pragma: no cover
                try:
                    # Try turning the port into an integer
                    port = int(port)

                except Exception:
                    raise ValueError(
                        "Windows does not support service names instead of port numbers"
                    )

            try:
                if "[" in host and "]" in host:  # pragma: nocover
                    host = host.strip("[").rstrip("]")

                if host == "*":
                    host = None

                for s in socket.getaddrinfo(
                    host,
                    port,
                    enabled_families,
                    socket.SOCK_STREAM,
                    socket.IPPROTO_TCP,
                    socket.AI_PASSIVE,
                ):
                    (family, socktype, proto, _, sockaddr) = s

                    # It seems that getaddrinfo() may sometimes happily return
                    # the same result multiple times, this of course makes
                    # bind() very unhappy...
                    #
                    # Split on %, and drop the zone-index from the host in the
                    # sockaddr. Works around a bug in OS X whereby
                    # getaddrinfo() returns the same link-local interface with
                    # two different zone-indices (which makes no sense what so
                    # ever...) yet treats them equally when we attempt to bind().
                    if (
                        sockaddr[1] == 0
                        or (sockaddr[0].split("%", 1)[0], sockaddr[1]) not in hp_pairs
                    ):
                        wanted_sockets.append((family, socktype, proto, sockaddr))
                        hp_pairs.append((sockaddr[0].split("%", 1)[0], sockaddr[1]))

            except Exception:
>               raise ValueError("Invalid host/port specified.")
E               ValueError: Invalid host/port specified.

/gnu/store/bsmq338l3hm6jw9spkilja3phypg1i54-python-waitress-3.0.0/lib/python3.10/site-packages/waitress/adjustments.py:395: ValueError
=============================== warnings summary ===============================
tests/test_adjustments.py::TestAdjustments::test_goodvars
  /gnu/store/bsmq338l3hm6jw9spkilja3phypg1i54-python-waitress-3.0.0/lib/python3.10/site-packages/waitress/adjustments.py:312: DeprecationWarning: send_bytes will be removed in a future release
    warnings.warn(

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html

---------- coverage: platform linux, python 3.10.7-final-0 -----------
Name                            Stmts   Miss     Cover   Missing
----------------------------------------------------------------
src/waitress/__init__.py           11      0   100.00%
src/waitress/__main__.py            0      0   100.00%
src/waitress/adjustments.py       181      0   100.00%
src/waitress/buffers.py           212      0   100.00%
src/waitress/channel.py           257      0   100.00%
src/waitress/compat.py             10      0   100.00%
src/waitress/parser.py            199      0   100.00%
src/waitress/proxy_headers.py     162      0   100.00%
src/waitress/receiver.py          111      0   100.00%
src/waitress/rfc7230.py            26      0   100.00%
src/waitress/runner.py             69      0   100.00%
src/waitress/server.py            177      0   100.00%
src/waitress/task.py              325      0   100.00%
src/waitress/trigger.py            56      0   100.00%
src/waitress/utilities.py         114      0   100.00%
src/waitress/wasyncore.py         379      0   100.00%
----------------------------------------------------------------
TOTAL                            2289      0   100.00%

=========================== short test summary info ============================
FAILED tests/test_adjustments.py::TestAdjustments::test_service_port - ValueE...

I also noticed this:

tests/test_wasyncore.py::Test__strerror::test_gardenpath PASSED          [ 94%]
tests/test_wasyncore.py::Test__strerror::test_unknown PASSED             [ 94%]

WDYT?

[kgaughan]

This isn't an issue with Waitress. The issue here is with the platform you're running the tests in. It appears to lack an /etc/protocols file. Have a read of the documentation on getaddrinfo: part of its job is to resolve protocol/service names (such as 'http') to numeric port numbers. It can't do that without /etc/protocols, hence the "Servname not supported for ai_socktype" error.

You look to be running the tests on a highly unusual distro, possibly one meant for containers or with some kind of isolation mechanisms built into how it does packaging. I'm basing that off of the unusual paths that appear in the error.

Either way, you need to fix your system to ensure /etc/protocols exists.

[kgaughan]

I did a little more digging and it looks like you're using Guix. IIRC, that has some built-in isolation that you're almost certainly tripping up on. I think you need to ask for help on a Guix-related forum.

[jgarte]

Hi,

Yes, I was using Guix

Should waitress provide a test for test_service_port that is more agnostic so that the said test can pass without needing to require /etc/protocols?

I see also that nixpkgs has tests disabled for their waitress package. I imagine probably because of this very issue? Nix uses that same approach as Guix to package management with regards to using foo/store/75v854mmrclhy205z7q15ww0ipdirwfq-waitress-3.0.0 paths.

[kgaughan]

That issue is more one for the packager to solve. I don't know enough about Nix or Guix to say how the issue would be solved with either of them. Nix at least looks to provide some support for overlayfs, so if you want to get the tests working, that might be an approach to take.

[digitalresistor]

Sorry, but I am not going to cater towards systems where defaults are simply missing. There are also various tests for testing by port only.

I don't want to add a bunch of code to try and figure out if /etc/protocols (or other system equivalents) exists to make getaddrinfo work as expected, before running a test.

Your test environment is broken. Please fix it.