pytest-dev / pytest-testinfra

Testinfra test your infrastructures
https://testinfra.readthedocs.io
Apache License 2.0
2.36k stars 356 forks source link

Falsely reporting listening state? #389

Open aberends opened 5 years ago

aberends commented 5 years ago

Hi there,

I am using:

$ cat /etc/redhat-release 
CentOS Linux release 7.5.1804 (Core) 

$ pip freeze
ansible==2.7.2
asn1crypto==0.24.0
atomicwrites==1.2.1
attrs==18.2.0
bcrypt==3.1.4
cffi==1.11.5
cryptography==2.4.1
enum34==1.1.6
funcsigs==1.0.2
idna==2.7
importlib==1.0.4
ipaddress==1.0.22
Jinja2==2.10
MarkupSafe==1.1.0
more-itertools==4.3.0
paramiko==2.4.2
pathlib2==2.3.2
pluggy==0.8.0
py==1.7.0
pyasn1==0.4.4
pycparser==2.19
PyNaCl==1.3.0
pytest==4.0.0
PyYAML==3.13
scandir==1.9.0
six==1.11.0
testinfra==1.17.0

$ ss -ltn 'sport = :80'
State      Recv-Q Send-Q Local Address:Port               Peer Address:Port              
LISTEN     0      128          *:80                       *:*

$ ss -ltn 'sport = :22'
State      Recv-Q Send-Q Local Address:Port               Peer Address:Port              
LISTEN     0      128          *:22                       *:*                  
LISTEN     0      128         :::22                      :::*

$ python --version
Python 2.7.5

$ python
>>> import testinfra
>>> host = testinfra.host.Host.get_host('local://')
>>> for i in host.socket.get_listening_sockets():
...     print i
... 
unix:///run/lvm/lvmpolld.socket
unix:///run/systemd/journal/stdout
unix:///run/dbus/system_bus_socket
unix://private/tlsmgr
unix:///run/systemd/private
unix://private/rewrite
unix:///tmp/.s.PGSQL.5432
unix://private/bounce
unix://private/defer
unix://public/pickup
unix://private/trace
unix://private/verify
unix://public/cleanup
unix://private/proxymap
unix://public/qmgr
unix://public/flush
unix://public/showq
unix://private/proxywrite
unix://private/smtp
unix://private/relay
unix://private/error
unix://private/retry
unix://private/discard
unix://private/local
unix://private/virtual
unix://private/lmtp
unix://private/anvil
unix://private/scache
unix:///var/run/postgresql/.s.PGSQL.5432
unix:///run/lvm/lvmetad.socket
udp://127.0.0.1:323
udp://:::11211
udp://0.0.0.0:11211
udp://::1:323
udp://:::11211
tcp://:::25672
tcp://0.0.0.0:25672
tcp://:::11211
tcp://0.0.0.0:11211
tcp://:::80
tcp://0.0.0.0:80
tcp://:::4369
tcp://0.0.0.0:4369
tcp://127.0.0.1:8050
tcp://127.0.0.1:8051
tcp://:::22
tcp://0.0.0.0:22
tcp://127.0.0.1:5432
tcp://127.0.0.1:25
tcp://:::5672
tcp://:::11211
tcp://:::4369
tcp://:::22
tcp://::1:5432
tcp://::1:25

In the output of the code above, I notice that port 80 is reported to be in listening state on IPv4. In reality it is not. Hence, as far as I am able to understand the code, I also get all True's on the following:

>>> nginx = host.socket('tcp://0.0.0.0:80')
>>> nginx.is_listening
True
>>> ssh = host.socket('tcp://0.0.0.0:22')
>>> ssh.is_listening
True
>>> nginx = host.socket('tcp://:::80')
>>> nginx.is_listening
True
>>> ssh = host.socket('tcp://:::22')
>>> ssh.is_listening
True

My question is: did I hit a bug here, or do I misunderstand the way Testinfra works with sockets. I have configured Nginx in such a way that it is only allowed to listen on IPv4 and not on IPv4. So, I want to enforce this specification in code with:

assert not host.socket('tcp://:::80').is_listening
aberends commented 5 years ago

Correction: in the text above I said IPv4 twice. The second one must be IPv6.

philpep commented 5 years ago

Hi, looks like a bug, thanks for reporting it! Can you paste your nginx config (especially "server" bloc with "listen" statement) ?

In the meantime we have https://github.com/philpep/testinfra/issues/234 , did you tried to connect with ipv4 on your nginx ? (curl http://127.0.0.1 return "connection refused" ?).

Thanks!

aberends commented 5 years ago

First of all the nginx.conf

# cat /etc/nginx/nginx.conf
#user awx;

worker_processes  1;

error_log  /var/log/nginx/error_log warn;
pid        /var/run/nginx.pid;

events {
    worker_connections  1024;
}

http {
    include       /etc/nginx/mime.types;
    default_type  application/octet-stream;

    log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                      '$status $body_bytes_sent "$http_referer" '
                      '"$http_user_agent" "$http_x_forwarded_for"';

    access_log  /var/log/nginx/access_log  main;

    map $http_upgrade $connection_upgrade {
        default upgrade;
        ''      close;
    }

    sendfile        on;
    #tcp_nopush     on;
    #gzip  on;

    upstream uwsgi {
        server 127.0.0.1:8050;
        }

    upstream daphne {
        server 127.0.0.1:8051;
    }

    server {
 #       listen 8052 default_server;
#Modified to listen on port 80
        listen 80 default_server;

        # If you have a domain name, this is where to add it
        server_name _;
        keepalive_timeout 65;

        # HSTS (ngx_http_headers_module is required) (15768000 seconds = 6 months)
        add_header Strict-Transport-Security max-age=15768000;

        location /static/ {
            alias /opt/awx/static/;
        }

        location /favicon.ico { alias /opt/awx/static/favicon.ico; }

        location /websocket {
            # Pass request to the upstream alias
            proxy_pass http://daphne;
            # Require http version 1.1 to allow for upgrade requests
            proxy_http_version 1.1;
            # We want proxy_buffering off for proxying to websockets.
            proxy_buffering off;
            # http://en.wikipedia.org/wiki/X-Forwarded-For
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            # enable this if you use HTTPS:
            proxy_set_header X-Forwarded-Proto https;
            # pass the Host: header from the client for the sake of redirects
            proxy_set_header Host $http_host;
            # We've set the Host header, so we don't need Nginx to muddle
            # about with redirects
            proxy_redirect off;
            # Depending on the request value, set the Upgrade and
            # connection headers
            proxy_set_header Upgrade $http_upgrade;
            proxy_set_header Connection $connection_upgrade;
        }

        location / {
            # Add trailing / if missing
            rewrite ^(.*)$http_host(.*[^/])$ $1$http_host$2/ permanent;
            uwsgi_read_timeout 120s;
            uwsgi_pass uwsgi;
            include /etc/nginx/uwsgi_params;
        }
    }
}
aberends commented 5 years ago

The curl from the localhost to the loopback interface:

# curl http://127.0.0.1
<!DOCTYPE html>
<html>

<head>
.. remainder of the AWX (Tower) page skipped ..
aberends commented 5 years ago

And the same thing for IPv6:

# curl -g -6 http://[::1]
curl: (7) Failed connect to ::1:80; Connection refused