search

elastic / logstash

Logstash - transport and process your logs, events, or other data
https://www.elastic.co/products/logstash
Other
14.14k stars 3.49k forks source link

logstash-plugin install logstash-codec-sflow faild on IPv6/NAT64 #9309

Open ruben-herold opened 6 years ago

ruben-herold commented 6 years ago

For all general issues, please provide the following details for fast resolution:

Server has only IPv6 with NAT64 to reach IPv4 targets.

/usr/share/logstash/bin/logstash-plugin install logstash-codec-sflow

leads to:

ERROR: Something went wrong when installing logstash-codec-sflow, message: Network is unreachable - Failed to open TCP connection to artifacts.elastic.co:443 (Network is unreachable - No message available)

I have no system with nativ IPv4 to start logstash-plugin so it seems to be impossible to install plugins.

andrewvc commented 6 years ago

I've not used NAT64 before. What do we need to change to support this? AFAIK NAT64 should translate things cleanly no?

ruben-herold commented 6 years ago

NAT64 would translate cleanly but if you take a look at artifacts.elastic.co you will see:

ruben@schleppy:[~] > host artifacts.elastic.co
artifacts.elastic.co is an alias for dualstack.download-colb-770446651.us-east-1.elb.amazonaws.com.
dualstack.download-colb-770446651.us-east-1.elb.amazonaws.com has address 54.235.82.130
dualstack.download-colb-770446651.us-east-1.elb.amazonaws.com has address 23.21.67.46
dualstack.download-colb-770446651.us-east-1.elb.amazonaws.com has address 184.73.245.233
dualstack.download-colb-770446651.us-east-1.elb.amazonaws.com has address 184.72.242.47
dualstack.download-colb-770446651.us-east-1.elb.amazonaws.com has address 54.235.171.120
dualstack.download-colb-770446651.us-east-1.elb.amazonaws.com has address 23.23.109.100
dualstack.download-colb-770446651.us-east-1.elb.amazonaws.com has IPv6 address 2406:da00:ff00::36eb:ab78
dualstack.download-colb-770446651.us-east-1.elb.amazonaws.com has IPv6 address 2406:da00:ff00::b848:f22f
dualstack.download-colb-770446651.us-east-1.elb.amazonaws.com has IPv6 address 2406:da00:ff00::1717:6d64
dualstack.download-colb-770446651.us-east-1.elb.amazonaws.com has IPv6 address 2406:da00:ff00::b849:f5e9
dualstack.download-colb-770446651.us-east-1.elb.amazonaws.com has IPv6 address 2406:da00:ff00::1715:432e
dualstack.download-colb-770446651.us-east-1.elb.amazonaws.com has IPv6 address 2406:da00:ff00::36eb:5282
ruben@schleppy:[~] >

That there are ipv6 available so normaly nat64 will have nothing to do. I can connect with curl to the url:

curl -I https://artifacts.elastic.co
HTTP/1.1 302 Moved Temporarily
Content-Length: 169
Content-Type: text/html
Date: Tue, 10 Apr 2018 18:51:23 GMT
Location: https://www.elastic.co/downloads/
Server: nginx/1.4.6 (Ubuntu)
x-ngx-hostname: www03
Connection: keep-alive

but_

/usr/share/logstash/bin/logstash-plugin install logstash-codec-sflow

leads to:

ERROR: Something went wrong when installing logstash-codec-sflow, message: Network is unreachable - Failed to open TCP connection to artifacts.elastic.co:443 (Network is unreachable - No message available)

Seems something with ipv6 is not working ....

skrivy commented 5 years ago

Quick fix/hack - return empty answer on A requests on your DNS server and let only IPv6 flow (either via NAT64 or directly).

We achieved this by installing powerdns recursor (our version is 4.8.18-1), which supports lua.

/etc/powerdns/recursor.conf:

lua-dns-script=/etc/powerdns/dns-script.lua

/etc/powerdns/dns-script.lua:

function preresolve(dq)
    if dq.qtype == pdns.A then
        return true;
    end

    return false;
end

Result - successful installation.

root@server:~ $ /usr/share/logstash/bin/logstash-plugin install logstash-filter-prune
Validating logstash-filter-prune
Installing logstash-filter-prune
Installation successful

For some reason it was trying to connect to IPv4 to IPv6 mapped address and crashed on it. Even though IPv6 addresses were available and it successfully connected via IPv6. I don't understand why.

connect(25, {sa_family=AF_INET6, sin6_port=htons(0), inet_pton(AF_INET6, "2406:da00:ff00::36e1:d64a", &sin6_addr), sin6_flowinfo=htonl(0), sin6_scope_id=0}, 28) = 0
connect(25, {sa_family=AF_UNSPEC, sa_data="\0\0\0\0\0\0\0\0\0\0\0\0\0\0"}, 16) = 0
connect(25, {sa_family=AF_INET6, sin6_port=htons(0), inet_pton(AF_INET6, "2406:da00:ff00::6b15:ca0f", &sin6_addr), sin6_flowinfo=htonl(0), sin6_scope_id=0}, 28) = 0
connect(25, {sa_family=AF_UNSPEC, sa_data="\0\0\0\0\0\0\0\0\0\0\0\0\0\0"}, 16) = 0
connect(25, {sa_family=AF_INET6, sin6_port=htons(0), inet_pton(AF_INET6, "2406:da00:ff00::6b15:7fb8", &sin6_addr), sin6_flowinfo=htonl(0), sin6_scope_id=0}, 28) = 0
connect(25, {sa_family=AF_UNSPEC, sa_data="\0\0\0\0\0\0\0\0\0\0\0\0\0\0"}, 16) = 0
connect(25, {sa_family=AF_INET6, sin6_port=htons(0), inet_pton(AF_INET6, "2406:da00:ff00::6b15:ed5f", &sin6_addr), sin6_flowinfo=htonl(0), sin6_scope_id=0}, 28) = 0
connect(25, {sa_family=AF_UNSPEC, sa_data="\0\0\0\0\0\0\0\0\0\0\0\0\0\0"}, 16) = 0
connect(25, {sa_family=AF_INET6, sin6_port=htons(0), inet_pton(AF_INET6, "2406:da00:ff00::36e1:dd80", &sin6_addr), sin6_flowinfo=htonl(0), sin6_scope_id=0}, 28) = 0
connect(25, {sa_family=AF_UNSPEC, sa_data="\0\0\0\0\0\0\0\0\0\0\0\0\0\0"}, 16) = 0
connect(25, {sa_family=AF_INET6, sin6_port=htons(0), inet_pton(AF_INET6, "2406:da00:ff00::36e1:d00d", &sin6_addr), sin6_flowinfo=htonl(0), sin6_scope_id=0}, 28) = 0
connect(25, {sa_family=AF_UNSPEC, sa_data="\0\0\0\0\0\0\0\0\0\0\0\0\0\0"}, 16) = 0
connect(25, {sa_family=AF_INET6, sin6_port=htons(0), inet_pton(AF_INET6, "2406:da00:ff00::36eb:5282", &sin6_addr), sin6_flowinfo=htonl(0), sin6_scope_id=0}, 28) = 0
connect(25, {sa_family=AF_UNSPEC, sa_data="\0\0\0\0\0\0\0\0\0\0\0\0\0\0"}, 16) = 0
connect(25, {sa_family=AF_INET6, sin6_port=htons(0), inet_pton(AF_INET6, "2406:da00:ff00::6b15:e20b", &sin6_addr), sin6_flowinfo=htonl(0), sin6_scope_id=0}, 28) = 0
connect(25, {sa_family=AF_UNSPEC, sa_data="\0\0\0\0\0\0\0\0\0\0\0\0\0\0"}, 16) = 0
connect(25, {sa_family=AF_INET, sin_port=htons(0), sin_addr=inet_addr("107.21.237.95")}, 16) = -1 ENETUNREACH (Network is unreachable)
connect(25, {sa_family=AF_UNSPEC, sa_data="\0\0\0\0\0\0\0\0\0\0\0\0\0\0"}, 16) = 0
connect(25, {sa_family=AF_INET, sin_port=htons(0), sin_addr=inet_addr("54.225.221.128")}, 16) = -1 ENETUNREACH (Network is unreachable)
connect(25, {sa_family=AF_UNSPEC, sa_data="\0\0\0\0\0\0\0\0\0\0\0\0\0\0"}, 16) = 0
connect(25, {sa_family=AF_INET, sin_port=htons(0), sin_addr=inet_addr("54.225.214.74")}, 16) = -1 ENETUNREACH (Network is unreachable)
connect(25, {sa_family=AF_UNSPEC, sa_data="\0\0\0\0\0\0\0\0\0\0\0\0\0\0"}, 16) = 0
connect(25, {sa_family=AF_INET, sin_port=htons(0), sin_addr=inet_addr("23.21.67.46")}, 16) = -1 ENETUNREACH (Network is unreachable)
connect(25, {sa_family=AF_UNSPEC, sa_data="\0\0\0\0\0\0\0\0\0\0\0\0\0\0"}, 16) = 0
connect(25, {sa_family=AF_INET, sin_port=htons(0), sin_addr=inet_addr("184.72.242.47")}, 16) = -1 ENETUNREACH (Network is unreachable)
connect(25, {sa_family=AF_UNSPEC, sa_data="\0\0\0\0\0\0\0\0\0\0\0\0\0\0"}, 16) = 0
connect(25, {sa_family=AF_INET, sin_port=htons(0), sin_addr=inet_addr("54.225.208.13")}, 16) = -1 ENETUNREACH (Network is unreachable)
connect(25, {sa_family=AF_UNSPEC, sa_data="\0\0\0\0\0\0\0\0\0\0\0\0\0\0"}, 16) = 0
connect(25, {sa_family=AF_INET, sin_port=htons(0), sin_addr=inet_addr("107.21.239.197")}, 16) = -1 ENETUNREACH (Network is unreachable)
connect(25, {sa_family=AF_UNSPEC, sa_data="\0\0\0\0\0\0\0\0\0\0\0\0\0\0"}, 16) = 0
connect(25, {sa_family=AF_INET, sin_port=htons(0), sin_addr=inet_addr("184.73.228.50")}, 16) = -1 ENETUNREACH (Network is unreachable)
connect(5, {sa_family=AF_INET6, sin6_port=htons(443), inet_pton(AF_INET6, "::ffff:107.21.237.95", &sin6_addr), sin6_flowinfo=htonl(0), sin6_scope_id=0}, 28) = -1 ENETUNREACH (Network is unreachable)