Open dominics opened 8 years ago
GOnetstat isn't really "netstat" per say, but rather a low level library that parses /proc/net/*
files.
I'll look into the impact of net.ipv6.bindv6only
on the proc files and see if there's a way to better handle this setting. But I can't make promises on timeframe, since I'm not sure how easy it will be for me to find this information.
That said, if you or anyone else knows how net.ipv6.bindv6only
is handled behind the scenes in ss
or impacts /proc/net/
, it would be a major help and speed up the fix greatly.
Maybe it will be useful: Try to get info about port 2200
$ netstat -lapn | grep 2200
tcp 0 0 0.0.0.0:2200 0.0.0.0:* LISTEN -
tcp 0 0 10.148.198.222:2200 10.128.104.133:53786 ESTABLISHED -
tcp 0 200 10.148.198.222:2200 10.128.82.234:52288 ESTABLISHED -
tcp6 0 0 :::2200 :::* LISTEN -
Convert decimal 2200 to hex
$ printf "%x\n" 2200
898
Find out information about 0898 in /proc/net/tcp
$ cat /proc/net/tcp | grep 0898
16: 00000000:0898 00000000:0000 0A 00000000:00000000 00:00000000 00000000 0 0 2568782053 1 ffff88345d94e0c0 99 0 0 10 -1
94: DEC6940A:0898 8568800A:D21A 01 00000000:00000000 02:000087B6 00000000 0 0 2255852539 2 ffff8801327640c0 48 3 30 10 -1
242: DEC6940A:0898 EA52800A:CC40 01 000000BC:00000000 01:00000017 00000000 0 0 2476527846 4 ffff88188853c7c0 24 3 17 10 15
And in /proc/net/tcp6
$ cat /proc/net/tcp6 | grep 0898
4: 00000000000000000000000000000000:0898 00000000000000000000000000000000:0000 0A 00000000:00000000 00:00000000 00000000 0 0 2568782055 1 ffff885fd7238080 99 0 0 2 -1
Headers of /proc/net/tcp
$ cat /proc/net/tcp | head -n 1
sl local_address rem_address st tx_queue rx_queue tr tm->when retrnsmt uid timeout inode
Headers of /proc/net/tcp6
$ cat /proc/net/tcp6 | head -n 1
sl local_address remote_address st tx_queue rx_queue tr tm->when retrnsmt uid timeout inode
st value for LISTENING state is 0A
and local_address keep IP:PORT data in hex
so try to find out listening ipv6 ports from /proc/net/tcp6:
$ cat /proc/net/tcp6 | grep " 0A " | awk '{print $2}' | cut -d: -f2 | xargs -I %% printf "%d\n" 0x%%
5672
3306
11211
22
2200
and verify this information with netstat:
$ netstat -tunap6 | grep LISTEN | awk '{print $4}' |rev |cut -d: -f1|rev
(Not all processes could be identified, non-owned process info
will not be shown, you would have to be root to see it all.)
5672
3306
11211
22
2200
similar approach with /proc/net/tcp for ipv4
And with docker ports
# docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
1f142a08c558 mysql:5.6 "docker-entrypoint.sh" 9 hours ago Up 9 hours 0.0.0.0:29999->3306/tcp some-mysql
# sudo netstat -lapn | grep 29999
tcp6 0 0 :::29999 :::* LISTEN 7631/docker-proxy
# cat /proc/net/tcp6 | grep " 0A " | awk '{print $2}' | cut -d: -f2 | xargs -I %% printf "%d\n" 0x%% | grep 29999
29999
ss
does not print out the correct values either, it just says tcp for all, for example:
netstat:
$ netstat -lna|grep 8080
tcp6 0 0 :::8080 :::* LISTEN
ss:
ss -lna |grep 8080
tcp LISTEN 0 128 :::8080 :::*
tcp LISTEN 0 0 :::8080 :::*
Copied from duplicate thread:
Random thought.. maybe it would be simpler to get rid of the whole tcp:
vs tcp6:
thing all together.
We can collapse the tests into two categories: tcp
(default) and udp
, if the user wants to make more specific assertions, they can use the ip:
attribute.
The only downside is if a user wants to test one or the other, it would be an awkward syntax:
port:
tcp:22:
listening: true
ip:
and:
- not:
contain-element: '::'
- contain-element: 0.0.0.0
Thoughts on this approach? I'm just thinking out-loud on this.. not 100% sure it's a good idea yet.
@frezbo Brought up a good point about this complicating the syntax. It seems like a trade off, the syntax would be simpler for just checking if a port is listening, but more complicated if you're trying to test that it's listening only tcp but not tcp6 and vice versa.
Okay, I have an idea:
Things we know: If net.ipv6.bindv6only
is 0 then anything bound to all interfaces will be presented to the OS as listening on the IPv6 address: "::" but it's also listening on the IPv4 address "0.0.0.0".
Processes bound to specific addresses aren't bound to both IPv6 and IPv4 unless specified.
So, my thought is that the simplest way to get the expected behaviour from Goss would be to do something like this in the port code:
....
v6only, _ := sysctl.Get("net.ipv6.bindv6only")
....
net = "tcp6"
for _, entry := range netstat {
if entry.State == "LISTEN" {
port := strconv.FormatInt(entry.Port, 10)
ports[net+":"+port] = append(ports[net+":"+port], entry)
if entry.Ip == "::" && v6only == "0" {
entry.Ip = "0.0.0.0"
ports["tcp:"+port] = append(ports["tcp:"+port], entry)
}
}
}
....
I know this may not look ideal immediately, but considering the alternatives (try to find and integrate another go package or use the above syntax) I think it probably is the MVP of achieving the output we expect from the test. I would need to investigate around the errors thrown by sysctl.Get()
and the value returned on error.
Any thoughts or critique?
Processes bound to specific addresses aren't bound to both IPv6 and IPv4 unless specified.
Are you sure this is true? I haven't tinkered enough with it to know.. but if the bindv6only affects only wildcard IPs, then your solution would be sufficient.
By that statement I mean that if I run say tomcat bound to an interface such as: 192.168.10.10:8080
it wouldn't be (this is an assumption) bound to an IPv6 address unless specified.
I may have to do some testing around this though tbh. Maybe it's possible to bind to a specific IPv4 address but bind to all IPv6 interfaces which would result in "::" but shouldn't output 0.0.0.0
in tcp port binding. as I would expect the process to report that port 8080 is already bound to.
I'll double check it, I'm confused too now...
After added the "spew.Dump(system.Ports())" in the system/port.go file then could get all ports information on the machine.
func NewDefPort(port string, system *System, config util.Config) Port {
p := normalizePort(port)
spew.Dump(system.Ports())
return &DefPort{
port: p,
sysPorts: system.Ports(),
}
}
For example, the output of the ss command:
ss -lna|grep 9090
LISTEN 0 128 :::9090 :::*
the above will print like as the flowing:
....
(string) (len=9) "tcp6:9090": ([]GOnetstat.Process) (len=1 cap=1) {
(GOnetstat.Process) {
User: (string) (len=4) "root",
Name: (string) "",
Pid: (string) "",
Exe: (string) "",
State: (string) (len=6) "LISTEN",
Ip: (string) (len=2) "::",
Port: (int64) 9090,
ForeignIp: (string) (len=2) "::",
ForeignPort: (int64) 0
}
....
So the goss.yaml could as
port:
tcp6:9090:
listening: true
ip:
- '::'
Is it finally implemented ?
I've marked this ticket as "help wanted" a while back. I would be interested in knowing which tools handle this correctly. See my comment above regarding netstat and ss https://github.com/aelsabbahy/goss/issues/149#issuecomment-290194285
When
sysctl net.ipv6.bindv6only
is0
(the default value), ports shown astcp6
innetstat
outputaremay be actually listening on both protocols. If I start with a Goss check like this:And I then create a simple container, exposing a port on all interfaces (ipv4 and ipv6), like so:
Then I'll find that I can correctly use the listening port via ipv4:
However, goss will not validate the port correctly:
What this boils down to is: goss is ensuring netstat's output, not the actual state of the ports.
Isn't
netstat
generally deprecated in favour ofss
, becausess
does a better job of not confusing the user in ipv6 situations like this one? e.g.versus
(ss is more correct; this is a tcp port, not a tcp6 specific one)