search

ftao / python-ifcfg

Python cross-platform network interface discovery (ifconfig/ipconfig/ip)
BSD 3-Clause "New" or "Revised" License
55 stars 37 forks source link

Tune "flags" regex for ip and ifconfig output + add test case #43

Open claireyywang opened 4 years ago

claireyywang commented 4 years ago

Currently the flags are only parsed in ifconfig command pattern. It would be nice to have that in ip too since ifconfig is deprecated. Thanks!

benjaoming commented 4 years ago

Hi @claireyywang - thanks for filing the issue! A PR is most welcome.

I think there's already test data for this purpose. The behavior can be implemented and already existing unit tests can be expanded to cover flags.

shr-project commented 4 years ago

And even with ifconfig command it doesn't work with some versions of net-tools

e.g. 

ifconfig --version
net-tools 1.60
ifconfig 1.42 (2001-04-13)

with this format:

root@qemux86-64:~# ifconfig
eth0      Link encap:Ethernet  HWaddr 52:54:00:12:34:02  
          inet addr:192.168.7.2  Bcast:192.168.7.255  Mask:255.255.255.0
          inet6 addr: fe80::5054:ff:fe12:3402/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:6 errors:0 dropped:0 overruns:0 frame:0
          TX packets:106 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:496 (496.0 B)  TX bytes:42832 (41.8 KiB)

doesn't parse the flags, while net-tools 1.60+git20180626.aebd88e-1ubuntu1 in ubuntu works with this format:

# ifconfig eth0
eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 172.17.0.2  netmask 255.255.0.0  broadcast 172.17.255.255
        ether 02:42:ac:11:00:02  txqueuelen 0  (Ethernet)
        RX packets 39299  bytes 89614740 (89.6 MB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 33767  bytes 3807354 (3.8 MB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
ifconfig --version
net-tools 2.10-alpha

works

The format was probably changed in: https://sourceforge.net/p/net-tools/code/ci/eb04ef31571f6c707eacaba6846feeebfab518e6 which unfortunately doesn't mention anything about format change in commit message and considering state of net-tools package in various distributions it's hard to tell which distributions have this included, e.g. debian probably included it since:

net-tools (1.60+git20150829.73cef8a-1) unstable; urgency=medium

  * After 14 years without an upstream release, I am producing a new package
    based on today's upstream repository.
    Closes: #391495, #486448, #323261, #260587, #545328, #511395.
  * Remove many patches now merged upstream, delete unmaintainable and
    undocumented local changes, and update the rest.

 -- Martín Ferrari <tincho@debian.org>  Mon, 07 Sep 2015 01:54:07 +0000
benjaoming commented 4 years ago

@shr-project the first example (that's the broken one, right) has a test case written:

https://github.com/ftao/python-ifcfg/blob/d936468c3cb3f1dad58ca2a25d55175020d43f3b/tests/ifconfig_out.py#L4-L23

...but upon further investigation it seems like the test case doesn't check if the flags were found:

https://github.com/ftao/python-ifcfg/blob/d936468c3cb3f1dad58ca2a25d55175020d43f3b/tests/ifconfig_tests.py#L43-L52

shr-project commented 4 years ago

Sorry about the spam from meta-ros commits (it's useful to mention related issue ticket in commit message and in many cases I didn't want to trigger notification from the other side, but github is just too clever).

Anyway, I've also sent patch to OpenEmbedded (where I've seen the issue first) to update the ifconfig format to match what debian/ubuntu currently use: https://lists.openembedded.org/g/openembedded-core/message/139567 now python-ifcfg works fine for me and ros is happily finding those flags.

If you can extend the regex to cover older ifconfig (which will probably work on busybox's version of ifconfig as well) then it would be still nice for other people I guess (especially as you already have a test data for it).

Thanks

benjaoming commented 3 years ago

In #61, @dre1902 has implemented a very simple approach to parsing and storing flags from ip. It's not platform transparent, but does it work well enough for you cases?

shr-project commented 3 years ago

@benjaoming Thanks for update, I've reverted the change in net-tools to reproduce this issue again and backported the change from https://github.com/ftao/python-ifcfg/pull/61 to python-ifcfg-0.21. Unfortunately it still doesn't parse correctly:

root@qemux86-64:~# grep -B 3 flags= /usr/lib/python3.8/site-packages/ifcfg/parser.py
    @classmethod
    def get_patterns(cls):
        return [
            r'(?P<device>^[-a-zA-Z0-9:\.]+): flags=(?P<flags>.*) mtu (?P<mtu>\d+)',
root@qemux86-64:~# ifconfig
eth0      Link encap:Ethernet  HWaddr 52:54:00:12:34:02  
          inet addr:192.168.7.2  Bcast:192.168.7.255  Mask:255.255.255.0
          inet6 addr: fe80::5054:ff:fe12:3402/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:6 errors:0 dropped:0 overruns:0 frame:0
          TX packets:49 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:496 (496.0 B)  TX bytes:15914 (15.5 KiB)

lo        Link encap:Local Loopback  
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:65536  Metric:1
          RX packets:340 errors:0 dropped:0 overruns:0 frame:0
          TX packets:340 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:22124 (21.6 KiB)  TX bytes:22124 (21.6 KiB
root@qemux86-64:~# python3 -m ifcfg.cli | python3 -m json.tool
{
    "eth0": {
        "inet": "192.168.7.2",
        "inet4": [
            "192.168.7.2"
        ],
        "ether": "52:54:00:12:34:02",
        "inet6": [
            "add",
            "fe80::5054:ff:fe12:3402"
        ],
        "netmask": "255.255.255.0",
        "device": "eth0",
        "broadcast": "192.168.7.255",
        "mtu": "1500",
        "rxbytes": "496",
        "txbytes": "18864"
    },
    "lo": {
        "inet": "127.0.0.1",
        "inet4": [
            "127.0.0.1"
        ],
        "ether": null,
        "inet6": [
            "add",
            "::1"
        ],
        "netmask": "255.0.0.0",
        "device": "lo",
        "mtu": "65536",
        "rxbytes": "25540",
        "txbytes": "25540"
    },
    "sit0": {
        "inet": null,
        "inet4": [],
        "ether": "00-00-00-00-30-30-30-00-00-00-00-00-00-00-00-00",
        "inet6": [],
        "netmask": null,
        "device": "sit0",
        "mtu": "1480",
        "rxbytes": "0",
        "txbytes": "0"
    }
}

the old format from net-tools's ifconfig as well as busybox's (as in v1.33.0) ifconfig doesn't have 'flags=' at all, the flags are listed only on this line:

UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1

For my case (https://github.com/ros/meta-ros) I've just backported the format change to net-tools for python-ifcfg to parse the flags correctly, I don't know how many distributions still ship older net-tools especially now with new upstream release 2.10 (from 2021-01-07) which includes the format change from 2003 https://sourceforge.net/p/net-tools/code/ci/eb04ef31571f6c707eacaba6846feeebfab518e6

benjaoming commented 3 years ago

@shr-project thanks for testing. IIRC (don't have time to verify it ATM) then ifconfig is used if it exists. I guess on second thought that since the new behavior only works for systems that only have the ip command.