search

sanketsudake / ifconfig-parser

Parse ifconfig output collected from local/remote server and retrieve values with goodies
https://pypi.python.org/pypi/ifparser/
BSD 3-Clause "New" or "Revised" License
40 stars 10 forks source link

AttributeError: can't set attribute #2

Closed akr8986 closed 5 years ago

akr8986 commented 7 years ago

I am using python version 3.5

$ python --version Python 3.5.1

I am facing the below error when i am importing the ifparser package i am seeing an attribute error:

In [1]: from ifparser import Ifcfg
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-1-e4742fe46d86> in <module>()
----> 1 from ifparser import Ifcfg

/*****/site-packages/ifparser/__init__.py in <module>()
----> 1 from .ifconfig import Ifcfg, Interface
      2
      3 __title__ = 'ifparser'
      4 __version__ = '0.5.0'
      5 __author__ = 'Sanket Sudake'

/******/site-packages/ifparser/ifconfig.py in <module>()
     51
     52
---> 53 class Ifcfg(object):
     54     scanner = Scanner([
     55         ('process_interface', r"(?P<interface>^[a-zA-Z0-9:-]+)\s+"

/******/site-packages/ifparser/ifconfig.py in Ifcfg()
     67          ".*?mtu (?P<mtu>[0-9]+).*"),
     68         ('process_ignore', r"(Ifconfig|Infiniband|Because)\s.*"),
---> 69         ('process_ignore', r"\s+.*"),
     70     ])
     71

/***********/site-packages/ifparser/re_scan.py in __init__(self, rules, flags)
     99         pattern = Pattern()
    100         pattern.flags = flags
--> 101         pattern.groups = len(rules) + 1
    102         _og = pattern.opengroup
    103         pattern.opengroup = lambda n: _og(n and '%s\x00%s' % (name, n) or n)

AttributeError: can't set attribute
sanketsudake commented 7 years ago

@akr8986 Currently python 3 support is not completely added. I am planning to add it.

akr8986 commented 7 years ago

The issue seems to be with re_scan.py file.. The pattern object doesnt allow to set the value of groups variable..It auto-populates the variable value during execution.. i tried removing the line and executing the script.. it works fine... Not sure if its the right fix though!!

sanketsudake commented 7 years ago

I have tried it earlier. Test didn't pass. I am exploring it. Let me know if all tests pass with your solution.

akr8986 commented 7 years ago

GIT DIFF:

git diff ifparser/re_scan.py
diff --git a/ifparser/re_scan.py b/ifparser/re_scan.py
index 4085365..ce9ba7f 100644
--- a/ifparser/re_scan.py
+++ b/ifparser/re_scan.py
@@ -98,7 +98,6 @@ class Scanner(object):
     def __init__(self, rules, flags=0):
         pattern = Pattern()
         pattern.flags = flags
-        pattern.groups = len(rules) + 1
         _og = pattern.opengroup
         pattern.opengroup = lambda n: _og(n and '%s\x00%s' % (name, n) or n)

UT Result:

Results (1.82s): 2 passed 5 failed

     - /phaedrus/home/abaskaran/Py3rdParty/ifconfig-parser/ifparser/ifconfig.py:136: KeyError: 'vcsbr'
     - /phaedrus/home/abaskaran/Py3rdParty/ifconfig-parser/ifparser/ifconfig.py:136: KeyError: 'p1p1'
     - /phaedrus/home/abaskaran/Py3rdParty/ifconfig-parser/tests/test_ifcfg_parser.py:77: IndexError: list index out of range
     - /phaedrus/home/abaskaran/Py3rdParty/ifconfig-parser/tests/test_ifcfg_parser.py:71: AssertionError: dict_keys(['lo        Link encap:Local Lo[128 chars]  ']) != ['lo', 'docker0', 'eth0']
     - /phaedrus/home/abaskaran/Py3rdParty/ifconfig-parser/tests/test_ifcfg_parser.py:97: AssertionError: dict_keys(['lo', 'eth0', 'wlan0']) != ['lo', 'wlan0', 'eth0']

The last assert seems wrong to me.. since both of them have same elements but only in a different order. am not sure about the other errors!

sanketsudake commented 7 years ago

Thanks for pointing out and debugging. I have updated interfaces property to return sorted list instead of directly returning from dictionary keys list. With it 2 testcases are fixed. I also focusing on rest to include python 3 support. Temporarily I am setting pattern groups len for python 2 only.

The last assert seems wrong to me.. since both of them have same elements but only in a different order. am not sure about the other errors!

akr8986 commented 7 years ago

There are few more things for which support has to be added..

  1. i see some more flags for the interfaces in my machines : 'PROMISC', 'NOARP', 'SIMPLEX',. These flags have to be added in the _flags forzenset.

  2. Support for IPV6:

    inet 10.31.96.29  netmask 255.255.255.0  broadcast 10.31.96.255
inet6 fe80::20c:29ff:fe91:a979  prefixlen 64  scopeid 0x20<link>
ether <.........>  txqueuelen 1000  (Ethernet)

  3. I have a ifconfig ouput where the RX and TX bytes are not parsed from the output.

    RX packets 198237  bytes 283303360 (270.1 MiB)
RX errors 0  dropped 0  overruns 0  frame 0
TX packets 21948  bytes 2979032 (2.8 MiB)
TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

The current regExp doesnt take care of packets and bytes info present in the same line..

  4. A format of ifconfig doesnt seem to be handled:

    
eth0      Link encap:Ethernet  HWaddr <dummy_val>
      inet addr:10.31.96.30  Bcast:10.31.96.255  Mask:255.255.255.0
      inet6 addr: fe80::20c:29ff:fecc:1ede/64 Scope:Link
      UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
      RX packets:330418 errors:0 dropped:0 overruns:0 frame:0
      TX packets:38340 errors:0 dropped:0 overruns:0 carrier:0
      collisions:0 txqueuelen:1000
      RX bytes:464157333 (442.6 MiB)  TX bytes:6906511 (6.5 MiB)

eth1 Link encap:Ethernet HWaddr inet addr:2.0.0.3 Bcast:2.255.255.255 Mask:255.0.0.0 inet6 addr: fe80::20c:29ff:fecc:1ee8/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:28041 errors:0 dropped:0 overruns:0 frame:0 TX packets:37411 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:7572430 (7.2 MiB) TX bytes:8171676 (7.7 MiB)

lo Link encap:Local Loopback inet addr:127.0.0.1 Mask:255.0.0.0 inet6 addr: ::1/128 Scope:Host UP LOOPBACK RUNNING MTU:65536 Metric:1 RX packets:3617825 errors:0 dropped:0 overruns:0 frame:0 TX packets:3617825 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:778195827 (742.1 MiB) TX bytes:778195827 (742.1 MiB)

akr8986 commented 7 years ago

For inet6 i defined a new method:

    def process_ipV6(self, group, groupdict, matched_str):
        if 'inet6 addr:' in matched_str:
            splitStr = matched_str.strip().lower().replace('inet6 addr:',
                                                           'ipv6').split()
            setattr(self._interfaces[self.curr_interface], 'ipv6', splitStr[1])
        else:
            map_dict = {'inet6': 'ipv6', 'prefixlen': 'prefix', 'scopeid': 'scope'}
            kv = iter(matched_str.split())
            for k, v in zip(kv, kv):
                groupdict[map_dict[k]] = v
            self.set_curr_interface_attr(groupdict)

For other two issues i did not see any other way other than creating new methods and new set of RegExp..

After modification my RegExp list fed to Scanner class is below:

    scanner = Scanner([
        ('process_interface', r"(?P<interface>^[a-zA-Z0-9:-]+)\s+"
         "Link encap:(?P<itype>[A-Za-z]+)"
         "((?:Loopback)|(?:HWaddr\s(?P<hwaddr>[0-9A-Fa-f:]+))).*"),
        ('process_any', r"\s+ether\s(?P<hwaddr>[0-9A-Fa-f:]+).*"),
        ('process_ip', r"\s+inet[\s:].*"),
        ('process_ipV6', r"\s+inet6[\s:].*"),
        ('process_mtu', r"\s+(?P<states>[A-Z\s]+\s*)+MTU:(?P<mtu>[0-9]+).*"),
        ('process_any', r"\s+RX bytes:(?P<rxbytes>\d+).*?"),
        ('process_any', r"\s+TX bytes:(?P<txbytes>\d+).*?"),
        ('process_any', r"\s+RX packets:(?P<rxpkts>\d+).*"),
        ('process_any', r"\s+TX packets:(?P<txpkts>\d+).*"),
        ('process_RxTx', r"\s+RX packets\s(?P<rxpkts>\d+)\s+bytes\s(?P<rxbytes>\d+).*"),
        ('process_RxTx', r"\s+TX packets\s(?P<txpkts>\d+)\s+bytes\s(?P<txbytes>\d+).*"),
        ('process_interface2',
         r"(?P<interface>^[a-zA-Z0-9-]+).*?<(?P<states>[A-Z,]+\s*)>"
         ".*?mtu (?P<mtu>[0-9]+).*"),
        ('process_interface3', r"(?P<interface>^[a-zA-Z0-9:-]+)\s+"
         "Link encap:(?P<itype>.*)$"),
        ('process_ignore', r"(Ifconfig|Infiniband|Because)\s.*"),
        ('process_ignore', r"\s+.*"),
    ])

process_RxTx method is simple..


    def process_RxTx(self, group, groupdict, matched_str):
        self.set_curr_interface_attr(groupdict)

and process_interface3 is just a copy of process_interface2.. i had to create a new method since Scanner class was cribbing about having same name (which is our func ptr)..

sanketsudake commented 7 years ago
  1. i see some more flags for the interfaces in my machines : 'PROMISC', 'NOARP', 'SIMPLEX',. These flags have to be added in the _flags forzenset.

Mostly I have added things as I found them, I havn't referred ifconfig(net-tools) source as such. Might have missed these flags. Of course, we can add as we explore.

  1. Support for IPV6:

    inet 10.31.96.29  netmask 255.255.255.0  broadcast 10.31.96.255
inet6 fe80::20c:29ff:fe91:a979  prefixlen 64  scopeid 0x20<link>
ether <.........>  txqueuelen 1000  (Ethernet)

As you have worked out we can add ipv6 support. Though will need to add testcases around this.

  1. I have a ifconfig ouput where the RX and TX bytes are not parsed from the output.

    RX packets 198237  bytes 283303360 (270.1 MiB)
RX errors 0  dropped 0  overruns 0  frame 0
TX packets 21948  bytes 2979032 (2.8 MiB)
TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

The current regExp doesnt take care of packets and bytes info present in the same line..

We can merge following regexes as there is only difference of optional ':' character.

        ('process_any', r"\s+RX packets:(?P<rxpkts>\d+).*"),
        ('process_any', r"\s+TX packets:(?P<txpkts>\d+).*"),
        ('process_RxTx', r"\s+RX packets\s(?P<rxpkts>\d+)\s+bytes\s(?P<rxbytes>\d+).*"),
        ('process_RxTx', r"\s+TX packets\s(?P<txpkts>\d+)\s+bytes\s(?P<txbytes>\d+).*"),

Also you don't need to define separate process_RxTx method as it does same work as process_any, so you can keep process_any as handler for regex.

  1. A format of ifconfig doesn't seem to be handled:

Can you share txt file of your output (may be filtered) ? Seems like process_interface3 is subset of process_interface . Might need few tweaks in process_interface

    ('process_interface3', r"(?P<interface>^[a-zA-Z0-9:-]+)\s+"
     "Link encap:(?P<itype>.*)$"),

Thanks for all great findings, I would really appreciate if you raise pull request for your changes and I can also have deep look at changes.

akr8986 commented 7 years ago

Hi,

process_interface3 is just a copy of process_interface method..

scanner class was throwing an error if i tried to reuse the same method. I dint debug further why..

And the issue (which is solved by process_interface3) can be reproduced with the ifconfig output i have given above..

sanketsudake commented 7 years ago

@akr8986 Can you please create pull request with your changes ? We will take this forward from there.

sanketsudake commented 5 years ago

Related #15 . Closing this for now.