search

CiscoTestAutomation / genieparser

sub-component of Genie that parse the device output into structured datastructure
Apache License 2.0
248 stars 385 forks source link

IOS-XR Static route parser contains a bug on routes with multiple next hops #738

Open naveci opened 1 year ago

naveci commented 1 year ago

Hi all,

I just found a bug when trying to test for some different situations and it seems to come from the regex interpretation in src/genie/libs/parser/iosxr/show_static_routing.py

In short, it cannot handle routes that have multiple next hop interfaces. Example router CLI output from show static vrf all topology detail

10.252.252.31/32    None                     1.1.1.1             None                             None                [0/0/5/0/1]       No label  
  Path is configured at Mar 15 16:20:37.770
  Path version: 0, Path status: 0x0

10.255.255.224/32   None                     1.3.2.1             None                             None                [0/2048/10/0/1]   No label  
  Path is installed into RIB at Mar 20 15:40:34.674 
  Path version: 1, Path status: 0x21
  Path has best tag: 0

10.255.255.253/32   Bundle-Ether5.507        None                None                             None                [0/4096/1/0/1]    No label  
  Path is configured at Mar 15 16:20:37.770
  Path version: 0, Path status: 0x0
                    Bundle-Ether5.506        None                None                             None                [0/4096/1/0/1]    No label  
  Path is configured at Mar 15 16:20:37.770
  Path version: 0, Path status: 0x0

10.255.255.240/28   Bundle-Ether5.507        None                None                             None                [0/4096/1/0/1]    No label  
  Path is configured at Mar 15 16:20:37.770
  Path version: 0, Path status: 0x0
                    Bundle-Ether5.506        None                None                             None                [0/4096/1/0/1]    No label  
  Path is configured at Mar 15 16:20:37.770
  Path version: 0, Path status: 0x0

The results are as follows (pyats returned JSON dict at the bottom):

Config to replicate:

router static address-family ipv4 unicast 10.252.252.31/32 1.1.1.1 5
router static address-family ipv4 unicast 10.255.255.224/32 1.3.2.1 10 permanent
router static address-family ipv4 unicast 10.255.255.240/28 Bundle-Ether5.506 description Purposefully_misconfgd_route
router static address-family ipv4 unicast 10.255.255.240/28 Bundle-Ether5.507 description Purposefully_misconfgd_route
router static address-family ipv4 unicast 10.255.255.253/32 Bundle-Ether5.506 description Purposefully_misconfgd_route
router static address-family ipv4 unicast 10.255.255.253/32 Bundle-Ether5.507 description Purposefully_misconfgd_route

Version:

You are currently running pyATS version: 23.2
Python: 3.9.6 [64bit]

Checking for outdated packages...

  Package                      Version Latest
  ---------------------------- ------- ------
  genie                        23.2
  genie.libs.clean             23.2
  genie.libs.conf              23.2
  genie.libs.filetransferutils 23.2
  genie.libs.health            23.2
  genie.libs.ops               23.2
  genie.libs.parser            23.2
  genie.libs.robot             23.2
  genie.libs.sdk               23.2
  genie.telemetry              23.2
  genie.trafficgen             23.2
  pyats                        23.2
  pyats.aereport               23.2
  pyats.aetest                 23.2
  pyats.async                  23.2
  pyats.connections            23.2
  pyats.contrib                23.2
  pyats.datastructures         23.2
  pyats.easypy                 23.2
  pyats.kleenex                23.2
  pyats.log                    23.2
  pyats.reporter               23.2
  pyats.results                23.2
  pyats.robot                  23.2
  pyats.tcl                    23.2
  pyats.topology               23.2
  pyats.utils                  23.2
  unicon                       23.2
  unicon.plugins               23.2
  yang.connector               23.2

All your packages are up to date!

Pyats returned dict:

{
    "safi": "unicast",
    "table_id": "0xe0000000",
    "routes": {
        "10.252.252.31/32": {
            "route": "10.252.252.31/32",
            "next_hop": {
                "next_hop_list": {
                    "1": {
                        "index": 1,
                        "next_hop": "1.1.1.1",
                        "metrics": 1,
                        "preference": 5,
                        "local_label": "No label",
                        "active": false,
                        "path_event": "Path is configured at Mar 15 16:20:37.770",
                        "path_version": 0,
                        "path_status": "0x0"
                    }
                }
            }
        },
        "10.255.255.224/32": {
            "route": "10.255.255.224/32",
            "next_hop": {
                "next_hop_list": {
                    "1": {
                        "index": 1,
                        "next_hop": "1.3.2.1",
                        "metrics": 1,
                        "preference": 10,
                        "local_label": "No label",
                        "active": true,
                        "path_event": "Path is installed into RIB at Mar 20 15:40:34.674",
                        "path_version": 1,
                        "path_status": "0x21",
                        "tag": 0
                    }
                }
            }
        },
        "10.255.255.253/32": {
            "route": "10.255.255.253/32",
            "next_hop": {
                "outgoing_interface": {
                    "Bundle-Ether5.507": {
                        "outgoing_interface": "Bundle-Ether5.507",
                        "metrics": 1,
                        "preference": 1,
                        "local_label": "No label",
                        "active": false,
                        "path_event": "Path is configured at Mar 15 16:20:37.770",
                        "path_version": 0,
                        "path_status": "0x0"
                    }
                }
            }
        },
        "B": {
            "route": "B",
            "next_hop": {
                "outgoing_interface": {
                    "Undle-Ether5.506": {
                        "outgoing_interface": "Undle-Ether5.506",
                        "metrics": 1,
                        "preference": 1,
                        "local_label": "No label",
                        "active": false,
                        "path_event": "Path is configured at Mar 15 16:20:37.770",
                        "path_version": 0,
                        "path_status": "0x0"
                    }
                }
            }
        },
        "10.255.255.240/28": {
            "route": "10.255.255.240/28",
            "next_hop": {
                "outgoing_interface": {
                    "Bundle-Ether5.507": {
                        "outgoing_interface": "Bundle-Ether5.507",
                        "metrics": 1,
                        "preference": 1,
                        "local_label": "No label",
                        "active": false,
                        "path_event": "Path is configured at Mar 15 16:20:37.770",
                        "path_version": 0,
                        "path_status": "0x0"
                    }
                }
            }
        },
        "0.0.0.0/0": {
            "route": "0.0.0.0/0",
            "next_hop": {
                "next_hop_list": {
                    "1": {
                        "index": 1,
                        "next_hop": "10.215.132.1",
                        "metrics": 1,
                        "preference": 1,
                        "local_label": "No label",
                        "active": true,
                        "path_event": "Path is installed into RIB at Mar 15 16:23:29.895",
                        "path_version": 1,
                        "path_status": "0x21",
                        "tag": 0
                    }
                }
            }
        }
    }
}
iamsatyanarayan commented 1 year ago

Hi , I am looking at your ticket and reviewing it, Please let me know are you facing the issue still or not ?

iamsatyanarayan commented 1 year ago

Hi , I am looking at your ticket and reviewing it, Please let me know are you facing the issue still or not ?

naveci commented 1 year ago

hi @iamsatyanarayan,

This is still an issue as it seems to be problem in the library with the parsing of the output with regex. The reason is that the CLI outputs multiple exit interfaces under the same route and your regex code doesn't handle the lack of route prefix there.

Thanks

iamsatyanarayan commented 1 year ago

Hi naveci, Could you please share with me the regex. pattern. I will check and let you know.

naveci commented 1 year ago

Hi @iamsatyanarayan,

I was referring to this file: https://github.com/CiscoTestAutomation/genieparser/blob/master/src/genie/libs/parser/iosxr/show_static_routing.py

Somewhere in that regex is the problem that misinterprets the outgoing interfaces if there are multiple configured for the same prefix.

I've updated to Pyats 23.3 in the meanwhile, but that hasn't fixed it.

ghost commented 1 year ago

Hi naveci, I'm finding some discrepancy in the cli and parser output shared above as I'm getting error while trying to execute the parser command. The cli output starts with routes definition but whereas parser output has "safi" and "table_id" which I don't see them in the cli output shared. Could you please share the actual cli output(show static vrf all topology detail) for the issue you are facing. So that we can help you in fixing the issue.

naveci commented 1 year ago

Hi,

Yes, I did trim the output a bit to not share unnecessary output. Here's the output with the headers included.

VRF: default Table Id: 0xe0000000 AFI: IPv4 SAFI: Unicast
  Last path event occured at May  1 22:11:04.487
Prefix/Len          Interface                Nexthop             Object                           Explicit-path       Metrics       Local-Label   
10.248.4.16/32      Bundle-Ether100          None                None                             None                [0/4096/1/0/1]    No label  
  Path is installed into RIB at May  1 22:10:34.079 
  Path version: 1, Path status: 0x21
  Path has best tag: 0

10.248.4.18/32      Bundle-Ether200          None                None                             None                [0/4096/1/0/1]    No label  
  Path is installed into RIB at May  1 22:11:04.487 
  Path version: 1, Path status: 0x21
  Path has best tag: 0

10.252.252.31/32    None                     1.1.1.1             None                             None                [0/0/5/0/1]       No label  
  Path is configured at May  1 22:04:12.804
  Path version: 0, Path status: 0x0

10.255.255.224/32   None                     1.3.2.1             None                             None                [0/2048/10/0/1]   No label  
  Path is installed into RIB at May  1 22:06:22.240 
  Path version: 1, Path status: 0x21
  Path has best tag: 0

10.255.255.253/32   Bundle-Ether5.507        None                None                             None                [0/4096/1/0/1]    No label  
  Path is configured at May  1 22:04:12.804
  Path version: 0, Path status: 0x0
                    Bundle-Ether5.506        None                None                             None                [0/4096/1/0/1]    No label  
  Path is configured at May  1 22:04:12.804
  Path version: 0, Path status: 0x0

10.255.255.240/28   Bundle-Ether5.507        None                None                             None                [0/4096/1/0/1]    No label  
  Path is configured at May  1 22:04:12.804
  Path version: 0, Path status: 0x0
                    Bundle-Ether5.506        None                None                             None                [0/4096/1/0/1]    No label  
  Path is configured at May  1 22:04:12.804
  Path version: 0, Path status: 0x0
ghost commented 1 year ago

Hi naveci, I did some changes in regx. and i am getting expected output now.

p2 = re.compile(r'^(?P([\s]+)?|([a-fA-F\d\/.\:]+)?) ' r'(?P[a-zA-Z][\w\/.-]+) ' r'+(?P[\w\/.\:]+) +(?P[\w]+) ' r'+(?P[\w]+) +(?P[\w\/[]]+)' r'(\s+(?P[\w\s]+?))?' r'(\s+(?P(Path|Last).))?$')

Could you please try with the above pattern and let me know. i hope it will work for you.

iamsatyanarayan commented 1 year ago

Hi naveci, I hope you have seen my previous comment. have you tried that regex. pattern in your script ?

iamsatyanarayan commented 1 year ago

Hi naveci, I hope you have seen my previous comment. have you tried that regex. pattern in your script ?

naveci commented 1 year ago

It already breaks on the first line:

RP/0/RSP0/CPU0:test-pe#
Issue with the parser show static vrf all topology detail

Traceback (most recent call last):
  File "src/genie/cli/commands/parser.py", line 339, in genie.cli.commands.parser.ParserCommand.parse
  File "src/genie/conf/base/device.py", line 531, in genie.conf.base.device.Device.parse
  File "src/genie/conf/base/device.py", line 570, in genie.conf.base.device.Device._get_parser_output
  File "src/genie/conf/base/device.py", line 568, in genie.conf.base.device.Device._get_parser_output
  File "src/genie/metaparser/_metaparser.py", line 308, in genie.metaparser._metaparser.MetaParser.parse
  File "/Users/naveci/coding/pyats/.venv/lib/python3.9/site-packages/genie/libs/parser/iosxr/show_static_routing.py", line 146, in cli
    p2 = re.compile(
  File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.9/lib/python3.9/re.py", line 252, in compile
    return _compile(pattern, flags)
  File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.9/lib/python3.9/re.py", line 304, in _compile
    p = sre_compile.compile(pattern, flags)
  File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.9/lib/python3.9/sre_compile.py", line 764, in compile
    p = sre_parse.parse(p, flags)
  File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.9/lib/python3.9/sre_parse.py", line 948, in parse
    p = _parse_sub(source, state, flags & SRE_FLAG_VERBOSE, 0)
  File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.9/lib/python3.9/sre_parse.py", line 443, in _parse_sub
    itemsappend(_parse(source, state, verbose, nested + 1,
  File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.9/lib/python3.9/sre_parse.py", line 725, in _parse
    raise source.error("unknown extension ?P" + char,
re.error: unknown extension ?P( at position 2
100%|██████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:01<00:00,  1.75s/it]

My code editor colors the top three lines differently from the bottom three. This has to do with the <> following the ?P.

iamsatyanarayan commented 1 year ago

Hi naveci, Issue is fixed and PR merged. The parser will be release in next version.

  • © Githubissues.
  • Githubissues is a development platform for aggregating issues.