search

CiscoTestAutomation / pyats

Cisco DevNet pyATS Test Framework Bug Tracker
Apache License 2.0
148 stars 31 forks source link

genie learn isis fails to parse alternate configurations #197

Open roguerouter opened 1 year ago

roguerouter commented 1 year ago

Running into issues with the "genie learn isis" command when configurations deviate from what the parser expects to process.

First, when using ip unnumbered interfaces, the parser looks for the key levels which doesn't exist on an unnumbered interface (see below)

Issue while parsing: <class 'genie.libs.parser.nxos.show_isis.ShowIsisInterface'>

Traceback (most recent call last): File "src/genie/cli/commands/learn.py", line 365, in genie.cli.commands.learn.LearnCommand._retrieve_ops File "/Users/mleuschn/Builds/system_venv/venv/lib/python3.9/site-packages/genie/libs/ops/isis/nxos/isis.py", line 397, in learn self.make() File "/Users/mleuschn/Builds/system_venv/venv/lib/python3.9/site-packages/genie/ops/base/base.py", line 103, in make self.maker.make(*args, **kwargs) File "src/genie/ops/base/maker.py", line 315, in genie.ops.base.maker.Maker.make File "src/genie/ops/base/maker.py", line 427, in genie.ops.base.maker.Maker._call_parser File "src/genie/metaparser/_metaparser.py", line 342, in genie.metaparser._metaparser.MetaParser.parse File "src/genie/metaparser/_metaparser.py", line 322, in genie.metaparser._metaparser.MetaParser.parse File "src/genie/metaparser/util/schemaengine.py", line 419, in genie.metaparser.util.schemaengine.Schema.validate genie.metaparser.util.exceptions.SchemaMissingKeyError: Missing keys: [['instance', '100', 'vrf', 'default', 'interfaces', 'Ethernet1/1', 'levels'], ['instance', '100', 'vrf', 'default', 'interfaces', 'Ethernet1/2', 'levels']]

As a result, the parser fails to collect data and stops processing. I switched levels and topologies in the parser to be Optional() values showIsisInterfaceSchema which "appears" to fix the issue but I'm not sure if this potentially breaks other things within genie.

Second, there appears to be an issue across the board with genie learn isis when the default vrf is not used. If a user deploys a non-default vrf (say vrf blue) across the system and associates all interfaces with that vrf, genie learn still expects to see interfaces, or a system/area id configured in the default vrf. As a result, we see failures because no interfaces are associated with the default vrf, only vrf blue. Without an interface configured in the default vrf, ShowIsisInterface parser fails. If you put an interface in the default vrf, then showisisadjacency fails. While this is expected in a configuration without a default vrf, in a situation where default is not configured, genie should ignore and parse the other configured vrfs.

Issue while parsing: <class 'genie.libs.parser.nxos.show_isis.ShowIsisInterface'>

Traceback (most recent call last): File "src/genie/cli/commands/learn.py", line 365, in genie.cli.commands.learn.LearnCommand._retrieve_ops File "/Users/mleuschn/Builds/system_venv/venv/lib/python3.9/site-packages/genie/libs/ops/isis/nxos/isis.py", line 397, in learn self.make() File "/Users/mleuschn/Builds/system_venv/venv/lib/python3.9/site-packages/genie/ops/base/base.py", line 103, in make self.maker.make(*args, **kwargs) File "src/genie/ops/base/maker.py", line 315, in genie.ops.base.maker.Maker.make File "src/genie/ops/base/maker.py", line 427, in genie.ops.base.maker.Maker._call_parser File "src/genie/metaparser/_metaparser.py", line 342, in genie.metaparser._metaparser.MetaParser.parse File "src/genie/metaparser/_metaparser.py", line 322, in genie.metaparser._metaparser.MetaParser.parse File "src/genie/metaparser/util/schemaengine.py", line 419, in genie.metaparser.util.schemaengine.Schema.validate genie.metaparser.util.exceptions.SchemaMissingKeyError: Missing keys: [['instance', '100', 'vrf', 'default', 'interfaces']]

Issue while parsing: <class 'genie.libs.parser.nxos.show_isis.ShowIsisAdjacency'>

Traceback (most recent call last): File "src/genie/cli/commands/learn.py", line 365, in genie.cli.commands.learn.LearnCommand._retrieve_ops File "/Users/mleuschn/Builds/system_venv/venv/lib/python3.9/site-packages/genie/libs/ops/isis/nxos/isis.py", line 397, in learn self.make() File "/Users/mleuschn/Builds/system_venv/venv/lib/python3.9/site-packages/genie/ops/base/base.py", line 103, in make self.maker.make(*args, **kwargs) File "src/genie/ops/base/maker.py", line 315, in genie.ops.base.maker.Maker.make File "src/genie/ops/base/maker.py", line 427, in genie.ops.base.maker.Maker._call_parser File "src/genie/metaparser/_metaparser.py", line 342, in genie.metaparser._metaparser.MetaParser.parse File "src/genie/metaparser/_metaparser.py", line 322, in genie.metaparser._metaparser.MetaParser.parse File "src/genie/metaparser/util/schemaengine.py", line 419, in genie.metaparser.util.schemaengine.Schema.validate genie.metaparser.util.exceptions.SchemaMissingKeyError: Missing keys: [['instance', '100', 'vrf', 'default', 'interfaces']]

Taarini commented 1 year ago

Hi

Thanks for reporting. Could you please share the device output of the command, the os, platform and the command used. I will debug and let you the know what the issue is about.

roguerouter commented 1 year ago

Hello,

The commands are being run on a Nexus 9kV running 9.3.6 code. However, I also have a customer who is using physical Nexus 9000 (9336-FX) that is experiencing the same issue. I also believe they have seen this issue on some IOS-XE platforms as well (ISR/ASR routers they ran genie learn isis).

The command being run in genie is "genie learn isis --testbed isis_testbed.yaml"

The first error is experienced when we lack an interface in a default vrf configuration.
genie.metaparser.util.exceptions.SchemaMissingKeyError: Missing keys: [['instance', '100', 'vrf', 'default', 'interfaces']]

In the case of my system I only have vrf blue configured on this device. You can see the output of "show isis interface vrf all":

` show isis interface vrf all IS-IS process: 100 VRF: blue loopback5, Interface status: protocol-up/link-up/admin-up IP address: 172.168.0.48, IP subnet: 172.168.0.48/32 IPv6 routing is disabled Level1 No auth type and keychain Auth check set Level2 No auth type and keychain Auth check set Index: 0x0001, Local Circuit ID: 0x01, Circuit Type: L1-2 BFD IPv4 is locally disabled for Interface loopback5 BFD does not support AF IPv4 BFD IPv6 is locally disabled for Interface loopback5 BFD does not support AF IPv6 MTR is disabled Level Metric 1 1 2 1 Topologies enabled: L MT Metric MetricCfg Fwdng IPV4-MT IPV4Cfg IPV6-MT IPV6Cfg 1 0 1 no UP UP yes DN no 2 0 1 no UP UP yes DN no

Ethernet1/1, Interface status: protocol-up/link-up/admin-up IP unnumbered interface (loopback5) IPv6 routing is disabled No auth type and keychain Auth check set Index: 0x0002, Local Circuit ID: 0x01, Circuit Type: L1-2 BFD IPv4 is locally disabled for Interface Ethernet1/1 BFD IPv6 is locally disabled for Interface Ethernet1/1 MTR is disabled Extended Local Circuit ID: 0x1A000000, P2P Circuit ID: 0000.0000.0000.00 Retx interval: 5, Retx throttle interval: 66 ms LSP interval: 33 ms, MTU: 9216 MTU check OFF on P2P interface P2P Adjs: 1, AdjsUp: 1, Priority 64 Hello Interval: 10, Multi: 3, Next IIH: 00:00:05 MT Adjs AdjsUp Metric CSNP Next CSNP Last LSP ID 1 1 1 40 10 00:00:02 ffff.ffff.ffff.ff-ff 2 1 1 40 10 00:00:04 ffff.ffff.ffff.ff-ff Topologies enabled: L MT Metric MetricCfg Fwdng IPV4-MT IPV4Cfg IPV6-MT IPV6Cfg 1 0 40 no UP UP yes DN no 2 0 40 no UP UP yes DN no

Ethernet1/2, Interface status: protocol-up/link-up/admin-up IP unnumbered interface (loopback5) IPv6 routing is disabled No auth type and keychain Auth check set Index: 0x0003, Local Circuit ID: 0x01, Circuit Type: L1-2 BFD IPv4 is locally disabled for Interface Ethernet1/2 BFD IPv6 is locally disabled for Interface Ethernet1/2 MTR is disabled Extended Local Circuit ID: 0x1A000200, P2P Circuit ID: 0000.0000.0000.00 Retx interval: 5, Retx throttle interval: 66 ms LSP interval: 33 ms, MTU: 9216 MTU check OFF on P2P interface P2P Adjs: 1, AdjsUp: 1, Priority 64 Hello Interval: 10, Multi: 3, Next IIH: 00:00:05 MT Adjs AdjsUp Metric CSNP Next CSNP Last LSP ID 1 1 1 40 10 00:00:10 ffff.ffff.ffff.ff-ff 2 1 1 40 10 00:00:08 ffff.ffff.ffff.ff-ff Topologies enabled: L MT Metric MetricCfg Fwdng IPV4-MT IPV4Cfg IPV6-MT IPV6Cfg 1 0 40 no UP UP yes DN no 2 0 40 no UP UP yes DN no

IS-IS process: 100 VRF: default `

Now, if I add an interface to the default vrf as part of IS-IS process, the Missing key for interface goes away, but instead I receive this error:

genie.metaparser.util.exceptions.SchemaMissingKeyError: Missing keys: [['instance', '100', 'vrf', 'default', 'area_address']]

This is due to the lack of net address in the ISIS configuration on the router. As we are not using that vrf we do not want a System ID/NET configured under the isis router process (see router config below) and we need the genie isis parser to be ok with that scenario.

router isis 100 is-type level-1 address-family ipv4 unicast vrf blue net 49.9300.0100.8212.8048.00 address-family ipv4 unicast

If I place a physical interface into the default vrf, add a net in the default vrf and establish adjacency to another IS-IS router, and proceed to run genie learn isis again. The errors related to the default vrf go away but now I am presented with my VRF blue interfaces complaining they are missing the key levels (which is missing because they are IP unnumbered). Note the difference between the Ethernet 1/1 and 1/2 interfaces in vrf blue (ip unnumbered) and the Ethernet 1/3 interface in vrf default (bottom)

genie.metaparser.util.exceptions.SchemaMissingKeyError: Missing keys: [['instance', '100', 'vrf', 'blue', 'interfaces', 'Ethernet1/1', 'levels'], ['instance', '100', 'vrf', 'blue', 'interfaces', 'Ethernet1/2', 'levels']]

`n9kv-s2(config-router)# show isis interface vrf all IS-IS process: 100 VRF: blue loopback5, Interface status: protocol-up/link-up/admin-up IP address: 172.168.0.48, IP subnet: 172.168.0.48/32 IPv6 routing is disabled Level1 No auth type and keychain Auth check set Level2 No auth type and keychain Auth check set Index: 0x0001, Local Circuit ID: 0x01, Circuit Type: L1-2 BFD IPv4 is locally disabled for Interface loopback5 BFD does not support AF IPv4 BFD IPv6 is locally disabled for Interface loopback5 BFD does not support AF IPv6 MTR is disabled Level Metric 1 1 2 1 Topologies enabled: L MT Metric MetricCfg Fwdng IPV4-MT IPV4Cfg IPV6-MT IPV6Cfg 1 0 1 no UP UP yes DN no 2 0 1 no UP UP yes DN no

Ethernet1/1, Interface status: protocol-up/link-up/admin-up IP unnumbered interface (loopback5) IPv6 routing is disabled No auth type and keychain Auth check set Index: 0x0002, Local Circuit ID: 0x01, Circuit Type: L1-2 BFD IPv4 is locally disabled for Interface Ethernet1/1 BFD IPv6 is locally disabled for Interface Ethernet1/1 MTR is disabled Extended Local Circuit ID: 0x1A000000, P2P Circuit ID: 0000.0000.0000.00 Retx interval: 5, Retx throttle interval: 66 ms LSP interval: 33 ms, MTU: 9216 MTU check OFF on P2P interface P2P Adjs: 0, AdjsUp: 0, Priority 64 Hello Interval: 10, Multi: 3, Next IIH: 00:00:08 MT Adjs AdjsUp Metric CSNP Next CSNP Last LSP ID 1 0 0 40 10 0.339208 ffff.ffff.ffff.ff-ff 2 0 0 40 10 00:00:03 ffff.ffff.ffff.ff-ff Topologies enabled: L MT Metric MetricCfg Fwdng IPV4-MT IPV4Cfg IPV6-MT IPV6Cfg 1 0 40 no UP UP yes DN no 2 0 40 no UP UP yes DN no

Ethernet1/2, Interface status: protocol-up/link-up/admin-up IP unnumbered interface (loopback5) IPv6 routing is disabled No auth type and keychain Auth check set Index: 0x0003, Local Circuit ID: 0x01, Circuit Type: L1-2 BFD IPv4 is locally disabled for Interface Ethernet1/2 BFD IPv6 is locally disabled for Interface Ethernet1/2 MTR is disabled Extended Local Circuit ID: 0x1A000200, P2P Circuit ID: 0000.0000.0000.00 Retx interval: 5, Retx throttle interval: 66 ms LSP interval: 33 ms, MTU: 9216 MTU check OFF on P2P interface P2P Adjs: 1, AdjsUp: 1, Priority 64 Hello Interval: 10, Multi: 3, Next IIH: 00:00:08 MT Adjs AdjsUp Metric CSNP Next CSNP Last LSP ID 1 1 1 40 10 00:00:02 ffff.ffff.ffff.ff-ff 2 1 1 40 10 00:00:08 ffff.ffff.ffff.ff-ff Topologies enabled: L MT Metric MetricCfg Fwdng IPV4-MT IPV4Cfg IPV6-MT IPV6Cfg 1 0 40 no UP UP yes DN no 2 0 40 no UP UP yes DN no

IS-IS process: 100 VRF: default Ethernet1/3, Interface status: protocol-up/link-up/admin-up IP address: 10.1.1.1, IP subnet: 10.1.1.0/24 IPv6 routing is disabled Level1 No auth type and keychain Auth check set Level2 No auth type and keychain Auth check set Index: 0x0001, Local Circuit ID: 0x01, Circuit Type: L1 BFD IPv4 is locally disabled for Interface Ethernet1/3 BFD IPv6 is locally disabled for Interface Ethernet1/3 MTR is disabled LSP interval: 33 ms, MTU: 1500 MTU check OFF on LAN interface level-1 MTU check OFF on LAN interface level-2 Level-1 Designated IS: n9kv-s2 Level Metric-0 Metric-2 CSNP Next CSNP Hello Multi Next IIH 1 40 0 10 00:00:06 3 3 0.812277 2 40 0 10 Inactive 10 3 Inactive Level Adjs AdjsUp Pri Circuit ID Since 1 1 1 64 n9kv-s2.01 * 00:07:04 2 0 0 64 0000.0000.0000.01 never Topologies enabled: L MT Metric MetricCfg Fwdng IPV4-MT IPV4Cfg IPV6-MT IPV6Cfg 1 0 40 no UP UP yes DN no 2 0 40 no UP DN no DN no`

The issue with IP unnumbered not limited to a vrf configuration. Even interfaces in the default vrf experience this issue because they are missing the key 'levels' and based on reviewing the parser, would also error on the lack of 'topologies'.

mfierbaugh commented 1 year ago

The parser is also broken for IOS-XR. It seems that the parser also assumes specific configuration and if VRFs, IPv6+v4 topology, SRv6, etc are configured then it fails (albeit a bit differently).
Issue while building the feature

Traceback (most recent call last):
  File "src/genie/cli/commands/learn.py", line 365, in genie.cli.commands.learn.LearnCommand._retrieve_ops
  File "/home/cpoc/.local/lib/python3.8/site-packages/genie/libs/ops/isis/iosxr/isis.py", line 313, in learn
    self.make()
  File "/home/cpoc/.local/lib/python3.8/site-packages/genie/ops/base/base.py", line 103, in make
    self.maker.make(*args, **kwargs)
  File "src/genie/ops/base/maker.py", line 315, in genie.ops.base.maker.Maker.make
  File "src/genie/ops/base/maker.py", line 427, in genie.ops.base.maker.Maker._call_parser
  File "src/genie/metaparser/_metaparser.py", line 308, in genie.metaparser._metaparser.MetaParser.parse
  File "/home/cpoc/.local/lib/python3.8/site-packages/genie/libs/parser/iosxr/show_isis.py", line 757, in cli
    address_family_dict = topology_dict\
UnboundLocalError: local variable 'topology_dict' referenced before assignment

Note that if I manually fix the undefined var or change the topology/configuration (as originally reported in this issue), there are still many parsing issues with isis. This is against a topology with no VRFs, ipv4 and ipv6 configured. 


Traceback (most recent call last):
  File "src/genie/cli/commands/learn.py", line 365, in genie.cli.commands.learn.LearnCommand._retrieve_ops
  File "/Users/mfierbau/Library/Python/3.9/lib/python/site-packages/genie/libs/ops/isis/iosxr/isis.py", line 488, in learn
    self.make()
  File "/Users/mfierbau/Library/Python/3.9/lib/python/site-packages/genie/ops/base/base.py", line 103, in make
    self.maker.make(*args, **kwargs)
  File "src/genie/ops/base/maker.py", line 315, in genie.ops.base.maker.Maker.make
  File "src/genie/ops/base/maker.py", line 427, in genie.ops.base.maker.Maker._call_parser
  File "src/genie/metaparser/_metaparser.py", line 329, in genie.metaparser._metaparser.MetaParser.parse
  File "src/genie/metaparser/_metaparser.py", line 322, in genie.metaparser._metaparser.MetaParser.parse
  File "src/genie/metaparser/util/schemaengine.py", line 318, in genie.metaparser.util.schemaengine.Schema.validate
  File "src/genie/metaparser/util/schemaengine.py", line 288, in genie.metaparser.util.schemaengine.Schema.validate
  File "src/genie/metaparser/util/schemaengine.py", line 390, in genie.metaparser.util.schemaengine.Schema.validate
  File "src/genie/metaparser/util/schemaengine.py", line 363, in genie.metaparser.util.schemaengine.Schema.validate
  File "src/genie/metaparser/util/schemaengine.py", line 318, in genie.metaparser.util.schemaengine.Schema.validate
  File "src/genie/metaparser/util/schemaengine.py", line 288, in genie.metaparser.util.schemaengine.Schema.validate
  File "src/genie/metaparser/util/schemaengine.py", line 390, in genie.metaparser.util.schemaengine.Schema.validate
  File "src/genie/metaparser/util/schemaengine.py", line 363, in genie.metaparser.util.schemaengine.Schema.validate
  File "src/genie/metaparser/util/schemaengine.py", line 390, in genie.metaparser.util.schemaengine.Schema.validate
  File "src/genie/metaparser/util/schemaengine.py", line 363, in genie.metaparser.util.schemaengine.Schema.validate
  File "src/genie/metaparser/util/schemaengine.py", line 390, in genie.metaparser.util.schemaengine.Schema.validate
  File "src/genie/metaparser/util/schemaengine.py", line 363, in genie.metaparser.util.schemaengine.Schema.validate
  File "src/genie/metaparser/util/schemaengine.py", line 430, in genie.metaparser.util.schemaengine.Schema.validate
genie.metaparser.util.exceptions.SchemaUnsupportedKeyError: instance.RTP_LAB.interface.HundredGigE0/0/0/0.topology.ipv6 unicast: Unsupported keys: ['lsp_rexmit_queue_size']