search

napalm-automation / napalm-ios

Apache License 2.0
31 stars 40 forks source link

Multiple Getters Failing on IOS-XE (ASR1001) #74

Closed tyler-8 closed 7 years ago

tyler-8 commented 7 years ago

Description of Issue/Question

Did you follow the steps from https://github.com/napalm-automation/napalm#faq

Setup

napalm-ios version

(Paste verbatim output from pip freeze | grep napalm-ios between quotes below)

napalm-ios==0.4.0
napalm-iosxr==0.4.0

IOS version

(Paste verbatim output from show version between quotes below)

Cisco IOS XE Software, Version 03.16.01a.S - Extended Support Release
Cisco IOS Software, ASR1000 Software (X86_64_LINUX_IOSD-UNIVERSALK9-M), Version 15.5(3)S1a, 
ROM: IOS-XE ROMMON
iosxe-rtr uptime is 5 weeks, 6 days, 17 hours, 17 minutes
Uptime for this control processor is 5 weeks, 6 days, 17 hours, 18 minutes
System returned to ROM by reload at 04:39:13 UTC Wed Oct 19 2016
System image file is "bootflash:asr1001x-universalk9.03.16.01a.S.155-3.S1a-ext.SPA.bi"
Last reload reason: Reload Command
cisco ASR1001-X (1NG) processor (revision 1NG) with 3729813K/6147K bytes of memory.
Configuration register is 0x2102

Steps to Reproduce the Issue

device.get_bgp_neighbors() device.get_facts() device.get_arp_table() device.get_interfaces() device.get_interfaces_ip()

Error Traceback

(Paste the complete traceback of the exception between quotes below)

>>> device.get_bgp_neighbors()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File ".../lib/python2.7/site-packages/napalm_ios/ios.py", line 917, in get_bgp_neighbors
    out_queue, up_time, state_prefix = fields
ValueError: need more than 1 value to unpack

>>> device.get_facts()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File ".../lib/python2.7/site-packages/napalm_ios/ios.py", line 583, in get_facts
    interface = line.split()[0]
IndexError: list index out of range

>>> device.get_arp_table()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File ".../lib/python2.7/site-packages/napalm_ios/ios.py", line 1177, in get_arp_table
    raise ValueError("Unexpected output from: {}".format(line.split()))
ValueError: Unexpected output from: [u'Time', u'source', u'is', u'NTP,', u'*22:05:56.858', u'UTC', u'Tue', u'Nov', u'29', u'2016']

>>> device.get_interfaces()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File ".../lib/python2.7/site-packages/napalm_ios/ios.py", line 663, in get_interfaces
    raise ValueError(u"Unexpected Response from the device")
ValueError: Unexpected Response from the device

>>> device.get_interfaces_ip()                     
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File ".../lib/python2.7/site-packages/napalm_ios/ios.py", line 746, in get_interfaces_ip
    raise ValueError("Unexpected response from the router")
ValueError: Unexpected response from the router

The BGP neighbors error is most likely due to the 'Neighbors' column in IOS not being wide enough to accomadate an IP address with four full octets (*192.168.123.123). For the get facts I'm not sure what's causing the error.

BGP neighbors example:

*192.168.123.97 4        12345   20902   20895    13061    0    0 3d09h           1
*192.168.123.98 4        12345   967     971    13061    0    0 03:45:19        2
*192.168.123.99 4        12345   18277   18271    13061    0    0 2d23h           2
*192.168.123.100
                4        12345    8783    8788    13061    0    0 1d10h           1
*192.168.123.101
                4        12345   16235   16202    13061    0    0 2d15h           2
*192.168.123.102
                4        12345   21769   21796    13061    0    0 3d12h           2
*192.168.123.103
                4        12345    2284    2293    13061    0    0 08:53:20        2
dbarrosop commented 7 years ago

Are you shitting me? Is Cisco that st%&$d? They don't know that an IP represented as a string can have a max_length of 15 counting the dots? Interesting a networking vendor doesn't know about that.

mirceaulinic commented 7 years ago 
*192.168.123.100
                4        12345    8783    8788    13061    0    0 1d10h           1

Whut. the. f!? That's beyond my imagination!

itdependsnetworks commented 7 years ago

One way to "normalize" the data:

out = '''
*192.168.123.97 4        12345   20902   20895    13061    0    0 3d09h           1
*192.168.123.98 4        12345   967     971    13061    0    0 03:45:19        2
*192.168.123.99 4        12345   18277   18271    13061    0    0 2d23h           2
*192.168.123.100
                4        12345    8783    8788    13061    0    0 1d10h           1
*192.168.123.101
                4        12345   16235   16202    13061    0    0 2d15h           2
*192.168.123.102
                4        12345   21769   21796    13061    0    0 3d12h           2
*192.168.123.103
                4        12345    2284    2293    13061    0    0 08:53:20        2
'''
output = out.split('\n')

count = 1
regexp = re.compile(r'^(?:\*|\s+)(?:\d{3}\.){3}\d{3}')
tempoutput = []

for line in output:
    if len(line.split()) == 9:
        pass
    elif regexp.search(line) is not None and len(output[count].split()) == 9:
        tempoutput.append(line.strip() + output[count])
    else:
        tempoutput.append(line)
    count = count + 1

output = tempoutput
itdependsnetworks commented 7 years ago

Added this in slack, but thought I would comment here, should probably have: peer_id = peer_id.replace('*','') as well.

ktbyers commented 7 years ago

Can we just re.sub based on IP address pattern with a trailing newline and then eliminate the newline?

@itdependsnetworks Sounds good on your peer_id replacement as well?

tyler-8 commented 7 years ago

@ktbyers That would actually work quite well if done before the linesplit. Replace the trailing newline with a space and good to go. Genius!

I still need to figure out why the other getters are failing.

tyler-8 commented 7 years ago

get_facts() fails due to show ip interface brief on IOS-XE also returns load and time source stats. Fix is show ip interface brief | begin ^Interface

Here's the output:

#show ip interface brief
Load for five secs: 1%/0%; one minute: 1%; five minutes: 1%
Time source is NTP, *09:59:40.324 UTC Wed Nov 30 2016

Interface              IP-Address      OK? Method Status                Protocol
itdependsnetworks commented 7 years ago

@ktbyers

I'm not sure how that will work, in my mind it will still be two different elements in the same list, though I am sure I am missing something. If it works, then all the better. Was just offering up at potential solution being that it only took me 5-10 min, but again, whatever works :)

tyler-8 commented 7 years ago

@itdependsnetworks: What @ktbyers is proposing is eliminating the newlines following the IP before the .split("\n") function is performed. That should make the linesplit piece execute without issue.

ktbyers commented 7 years ago

@itdependsnetworks It's all good. I will try to do a few tests.

itdependsnetworks commented 7 years ago

@tyler-8 @ktbyers brilliant :)

output = re.sub(r'((?:\*|\s+)(?:\d{3}\.){3}\d{3}\s*)(?:\n|\r)', r'\1', output)

tyler-8 commented 7 years ago

get_arp_table() has the same issue as get_facts(). show ip arp output gives load and NTP stats.

Raw output:

#sh ip arp
Load for five secs: 1%/0%; one minute: 1%; five minutes: 0%
Time source is NTP, *11:57:42.689 UTC Wed Nov 30 2016

Protocol  Address          Age (min)  Hardware Addr   Type   Interface
Internet  207.11.25.233          12   0010.dbff.1001  ARPA   GigabitEthernet0/0/0

Here's the fix: sh ip arp | begin ^Protocol

ktbyers commented 7 years ago

@tyler-8 Okay, I will try to fix these in the same PR request.

tyler-8 commented 7 years ago

@ktbyers - Okay confirmed for sure. These functions are breaking due to the same lines in the output.

device.get_facts() device.get_arp_table() device.get_interfaces() device.get_interfaces_ip()

Adding the output modifiers to | begin ^Protocol or | begin ^Interface will fix it for all IOS devices.

tyler-8 commented 7 years ago

I just saw that the get_arp_table() is already using an output modifier. command = 'show arp | exclude Incomplete'

So for that command it'll have to be something like sh ip arp | exclude Incomplete|Load for|Time source but that still leaves a blank line at the beginning of the output. I'm not sure if there's a way to filter out blank lines in the CLI so that may be for NAPALM to do.

router#sh ip arp | exclude Incomplete|Load for|Time source  

Protocol  Address          Age (min)  Hardware Addr   Type   Interface
tyler-8 commented 7 years ago

Thanks to @ogenstad

Try typing terminal no exec prompt timestamp, before the command. You probably have something in your config to set that.

Which turned out to be the case for all the getters with issues except device.get_bgp_neighbors() - which still breaks due to IOS wrapping the neighbor lines.

Performing >>> device.cli(["terminal no exec prompt timestamp"]) prior to executing the other getters solves the problems.

ktbyers commented 7 years ago

@tyler-8 Yes, I figured it was probably a CLI setting for the ones excluding BGP.

It is probably worth fixing, however, so NAPALM just filters those lines out.

tyler-8 commented 7 years ago

Found some additional silliness, though netmiko may already be handling this by doing terminal length 0.

When encountering -- More --, IOS will insert additional header rows, which can split up the peer-id from its BGP data.

*192.168.123.100
                4        12345    1941    1943    13847    0    0 07:32:54        1
*192.168.123.101
Neighbor        V           AS MsgRcvd MsgSent   TblVer  InQ OutQ Up/Down  State/PfxRcd
                4        12345    3768    3769    13847    0    0 14:39:49        2
*192.168.123.102
                4        12345    1098    1101    13847    0    0 04:16:13        1
tyler-8 commented 7 years ago

Here's my regex stab at removing the line breaks/feeds after the peer-id:

line_merge = re.compile(r"((?:\d{1,3}\.){3}\d{1,3}\s*)(?:\n|\r)(.+)") output = line_merge.sub(r"\1\2", bgp_output.strip())

ktbyers commented 7 years ago

Will try to fix next week: 12/5 - 12/9

dbarrosop commented 7 years ago

Would be nice to migrate to the new testing framework and include this test cases before fixing. Will double check with @ogenstad as I think he was working on it and help him out to have it ready ASAP.

ktbyers commented 7 years ago

I submitted a PR on this including tests here:

https://github.com/napalm-automation/napalm-ios/pull/78

ktbyers commented 7 years ago

@tyler-8 Can you test this fix works with your BGP line wrapping issue.

ktbyers commented 7 years ago

I also put in a fix that should filter these lines:

Load for five secs: 1%/0%; one minute: 1%; five minutes: 1%
Time source is NTP, *09:59:40.324 UTC Wed Nov 30 2016