TimeoutError: pexpect timeout error

[narJH27]

Hello,

I am using pyIOSXR via napalm to gather facts from an ASR9K and on most of the 9K's, I get the following error:TimeoutError: pexpect timeout error.

The tracebacks point to pyIOSXR timing out while attempting to retrieve data from the device. I have tweaked the timeout values to as high as 6 minutes (360 sec) and the error persists. Can you throw some light on what else can be done to avoid the pexpect timeouts? Or are you able to provide some kind of a workaround to this issue?

[ktbyers]

@narJH27 Can you post the traceback.

[narJH27]

@ktbyers For instance, when I try to call the get_mac_address_table() method, the traceback is as follows:

In [13]: dev.get_mac_address_table()
---------------------------------------------------------------------------
TimeoutError                              Traceback (most recent call last)
<ipython-input-13-f6edd18a6419> in <module>()
----> 1 dev.get_mac_address_table()

/Users/nseshan/Desktop/napalm-iosxr/napalm_iosxr/iosxr.pyc in get_mac_address_table(self)
   1137         rpc_command = '<Get><Operational><L2VPNForwarding></L2VPNForwarding></Operational></Get>'
   1138 
-> 1139         result_tree = ET.fromstring(self.device.make_rpc_call(rpc_command))
   1140 
   1141         for mac_entry in result_tree.findall('.//L2FIBMACDetailTable/L2FIBMACDetail'):

/Library/Python/2.7/site-packages/pyIOSXR-0.14-py2.7.egg/pyIOSXR/iosxr.pyc in make_rpc_call(self, rpc_command)
    149         Allow a user to query a device directly using XML-requests
    150         """
--> 151         result = __execute_rpc__(self.device, rpc_command, self.timeout)
    152         return ET.tostring(result)
    153 

/Library/Python/2.7/site-packages/pyIOSXR-0.14-py2.7.egg/pyIOSXR/iosxr.pyc in __execute_rpc__(device, rpc_command, timeout)
     31             raise XMLCLIError('The XML document is not well-formed')
     32     except pexpect.TIMEOUT as e:
---> 33         raise TimeoutError("pexpect timeout error")
     34     except pexpect.EOF as e:
     35         raise EOFError("pexpect EOF error")

TimeoutError: pexpect timeout error

In [14]: 

[mirceaulinic]

@narJH27 This issue is pretty common and the traceback is not very helpful. Also, if the pexpect for one reason or another does not catch the </Reply> tag, the timeout value does not really make any difference. However, it is not normal to have that across all your devices, I suspect it's specific to your environment. The following steps will let us help you:

• Please go through this checklist: https://github.com/napalm-automation/napalm#faq and make sure you can access the device
• Provide the IOS-XR version
• Confirm that your config looks alike:

RP/0/RSP0/CPU0:edge03.mrs02#sh run xml
Mon Jul 25 22:45:45.885 UTC
xml agent tty
 iteration off
!

• Confirm that the XML agent works properly. To do so:

i. Enter in XML mode

RP/0/RSP0/CPU0:edge03.mrs02#xml echo format

ii. Paste the following request XML:

<?xml version="1.0" encoding="UTF-8"?><Request MajorVersion="1" MinorVersion="0">
<Get><Operational><L2VPNForwarding></L2VPNForwarding></Operational></Get>
</Request>

[narJH27]

@ktbyers @mirceaulinic

Ok I have the following things to report:

1. The device is accessible via ssh.
2. The version on the box is > 4.3.0
3. xml agent has been enabled and iteration off has been configured

The xml agent on the box, responds to mac/arp/lldp rpc calls within 7-8 mins! This is ridiculously slow compared to the other vendors. Also, while executing these rpc calls over pexpect, it only gets slower. I have increased the timeout on my box to 10 mins and I still see timeout errors whilst accessing the API over pexpect.

[mirceaulinic]

I would say it's time for an upgrade - why not to 6.0.1 or eventually to 6.1 when released (hopefully soon)? :) Moving from 5.3.3 to 6.0.1 already felt like a massive step forward.

[narJH27]

Its on the roadmap (sometime soon), but until then this problem needs to be dealt with.

[mirceaulinic]

OK - I asked you to go through the FAQ checklist - you did't. There is a step that says:

Does your device meet the minimum requrirements?

If you hit the link there, you'll navigate to a page that specifies the minimum IOS-XR version which is 5.1.0.

[mirceaulinic]

@narJH27 You might want to have a look at https://github.com/fooelisa/pyiosxr/pull/22. Can test the code using:

pip install -U git+https://github.com/mirceaulinic/pyiosxr.git@CF-NETMIKO#egg=pyIOSXR

However your main problem is the OS, not the library: 7-8 min reply is unacceptable (don't bother opening a TAC with Cisco - they are not interested in fixing bugs related to the XML agent anymore).

You should either upgrade to >5.1.0 and use this new code (to be released soon), either upgrade to >6.1 and use the new napalm-iosxr-rpc driver when released.

[mirceaulinic]

@narJH27 this should not occur anymore beginning with release 0.20