Got "Pattern not detected: 'terminal width 511' in output." when try to connect to Cisco 9500 switch using netmiko

[yixiuGit]

I try to connect to the Cisco 9500 switch using netmiko and get the output of "show inventory". I got below error. I ran the same code against Cisco Sandbox without issue.(not 9500 switch but running on IOS_XE) netmiko version is 4.2.0

Traceback (most recent call last):
  File "send_cmd.py", line 86, in <module>
    DEVICECONNECT().send_cmd("show inventory", device)
  File "send_cmd.py", line 41, in send_cmd
    net_connect = ConnectHandler(**device)
  File "/home/cisco/.local/lib/python3.8/site-packages/netmiko/ssh_dispatcher.py", line 388, in ConnectHandler
    return ConnectionClass(*args, **kwargs)
  File "/home/cisco/.local/lib/python3.8/site-packages/netmiko/base_connection.py", line 462, in __init__
    self._open()
  File "/home/cisco/.local/lib/python3.8/site-packages/netmiko/base_connection.py", line 468, in _open
    self._try_session_preparation()
  File "/home/cisco/.local/lib/python3.8/site-packages/netmiko/base_connection.py", line 963, in _try_session_preparation
    self.session_preparation()
  File "/home/cisco/.local/lib/python3.8/site-packages/netmiko/cisco/cisco_ios.py", line 19, in session_preparation
    self.set_terminal_width(command=cmd, pattern=cmd)
  File "/home/cisco/.local/lib/python3.8/site-packages/netmiko/base_connection.py", line 1322, in set_terminal_width
    output = self.read_until_pattern(pattern=pattern)
  File "/home/cisco/.local/lib/python3.8/site-packages/netmiko/base_connection.py", line 721, in read_until_pattern
    raise ReadTimeout(msg)
netmiko.exceptions.ReadTimeout:

Pattern not detected: 'terminal width 511' in output.

Things you might try to fix this:
1. Adjust the regex pattern to better identify the terminating string. Note, in
many situations the pattern is automatically based on the network device's prompt.
2. Increase the read_timeout to a larger value.

You can also look at the Netmiko session_log or debug log for more information.

Here is the code

from netmiko import ConnectHandler, file_transfer
from pprint import pprint

class DEVICECONNECT:

  def send_cmd(self,cmd,device):
    net_connect = ConnectHandler(**device)
    output = net_connect.send_command(cmd,read_timeout=90)
    # pprint(output)
    net_connect.disconnect()
    return output

# 
if __name__ == '__main__':
  device = {
    "device_type": "cisco_xe",
    "ip": "ip add",
    "username": "username",
    "password": "pw",
    "session_log": 'netmiko_session.log',
    "port": '22'
  }
  DEVICECONNECT().send_cmd("show inventory", device)

I changed the read_timeout to 90 seconds, the error appear before 90 seconds expire, so I do not think it is timer issue. Tried to get the debug information in the netmiko_session.log, but it is empty after I run the script. Thank you in advance.

[ktbyers]

What do you see in the session log if you do the following:

from netmiko import ConnectHandler

# I assume you have a real IP address, username, and password here
device = {
    "device_type": "cisco_xe",
    "ip": "ip add",
    "username": "username",
    "password": "pw",
    "session_log": 'netmiko_session.log',
    "port": '22'
}

nc = ConnectHandler(**device)

Also what do you see when you execute terminal width 500 on this device manually (i.e. login using SSH using the same credentials as your program and execute the above command).

[yixiuGit]

If I connect to the device directly with the command, there is no issue at the first time, the session log shows the outcome of the show inventory command. If I run it again, I get the same problem. It does not show anything after I issued "terminal width 500"

The problem happens when I call the send_cmd method from another script. Here is the script.

import os
import json
from send_cmd import DEVICECONNECT
from foldermanage import FOLDERMANAGEMENT
from create_devices import CREATEDEVICES
from read_data import NEWDEVICES

class FATCHECK:
    def __init__(self):
        pass

    def get_env_data(self, device):
        print('Connecting to device')
        #print(device[1])
        #print(type(device[1]))
        if device[1]["os"] == "ACI":
            with open('FAT/' + device[3]["serial_number"]+ "/show_environment.txt", "w") as env_data:
                env_data.write("\nshow hardware\n")
                # cmd_output = DEVICECONNECT().send_cmd("show environment")
                cmd_output = DEVICECONNECT().send_cmd("show hardware", device[0])
                env_data.write(cmd_output)
        else:
            # with open('FAT/' + device[3]["serial_number"] + "/show_environment.json", "w" ) as env_data:
            #     # env_data.write("\nshow environment\n")
            #     # cmd_output = DEVICECONNECT().send_cmd_pyats("show environment")
            #     cmd_output = DEVICECONNECT().send_cmd_pyats("show inventory", device[0])
            #     # env_data.write(json.dumps(cmd_output))
            #     json.dump(cmd_output, env_data, indent=2)

            with open('FAT/' + device[3]["serial_number"] + "/show_environment.txt", "w" ) as env_data:
                env_data.write("\nshow environment\n")
                # cmd_output = DEVICECONNECT().send_cmd("show environment")
                print(device[0])
                cmd_output = DEVICECONNECT().send_cmd("show version", device[0])
                env_data.write(cmd_output)

if __name__ == '__main__':

    excel_output = NEWDEVICES().read_from_excel()
    print(excel_output)
    print('*****************************')
    json_output = NEWDEVICES().read_from_json(excel_output)
    devices = CREATEDEVICES().create_devices(json_output)
    print(devices)
    print('#########################')
    # devices = [{'device_type': 'cisco_nxos', 'ip': 'sbx-nxos-mgmt.cisco.com', 'username': 'admin', 'password': 'Admin_1234!', 'port': '22'}, {'device_type': 'cisco_xe', 'ip': 'sandbox-iosxe-recomm-1.cisco.com', 'username': 'developer', 'password': 'lastorangerestoreball8876', 'port': '22'}]
    for device in devices:
        FOLDERMANAGEMENT().create_dir(device[3]["serial_number"])
        FATCHECK().get_env_data(device)

Here is the outcome when I the run the script, I replace the credential

python3 fat_data_collect.py
[{'mgmt_ip': 'ip', 'username': 'username', 'password': 'pw', 'serial_number': 'FDO264406U0'}]
*****************************
[[{'device_type': 'cisco_xe', 'ip': 'ip', 'username': 'username', 'password': 'pw', 'session_log': 'netmiko_session.log', 'port': '22'}, {'os': 'IOS_XE'}, {'file_system': 'flash:'}, {'serial_number': 'FDO264406U0'}]]
#########################
Connecting to device

#This is the device object will be used to connect
{'device_type': 'cisco_xe', 'ip': 'ip', 'username': 'username', 'password': 'pw', 'session_log': 'netmiko_session.log', 'port': '22'}
Traceback (most recent call last):
  File "fat_data_collect.py", line 57, in <module>
    FATCHECK().get_env_data(device)
  File "fat_data_collect.py", line 38, in get_env_data
    cmd_output = DEVICECONNECT().send_cmd("show version", device[0])
  File "/home/cisco/send_cmd.py", line 41, in send_cmd
    net_connect = ConnectHandler(**device)
  File "/home/cisco/.local/lib/python3.8/site-packages/netmiko/ssh_dispatcher.py", line 388, in ConnectHandler
    return ConnectionClass(*args, **kwargs)
  File "/home/cisco/.local/lib/python3.8/site-packages/netmiko/base_connection.py", line 462, in __init__
    self._open()
  File "/home/cisco/.local/lib/python3.8/site-packages/netmiko/base_connection.py", line 468, in _open
    self._try_session_preparation()
  File "/home/cisco/.local/lib/python3.8/site-packages/netmiko/base_connection.py", line 963, in _try_session_preparation
    self.session_preparation()
  File "/home/cisco/.local/lib/python3.8/site-packages/netmiko/cisco/cisco_ios.py", line 19, in session_preparation
    self.set_terminal_width(command=cmd, pattern=cmd)
  File "/home/cisco/.local/lib/python3.8/site-packages/netmiko/base_connection.py", line 1322, in set_terminal_width
    output = self.read_until_pattern(pattern=pattern)
  File "/home/cisco/.local/lib/python3.8/site-packages/netmiko/base_connection.py", line 721, in read_until_pattern
    raise ReadTimeout(msg)
netmiko.exceptions.ReadTimeout:

Pattern not detected: 'terminal width 511' in output.

Things you might try to fix this:
1. Adjust the regex pattern to better identify the terminating string. Note, in
many situations the pattern is automatically based on the network device's prompt.
2. Increase the read_timeout to a larger value.

You can also look at the Netmiko session_log or debug log for more information.

There is nothing in the session log when I run the script. I tried to run the script several times, for some reason, it worked randomly, I cannot find the pattern.

[yixiuGit]

I found out if the switch is in idle, the code does not work.

If I ssh to the switch and make it in active, it works

But it only works if I only send 1 command. When I try to send the same command twice(one to get raw output, one to parse with pyats), I got the same problem. It looks like after the first command, the switch moves back to idel mode.

[ktbyers]

It looks like you are connecting to the console port. How are you connecting to the console port (terminal server or some other way)?

[yixiuGit]

Yes, after check with the engineers who setup the switch, the switch is connected via console port using an application called opengear. In order to automate, I need to configure ssh via the console port first. Sorry for the confuse. I think the case can be close. Could you let me know if netmiko supports connection via console port, if not I will try pyserial that someone else advised. Thank you

[ktbyers]

Potenitally, use the terminal_server or generic device_type (they are the same thing) and then switch to cisco_xe device_type using redispatch.

The terminal server breaks normal SSH behavior, however i.e. introduces the need to send additional , potentially re-login, etc.

There is also a serial way to connect in Netmiko, but that presupposes your laptop/computer is directly connected to the console of the device.