Netmiko 2.2.2 failes when configuring Huawei devices

[horseinthesky]

I have a bunch of Huawei CE switched and after upgrading Netmiko to 2.2.2 version it can not configure them.

In [1]: from netmiko import ConnectHandler

In [2]: device_params = {
   ...:     'timeout': 10,
   ...:     'ip': 'cloud-test-man1-4s255.yndx.net',
   ...:     'username': 'admin',
   ...:     'password': 'admin',
   ...:     'device_type': 'huawei'
   ...: }

In [3]: ssh = ConnectHandler(**device_params)

In [4]: ssh.enable()
Out[4]: ''

In [5]: ssh.send_config_set(['interface 100ge1/0/14:2', 'port link-type access', 'port default vlan 333', 'commit'])
---------------------------------------------------------------------------
PipeTimeout                               Traceback (most recent call last)
/usr/local/lib/python3.6/dist-packages/paramiko/channel.py in recv(self, nbytes)
    682         try:
--> 683             out = self.in_buffer.read(nbytes, self.timeout)
    684         except PipeTimeout:

/usr/local/lib/python3.6/dist-packages/paramiko/buffered_pipe.py in read(self, nbytes, timeout)
    159                         if timeout <= 0.0:
--> 160                             raise PipeTimeout()
    161 

PipeTimeout: 

During handling of the above exception, another exception occurred:

timeout                                   Traceback (most recent call last)
/usr/local/lib/python3.6/dist-packages/netmiko/base_connection.py in _read_channel_expect(self, pattern, re_flags, max_loops)
    441                     self._lock_netmiko_session()
--> 442                     new_data = self.remote_conn.recv(MAX_BUFFER)
    443                     if len(new_data) == 0:

/usr/local/lib/python3.6/dist-packages/paramiko/channel.py in recv(self, nbytes)
    684         except PipeTimeout:
--> 685             raise socket.timeout()
    686 

timeout: 

During handling of the above exception, another exception occurred:

NetMikoTimeoutException                   Traceback (most recent call last)
<ipython-input-5-7aa36111b725> in <module>()
----> 1 ssh.send_config_set(['interface 100ge1/0/14:2', 'port link-type access', 'port default vlan 333', 'commit'])

/usr/local/lib/python3.6/dist-packages/netmiko/base_connection.py in send_config_set(self, config_commands, exit_config_mode, delay_factor, max_loops, strip_prompt, strip_command, config_mode_command)
   1362         output += self._read_channel_timing(delay_factor=delay_factor, max_loops=max_loops)
   1363         if exit_config_mode:
-> 1364             output += self.exit_config_mode()
   1365         output = self._sanitize_output(output)
   1366         log.debug("{}".format(output))

/usr/local/lib/python3.6/dist-packages/netmiko/huawei/huawei_ssh.py in exit_config_mode(self, exit_config)
     24     def exit_config_mode(self, exit_config='return'):
     25         """Exit configuration mode."""
---> 26         return super(HuaweiSSH, self).exit_config_mode(exit_config=exit_config)
     27 
     28     def check_config_mode(self, check_string=']'):

/usr/local/lib/python3.6/dist-packages/netmiko/cisco_base_connection.py in exit_config_mode(self, exit_config, pattern)
     45         """Exit from configuration mode."""
     46         return super(CiscoBaseConnection, self).exit_config_mode(exit_config=exit_config,
---> 47                                                                  pattern=pattern)
     48 
     49     def serial_login(self, pri_prompt_terminator=r'#\s*$', alt_prompt_terminator=r'>\s*$',

/usr/local/lib/python3.6/dist-packages/netmiko/base_connection.py in exit_config_mode(self, exit_config, pattern)
   1284         if self.check_config_mode():
   1285             self.write_channel(self.normalize_cmd(exit_config))
-> 1286             output = self.read_until_pattern(pattern=pattern)
   1287             if self.check_config_mode():
   1288                 raise ValueError("Failed to exit configuration mode")

/usr/local/lib/python3.6/dist-packages/netmiko/base_connection.py in read_until_pattern(self, *args, **kwargs)
    512     def read_until_pattern(self, *args, **kwargs):
    513         """Read channel until pattern detected. Return ALL data available."""
--> 514         return self._read_channel_expect(*args, **kwargs)
    515 
    516     def read_until_prompt_or_pattern(self, pattern='', re_flags=0):

/usr/local/lib/python3.6/dist-packages/netmiko/base_connection.py in _read_channel_expect(self, pattern, re_flags, max_loops)
    448                     self._write_session_log(new_data)
    449                 except socket.timeout:
--> 450                     raise NetMikoTimeoutException("Timed-out reading channel, data not available.")
    451                 finally:
    452                     self._unlock_netmiko_session()

NetMikoTimeoutException: Timed-out reading channel, data not available.

btw display commands work

In [6]: ssh
Out[6]: <netmiko.huawei.huawei_ssh.HuaweiSSH at 0x7f6d49461a90>

In [7]: ssh.send_command('disp clock')
Out[7]: '2018-08-06 13:51:04+03:00 DST\nMonday\nTime Zone(EEST) : UTC+02:00\nDaylight saving time\t:\n\t Name           :  EEST\n\t Repeat mode    :  repeat\n\t Start year     :  2000\n\t End year       :  2037\n\t Start time     :  last Sun Mar 03:00:00\n\t End time       :  last Sun Oct 04:00:00\n\t Saving time    :  01:00:00'

Version 2.1.1 works perfectly fine:

In [1]: from netmiko import ConnectHandler

In [2]: device_params = {
   ...:     'timeout': 10,
   ...:     'ip': 'cloud-test-man1-4s255.yndx.net',
   ...:     'username': 'admin',
   ...:     'password': 'admin',
   ...:     'device_type': 'huawei'
   ...: }

In [3]: ssh = ConnectHandler(**device_params)

In [4]: ssh.enable()
Out[4]: ''

In [5]: ssh.send_config_set(['interface 100ge1/0/14:2', 'port link-type access', 'port default vlan 333', 'commit'])
Out[5]: 'system-view\nEnter system view, return user view with return command.\n[~cloud-test-man1-4s255]interface 100ge1/0/14:2\n[~cloud-test-man1-4s255-100GE1/0/14:2]port link-type access\n[*cloud-test-man1-4s255-100GE1/0/14:2]port default vlan 333\n[*cloud-test-man1-4s255-100GE1/0/14:2]commit\n[~cloud-test-man1-4s255-100GE1/0/14:2]return\n<cloud-test-man1-4s255>'

[ktbyers]

Can you show me what this set of configuration changes looks like from the CLI (including entering/exiting config mode)?

It looks like it is failing on the exiting of config mode.

[horseinthesky]

It actually does the configuration but raises an NetMikoTimeoutException

interface 100GE1/0/22:1
 description \
 port default vlan 333
 stp edged-port enable
 qos drr 0 to 5
 qos queue 0 drr weight 20
 qos queue 1 drr weight 75
 qos queue 2 drr weight 5
 mac-address notification learning
 jumboframe enable 9712
#

The commands it does are

<cloud-test-man1-4s255>disp history-command all-users 
User   : admin, VTY3, 2A02:6B8:0:401:6016:C2A0:855A:FCA
Time   : 2018-08-07 10:52:03+03:00 DST
Command: return

User   : admin, VTY3, 2A02:6B8:0:401:6016:C2A0:855A:FCA
Time   : 2018-08-07 10:51:58+03:00 DST
Command: commit

User   : admin, VTY3, 2A02:6B8:0:401:6016:C2A0:855A:FCA
Time   : 2018-08-07 10:51:58+03:00 DST
Command: port default vlan 333

User   : admin, VTY3, 2A02:6B8:0:401:6016:C2A0:855A:FCA
Time   : 2018-08-07 10:51:58+03:00 DST
Command: port link-type access

User   : admin, VTY3, 2A02:6B8:0:401:6016:C2A0:855A:FCA
Time   : 2018-08-07 10:51:58+03:00 DST
Command: interface 100ge1/0/22:1

User   : admin, VTY3, 2A02:6B8:0:401:6016:C2A0:855A:FCA
Time   : 2018-08-07 10:51:56+03:00 DST
Command: system-view

[horseinthesky]

I've noticed that Netmiko 2.1.1 does

User   : admin, VTY3, 2A02:6B8:0:401:6016:C2A0:855A:FCA
Time   : 2018-08-07 10:55:12+03:00 DST
Command: screen-length 0 temporary

before system-view and Netmiko 2.2.2 does not.

[ktbyers]

@horseinthesky Yes, I need to see the actual CLI session of what this would look like (not the command history). Particularly I need to see what exiting system-view looks like from a CLI perspective.

Note, there were no direct changes in the Huawei driver between 2.1.1. and 2.2.2 so it is likely an underlying change in the base_connection.py that affected this:

https://github.com/ktbyers/netmiko/commits/develop/netmiko/huawei/huawei_ssh.py

I noticed the HP Comware driver does this for exit_config_mode:

 37     def exit_config_mode(self, exit_config='return', pattern=r'>'):
 38         """Exit config mode."""
 39         return super(HPComwareBase, self).exit_config_mode(exit_config=exit_config,
 40                                                            pattern=pattern)

Whereas Huawei does this:

 24     def exit_config_mode(self, exit_config='return'):
 25         """Exit configuration mode."""
 26         return super(HuaweiSSH, self).exit_config_mode(exit_config=exit_config)

I wonder if we just need to add that pattern=r'> to fix this.

Should be able to tell from the CLI output for when you exit system-view.

[horseinthesky]

Could you tell me how to get this info?

After exception raises a can see that prompt is:

In [2]: ssh.find_prompt()
Out[2]: '<cloud-test-man1-4s255>'

[ktbyers]

@horseinthesky Yes, I am just referring to SSHing into the device and repeating what you are trying to do programmatically, but doing it from the SSH session (and then posting it here).

It will probably be clear from this what is going wrong and what needs fixed in Netmiko.

[horseinthesky]

@ktbyers If you tell me how to do it in more detail I will gladly do it.

[ktbyers]

I am not understanding your question? Just SSH into the device and manually execute the commands you were going to execute via the script?

Then copy and paste the entire SSH session here? Obfuscate anything that is confidential.

[horseinthesky]

Here it is:

netinfra-rw@cloudvessel:~$ ssh cloud-test-man1-4s255.yndx.net

cloud-test-man1-4s255

Info: The max number of VTY users is 8, the number of current VTY users online is 1, and total number of terminal users online is 1.
      The current login time is 2018-08-10 11:52:15+03:00 DST.
<cloud-test-man1-4s255>sy
Enter system view, return user view with return command.
[~cloud-test-man1-4s255]interface 100ge1/0/22:1
[~cloud-test-man1-4s255-100GE1/0/22:1]port link-type access
[*cloud-test-man1-4s255-100GE1/0/22:1]port default vlan 333
[*cloud-test-man1-4s255-100GE1/0/22:1]commit
[~cloud-test-man1-4s255-100GE1/0/22:1]

[ktbyers]

See fix submitted by @duckhawk. It should be in the develop branch of Netmiko now. Let me know if this doesn't fix your issue.

[horseinthesky]

This fixes the issue. Thank you and @duckhawk =)