Linux-driver terminal width is not being set by Netmiko

[satnau]

Hi,

Using Netmiko version 3.0.0 I am trying to connect to a Linux box & then sending a grep command (grep -w {ip}/32 *.set | grep book | grep -v 1b | grep -v 2b) , however, it fails with the below error.

If I remove the pipes ( | ) from the command it works just fine.

Error is pipe related-

raise PipeTimeout() paramiko.buffered_pipe.PipeTimeout

I shouldn't say its a netmiko error rather some piece of code in the /paramiko/channel.py is complaining. However, if I downgrade the Netmiko to version 2.4.2, this error is never thrown & code finishes without any issues. Paramiko version that I am using is 2.7.1 .

Code:

from netmiko import ConnectHandler
secmt = {'device_type': 'linux','ip': 'secmt03','username': username,'password': '***********' }
secmt_connect = ConnectHandler(**secmt)
grep_op = secmt_connect.send_command_expect(
    f'grep -w {ip}/32 *.set | grep book | grep -v 1b | grep -v 2b', expect_string = 'secmt junos-firewall'
) 
print(grep_op)

Error:

Traceback (most recent call last): File "/home/satnau/.local/lib/python3.6/site-packages/paramiko/channel.py", line 699, in recv out = self.in_buffer.read(nbytes, self.timeout) File "/home/satnau/.local/lib/python3.6/site-packages/paramiko/buffered_pipe.py", line 164, in read raise PipeTimeout() paramiko.buffered_pipe.PipeTimeout During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/satnau/.local/lib/python3.6/site-packages/netmiko/base_connection.py", line 541, in _read_channel_expect new_data = self.remote_conn.recv(MAX_BUFFER) File "/home/satnau/.local/lib/python3.6/site-packages/paramiko/channel.py", line 701, in recv raise socket.timeout() socket.timeout During handling of the above exception, another exception occurred: Traceback (most recent call last): File "DNS_PING_FIREWALL_TESTING_V2.0.py", line 243, in junos_fw_info = secmt_connect.send_command_expect(f'grep -w {ip}/32 .set | grep book | grep -v 1b | grep -v 2b', expect_string = 'secmt junos-firewall') File "/home/satnau/.local/lib/python3.6/site-packages/netmiko/base_connection.py", line 1447, in send_command_expect return self.send_command(args, *kwargs) File "/home/satnau/.local/lib/python3.6/site-packages/netmiko/base_connection.py", line 1366, in send_command new_data = self.read_until_pattern(pattern=re.escape(cmd)) File "/home/satnau/.local/lib/python3.6/site-packages/netmiko/base_connection.py", line 618, in read_until_pattern return self._read_channel_expect(args, **kwargs) File "/home/satnau/.local/lib/python3.6/site-packages/netmiko/base_connection.py", line 552, in _read_channel_expect "Timed-out reading channel, data not available." netmiko.ssh_exception.NetmikoTimeoutException: Timed-out reading channel, data not available.

Thanks, Satish

[ktbyers]

Is this just a Linux box you are connecting to?

[ktbyers]

This PR will probably fix your issue if you want to test it:

net_connect.send_command(cmd, cmd_verify=False)

https://github.com/ktbyers/netmiko/pull/1566

I suspect the issue is that your very long line is causing the line to exceed the terminal width which is then breaking the command echo verification.

[satnau]

Is this just a Linux box you are connecting to?

Yes Kirk its just a linux box.

[satnau]

This PR will probably fix your issue if you want to test it:

net_connect.send_command(cmd, cmd_verify=False)

1566

I suspect the issue is that your very long line is causing the line to exceed the terminal width which is then breaking the command echo verification.

On using 'cmd_verify = False', I am getting this error - Traceback (most recent call last): File "github_netwmiko_issue.py", line 4, in grep_op = secmt_connect.send_command_expect(f'grep -w x.x.x.x/32 .set | grep book', expect_string = 'secmt junos-firewall', cmd_verify=False) File "/home/satnaidu/.local/lib/python3.6/site-packages/netmiko/base_connection.py", line 1447, in send_command_expect return self.send_command(args, **kwargs) TypeError: send_command() got an unexpected keyword argument 'cmd_verify'

This was the command that I had passed - secmt_connect.send_command_expect(f'grep -w {ip}/32 *.set | grep book | grep -v 1b | grep -v 2b', expect_string = 'secmt junos-firewall',cmd_verify=False)

[ktbyers]

Did you git clone the branch that I referenced in #1566? You will need that branch for cmd_verify to work.

[satnau]

Did you git clone the branch that I referenced in #1566? You will need that branch for cmd_verify to work.

I will do this & get back to you Kirk

[satnau]

@ktbyers -pardon my ignorance; I am new to Github. So, this is what I did & the code still fails with the same error-

[srv netmiko]$ git clone https://github.com/ktbyers/netmiko.git Cloning into 'netmiko'... remote: Enumerating objects: 32, done. remote: Counting objects: 100% (32/32), done. remote: Compressing objects: 100% (28/28), done. remote: Total 12485 (delta 13), reused 14 (delta 4), pack-reused 12453 Receiving objects: 100% (12485/12485), 3.52 MiB | 2.51 MiB/s, done. Resolving deltas: 100% (8188/8188), done.

After this, I ran the code & it failed with the same error- "TypeError: send_command() got an unexpected keyword argument 'cmd_verify'"

Guess I am missing some steps in cloning the branch?

[ktbyers]

You need to cd netmiko (i.e. CD into the branch you just cloned).

Then pip install -e . that branch (i.e. pip install the branch you just cloned).

Note, I assume you are doing this in a Python virtual environment as this will change the Netmiko that you are using.

[satnau]

You need to cd netmiko (i.e. CD into the branch you just cloned).

Then pip install -e . that branch (i.e. pip install the branch you just cloned).

Note, I assume you are doing this in a Python virtual environment as this will change the Netmiko that you are using.

Hi Kirk,

I followed the instructions & still getting the same error :(

[satnau@srv netmiko]$ pwd /home/satnau/Netmiko_GIT/netmiko

[satnau@srv netmiko]$ pip3 install --user . Processing /home/satnau/Netmiko_GIT/netmiko Requirement already satisfied: setuptools>=38.4.0 in /usr/lib/python3.6/site-packages (from netmiko==3.0.0) (39.2.0) Requirement already satisfied: paramiko>=2.4.3 in /home/satnau/.local/lib/python3.6/site-packages (from netmiko==3.0.0) (2.7.1) Requirement already satisfied: scp>=0.13.2 in /usr/local/lib/python3.6/site-packages (from netmiko==3.0.0) (0.13.2) Requirement already satisfied: pyserial in /usr/local/lib/python3.6/site-packages (from netmiko==3.0.0) (3.4) Requirement already satisfied: textfsm in /home/satnau/.local/lib/python3.6/site-packages (from netmiko==3.0.0) (1.1.0) Requirement already satisfied: bcrypt>=3.1.3 in /usr/local/lib64/python3.6/site-packages (from paramiko>=2.4.3->netmiko==3.0.0) (3.1.6) Requirement already satisfied: cryptography>=2.5 in /usr/local/lib64/python3.6/site-packages (from paramiko>=2.4.3->netmiko==3.0.0) (2.8) Requirement already satisfied: pynacl>=1.0.1 in /usr/local/lib64/python3.6/site-packages (from paramiko>=2.4.3->netmiko==3.0.0) (1.3.0) Requirement already satisfied: future in /usr/local/lib/python3.6/site-packages (from textfsm->netmiko==3.0.0) (0.17.1) Requirement already satisfied: six in /usr/local/lib/python3.6/site-packages (from textfsm->netmiko==3.0.0) (1.11.0) Requirement already satisfied: cffi>=1.1 in /usr/local/lib64/python3.6/site-packages (from bcrypt>=3.1.3->paramiko>=2.4.3->netmiko==3.0.0) (1.12.3) Requirement already satisfied: pycparser in /usr/local/lib/python3.6/site-packages (from cffi>=1.1->bcrypt>=3.1.3->paramiko>=2.4.3->netmiko==3.0.0) (2.19) Installing collected packages: netmiko Running setup.py install for netmiko ... done Successfully installed netmiko-3.0.0

Traceback (most recent call last): File "github_netwmiko_issue.py", line 4, in grep_op = secmt_connect.send_command_expect(f'grep -w x.x.x.x/32 .set | grep book', expect_string = 'secmt03 junos-firewall', cmd_verify=False) File "/home/satnaidu/Netmiko_GIT/netmiko/netmiko/base_connection.py", line 1447, in send_command_expect return self.send_command(args, **kwargs) TypeError: send_command() got an unexpected keyword argument 'cmd_verify' <<<<<<

[satnau@srvnetmiko]$ pwd /home/satnau/Netmiko_GIT/netmiko/netmiko

[satnau@srvnetmiko]$ ls | grep base_conn base_connection.py

[satnau@gimli netmiko]$ cat base_connection.py| grep cmd_verify cmd_verify=True, :param cmd_verify: Whether or not to verify command echo for each command in config_set :type cmd_verify: bool elif not cmd_verify:

Thanks, Satish

[satnau]

You need to cd netmiko (i.e. CD into the branch you just cloned). Then pip install -e . that branch (i.e. pip install the branch you just cloned). Note, I assume you are doing this in a Python virtual environment as this will change the Netmiko that you are using.

Hi Kirk,

I followed the instructions & still getting the same error :(

[satnau@srv netmiko]$ pwd /home/satnau/Netmiko_GIT/netmiko

[satnau@srv netmiko]$ pip3 install --user . Processing /home/satnau/Netmiko_GIT/netmiko Requirement already satisfied: setuptools>=38.4.0 in /usr/lib/python3.6/site-packages (from netmiko==3.0.0) (39.2.0) Requirement already satisfied: paramiko>=2.4.3 in /home/satnau/.local/lib/python3.6/site-packages (from netmiko==3.0.0) (2.7.1) Requirement already satisfied: scp>=0.13.2 in /usr/local/lib/python3.6/site-packages (from netmiko==3.0.0) (0.13.2) Requirement already satisfied: pyserial in /usr/local/lib/python3.6/site-packages (from netmiko==3.0.0) (3.4) Requirement already satisfied: textfsm in /home/satnau/.local/lib/python3.6/site-packages (from netmiko==3.0.0) (1.1.0) Requirement already satisfied: bcrypt>=3.1.3 in /usr/local/lib64/python3.6/site-packages (from paramiko>=2.4.3->netmiko==3.0.0) (3.1.6) Requirement already satisfied: cryptography>=2.5 in /usr/local/lib64/python3.6/site-packages (from paramiko>=2.4.3->netmiko==3.0.0) (2.8) Requirement already satisfied: pynacl>=1.0.1 in /usr/local/lib64/python3.6/site-packages (from paramiko>=2.4.3->netmiko==3.0.0) (1.3.0) Requirement already satisfied: future in /usr/local/lib/python3.6/site-packages (from textfsm->netmiko==3.0.0) (0.17.1) Requirement already satisfied: six in /usr/local/lib/python3.6/site-packages (from textfsm->netmiko==3.0.0) (1.11.0) Requirement already satisfied: cffi>=1.1 in /usr/local/lib64/python3.6/site-packages (from bcrypt>=3.1.3->paramiko>=2.4.3->netmiko==3.0.0) (1.12.3) Requirement already satisfied: pycparser in /usr/local/lib/python3.6/site-packages (from cffi>=1.1->bcrypt>=3.1.3->paramiko>=2.4.3->netmiko==3.0.0) (2.19) Installing collected packages: netmiko Running setup.py install for netmiko ... done Successfully installed netmiko-3.0.0

Traceback (most recent call last): File "github_netwmiko_issue.py", line 4, in grep_op = secmt_connect.send_command_expect(f'grep -w x.x.x.x/32 .set | grep book', expect_string = 'secmt junos-firewall', cmd_verify=False) File "/home/satnaidu/Netmiko_GIT/netmiko/netmiko/base_connection.py", line 1447, in send_command_expect return self.send_command(args, **kwargs) TypeError: send_command() got an unexpected keyword argument 'cmd_verify' <<<<<<

[satnau@srvnetmiko]$ pwd /home/satnau/Netmiko_GIT/netmiko/netmiko

[satnau@srv netmiko]$ ls | grep base_conn base_connection.py

[satnau@srv netmiko]$ cat base_connection.py| grep cmd_verify cmd_verify=True, :param cmd_verify: Whether or not to verify command echo for each command in config_set :type cmd_verify: bool elif not cmd_verify:

Thanks, Satish

[AuthenticAMD]

Seeing the same thing here when using the same versions as Satnau, but connecting to Cisco NXOS devices.

I downloaded the development repo and can confirm if I use net_connect.send_command(cmd, cmd_verify=False) on longer command lines, then it works fine.

Unfortunately I'll likely roll back to 2.4.2 as I have a lot of scripts that have long commands

[ktbyers]

@satnau Does changing your column width on the Netmiko-linux SSH session prior to executing your long command fix your issue?

net_connect.send_command('stty cols 1023')

Kirk

[ktbyers]

@AuthenticAMD I suspect your issue is the following:

https://github.com/ktbyers/netmiko/issues/1587

It would be interesting to see if this fixes your issue terminal width 511

i.e. configuring that prior to executing your long-commands.

[AuthenticAMD]

@ktbyers

It seems like it works with your suggestion.

[ktbyers]

@AuthenticAMD Okay, I will put a PR into the NX-OS driver to automatically set that terminal width after login.

[satnau]

@satnau Does changing your column width on the Netmiko-linux SSH session prior to executing your long command fix your issue?

net_connect.send_command('stty cols 1023')

Kirk

@kirk - Sorry for the delayed reponse. I will check this and let you know.

[satnau]

@ktbyers - Yes the solution that you provided worked like a charm! Thanks a ton..