Describe the bug
When running more than 1 command against a Cisco IOS-XE network device a SessionError exception is raised but is empty (no message or stack), this doesn't occur against NXOS devices or any other Linux host I was able to test against (Ubuntu, RHEL, CentOS) but does occur on both vIOS (using CML/Cisco Modeling Labs) and physical IOS-XE devices
To Reproduce
Steps to reproduce the behavior:
Connect to a Cisco IOS-XE device (virtual or physical)
Run 2 or more commands (commands are arbitrary as long as they are valid)
Expected behavior
HostOutput object is returned and stdout generator is iterable to get the command output
Actual behaviour
First command succeeds and returns as expected, 2nd command fails with a pssh.exceptions.SessionError that is empty and contains no detail or messages
Code
import logging
from pssh.clients import SSHClient
from pssh.utils import enable_debug_logger
def main():
logging.basicConfig(encoding='utf-8', level=logging.DEBUG, format='%(asctime)s [%(levelname)s]: %(message)s')
ssh_client = SSHClient(host="192.168.1.17", user="cisco", password="cisco", pkey=None, port=22)
logging.getLogger('pssh.host_logger').setLevel(logging.WARNING)
enable_debug_logger()
command_output_one = ssh_client.run_command(command="show ip interface brief")
for line in command_output_one.stdout:
logging.info(f"Command output: {line}")
command_output_two = ssh_client.run_command(command="show inventory")
for line in command_output_two.stdout:
logging.info(f"Command output: {line}")
command_output_three = ssh_client.run_command(command="show switch")
for line in command_output_three.stdout:
logging.info(f"Command output: {line}")
if __name__ == '__main__':
main()
Error Output
2024-03-30 23:11:12,197 [DEBUG]: Connecting to 192.168.1.17:22
2024-03-30 23:11:12,332 [DEBUG]: Agent auth failed with AgentConnectionError('Unable to connect to agent') continuing with other authentication methods
2024-03-30 23:11:12,332 [DEBUG]: Trying to authenticate with identity file /home/user/.ssh/id_ed25519
2024-03-30 23:11:14,346 [DEBUG]: Authentication with identity file /home/user/.ssh/id_ed25519 failed with , continuing with other identities
2024-03-30 23:11:14,346 [DEBUG]: Private key auth failed, trying password
2024-03-30 23:11:14,409 [DEBUG]: Authentication completed successfully - setting session to non-blocking mode
2024-03-30 23:11:14,419 DEBUG pssh.clients.native.single Executing command 'b'show ip interface brief''
2024-03-30 23:11:14,419 [DEBUG]: Executing command 'b'show ip interface brief''
2024-03-30 23:11:14,439 DEBUG pssh.clients.base.single Reading from stdout buffer, timeout=None
2024-03-30 23:11:14,439 [DEBUG]: Reading from stdout buffer, timeout=None
2024-03-30 23:11:14,440 [INFO]: Command output:
2024-03-30 23:11:14,440 [INFO]: Command output:
2024-03-30 23:11:14,452 [INFO]: Command output:
2024-03-30 23:11:14,452 [INFO]: Command output: Interface IP-Address OK? Method Status Protocol
2024-03-30 23:11:14,452 [INFO]: Command output: GigabitEthernet0/0 192.168.1.17 YES DHCP up up
2024-03-30 23:11:14,452 [INFO]: Command output: GigabitEthernet0/1 unassigned YES unset administratively down down
2024-03-30 23:11:14,452 [INFO]: Command output: GigabitEthernet0/2 unassigned YES unset administratively down down
2024-03-30 23:11:14,452 [INFO]: Command output: GigabitEthernet0/3 10.150.1.0 YES TFTP up up
2024-03-30 23:11:14,567 [INFO]: Command output: Loopback100 10.100.1.10 YES TFTP up up
Traceback (most recent call last):
File "/home/user/venv/lib/python3.10/site-packages/pssh/clients/native/single.py", line 271, in open_session
chan = self._open_session()
^^^^^^^^^^^^^^^^^^^^
File "/home/user/venv/lib/python3.10/site-packages/pssh/clients/native/single.py", line 265, in _open_session
chan = self._eagain(self.session.open_session)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/user/venv/lib/python3.10/site-packages/pssh/clients/native/single.py", line 349, in _eagain
return self._eagain_errcode(func, LIBSSH2_ERROR_EAGAIN, *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/user/venv/lib/python3.10/site-packages/pssh/clients/base/single.py", line 576, in _eagain_errcode
ret = func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "ssh2/session.pyx", line 389, in ssh2.session.Session.open_session
File "ssh2/utils.pyx", line 214, in ssh2.utils.handle_error_codes
ssh2.exceptions.SocketRecvError
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/user/_general-testing/test.py", line 298, in <module>
main()
File "/home/user/_general-testing/test.py", line 288, in main
command_output_two = ssh_client.run_command(command="show inventory")
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/user/venv/lib/python3.10/site-packages/pssh/clients/base/single.py", line 555, in run_command
channel = self.execute(_command, use_pty=use_pty)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/user/venv/lib/python3.10/site-packages/pssh/clients/native/single.py", line 297, in execute
channel = self.open_session() if channel is None else channel
^^^^^^^^^^^^^^^^^^^
File "/home/user/venv/lib/python3.10/site-packages/pssh/clients/native/single.py", line 273, in open_session
raise SessionError(ex)
pssh.exceptions.SessionError
Workaround
I was able to get multiple commands working only if I do a ssh_client.disconnect() then do another full connection to the device
Troubleshooting Steps
Tried forcing the channel to close using ssh_client.close_channel(command_output_one.channel)
Tried both SSHClient and ParallelSSHClient with the same behavior
Tried explicitly defining the Init/Constructor args with low and high values (timeouts, no pty, no shell, no sudo, ect)
Tried on python3.9, python3.10 and python3.11 with the same result
Tested IOS-XE 15.9 vIOS, IOS-XE 16.3.x physical and IOS-XE 17.03 physical - all with the same failure
This failure is repeatable with both SSHClient and ParallelSSHClient, on ParallelSSHClient I tried also doing stop_on_errors=True on run_command hoping to get more failure detail but didn't see anything different
Describe the bug When running more than 1 command against a Cisco IOS-XE network device a SessionError exception is raised but is empty (no message or stack), this doesn't occur against NXOS devices or any other Linux host I was able to test against (Ubuntu, RHEL, CentOS) but does occur on both vIOS (using CML/Cisco Modeling Labs) and physical IOS-XE devices
To Reproduce
Steps to reproduce the behavior:
Expected behavior HostOutput object is returned and
stdoutgenerator is iterable to get the command outputActual behaviour First command succeeds and returns as expected, 2nd command fails with a
pssh.exceptions.SessionErrorthat is empty and contains no detail or messagesCode
Error Output
Workaround I was able to get multiple commands working only if I do a
ssh_client.disconnect()then do another full connection to the deviceTroubleshooting Steps
ssh_client.close_channel(command_output_one.channel)python3.9,python3.10andpython3.11with the same resultAdditional information pip freeze output:
Tested IOS-XE 15.9 vIOS, IOS-XE 16.3.x physical and IOS-XE 17.03 physical - all with the same failure
This failure is repeatable with both SSHClient and ParallelSSHClient, on ParallelSSHClient I tried also doing
stop_on_errors=Trueonrun_commandhoping to get more failure detail but didn't see anything different