Cisco 3560 SSH problem "paramiko.ssh_exception.SSHException: No existing session"

[kaage]

I have problem to connect to Cisco catalyst 3560 switch using Napalm:

>>> from netmiko import ConnectHandler
>>> from napalm import get_network_driver
>>> get_network_driver('ios')
<class 'napalm_ios.ios.IOSDriver'>
>>> driver = get_network_driver('ios')
>>> device = driver('<REMOVED>', '<REMOVED>', '<REMOVED>')
>>> device.open()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/jaakkorau/ansible-env/lib/python2.7/site-packages/napalm_ios/ios.py", line 116, in open
    **self.netmiko_optional_args)
  File "/home/jaakkorau/ansible-env/lib/python2.7/site-packages/netmiko/ssh_dispatcher.py", line 105, in ConnectHandler
    return ConnectionClass(*args, **kwargs)
  File "/home/jaakkorau/ansible-env/lib/python2.7/site-packages/netmiko/base_connection.py", line 144, in __init__
    self.establish_connection()
  File "/home/jaakkorau/ansible-env/lib/python2.7/site-packages/netmiko/base_connection.py", line 468, in establish_connection
    self.remote_conn_pre.connect(**ssh_connect_params)
  File "/home/jaakkorau/ansible-env/lib/python2.7/site-packages/paramiko/client.py", line 341, in connect
    server_key = t.get_remote_server_key()
  File "/home/jaakkorau/ansible-env/lib/python2.7/site-packages/paramiko/transport.py", line 670, in get_remote_server_key
    raise SSHException('No existing session')
paramiko.ssh_exception.SSHException: No existing session

Manual SSH enforced to version 2 works well. Output of show ip ssh is below:

SSH Enabled - version 2.0
Authentication methods:publickey,keyboard-interactive,password
Encryption Algorithms:aes128-ctr,aes192-ctr,aes256-ctr,aes128-cbc,3des-cbc,aes192-cbc,aes256-cbc
MAC Algorithms:hmac-sha1,hmac-sha1-96
Authentication timeout: 120 secs; Authentication retries: 3
Minimum expected Diffie Hellman key size : 1024 bits
IOS Keys in SECSH format(ssh-rsa, base64 encoded):
ssh-rsa <KEY_REMOVED>

[kaage]

I did following to get more debug:

>>> import logging
>>> logging.basicConfig(filename='test.log', level=logging.DEBUG)
>>> logger = logging.getLogger("netmiko")
>>> 
>>> from netmiko import ConnectHandler
>>> from napalm import get_network_driver
>>> get_network_driver('ios')
<class 'napalm_ios.ios.IOSDriver'>
>>> driver = get_network_driver('ios')
>>> device = driver('<removed>', '<removed>', '<removed>')
>>> device.open()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/jaakkorau/ansible-env/lib/python2.7/site-packages/napalm_ios/ios.py", line 116, in open
    **self.netmiko_optional_args)
  File "/home/jaakkorau/ansible-env/lib/python2.7/site-packages/netmiko/ssh_dispatcher.py", line 105, in ConnectHandler
    return ConnectionClass(*args, **kwargs)
  File "/home/jaakkorau/ansible-env/lib/python2.7/site-packages/netmiko/base_connection.py", line 144, in __init__
    self.establish_connection()
  File "/home/jaakkorau/ansible-env/lib/python2.7/site-packages/netmiko/base_connection.py", line 468, in establish_connection
    self.remote_conn_pre.connect(**ssh_connect_params)
  File "/home/jaakkorau/ansible-env/lib/python2.7/site-packages/paramiko/client.py", line 341, in connect
    server_key = t.get_remote_server_key()
  File "/home/jaakkorau/ansible-env/lib/python2.7/site-packages/paramiko/transport.py", line 670, in get_remote_server_key
    raise SSHException('No existing session')
paramiko.ssh_exception.SSHException: No existing session

and this produced following log:

DEBUG:paramiko.transport:starting thread (client mode): 0x91863110L
DEBUG:paramiko.transport:Local version/idstring: SSH-2.0-paramiko_2.1.1
DEBUG:paramiko.transport:Remote version/idstring: SSH-2.0-Cisco-1.25
INFO:paramiko.transport:Connected (version 2.0, client Cisco-1.25)
DEBUG:paramiko.transport:kex algos:[u'diffie-hellman-group-exchange-sha1', u'diffie-hellman-group14-sha1', u'diffie-hellman-group1-sha1'] server key:[u'ssh-rsa'] client encrypt:[u'aes128-ctr', u'aes192-ctr', u'aes256-ctr', u'aes128-cbc', u'3des-cbc', u'aes192-cbc', u'aes256-cbc'] server encrypt:[u'aes128-ctr', u'aes192-ctr', u'aes256-ctr', u'aes128-cbc', u'3des-cbc', u'aes192-cbc', u'aes256-cbc'] client mac:[u'hmac-sha1', u'hmac-sha1-96'] server mac:[u'hmac-sha1', u'hmac-sha1-96'] client compress:[u'none'] server compress:[u'none'] client lang:[u''] server lang:[u''] kex follows?False
DEBUG:paramiko.transport:Kex agreed: diffie-hellman-group1-sha1
DEBUG:paramiko.transport:Cipher agreed: aes128-ctr
DEBUG:paramiko.transport:MAC agreed: hmac-sha1-96
DEBUG:paramiko.transport:Compression agreed: none
DEBUG:paramiko.transport:kex engine KexGroup1 specified hash_algo <built-in function openssl_sha1>
DEBUG:paramiko.transport:Switch to new keys ...
DEBUG:paramiko.transport:EOF in transport thread

[toghraee]

I have the same problem. the error is from Paramiko

python2.7

import paramiko ssh = paramiko.SSHClient() ssh.load_system_host_keys('/root/.ssh/known_hosts') ssh.set_missing_host_key_policy(paramiko.WarningPolicy()) ssh.connect('celort18', username='user',password='pass', key_filename='/root/.ssh/id_rsa', allow_agent='False')

Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/usr/local/lib/python2.7/site-packages/paramiko/client.py", line 381, in connect look_for_keys, gss_auth, gss_kex, gss_deleg_creds, gss_host) File "/usr/local/lib/python2.7/site-packages/paramiko/client.py", line 622, in _auth raise saved_exception paramiko.ssh_exception.SSHException: No existing session

[ktbyers]

@kaage What does debug ip ssh on the router side show?

[kaage]

hostname#debug ip ssh
Incoming SSH debugging is on
hostname#
May 30 2017 09:56:28.919 EEST: SSH2 0: send:packet of  length 80 (length also includes padlen of 17)
May 30 2017 09:56:28.919 EEST: SSH2 0: computed MAC for sequence no.#401 type 94
May 30 2017 09:56:30.815 EEST: SSH2 0: send:packet of  length 208 (length also includes padlen of 10)
May 30 2017 09:56:30.815 EEST: SSH2 0: computed MAC for sequence no.#402 type 94
May 30 2017 09:56:30.815 EEST: SSH2 0: send:packet of  length 48 (length also includes padlen of 13)
May 30 2017 09:56:30.815 EEST: SSH2 0: computed MAC for sequence no.#403 type 94
May 30 2017 09:56:32.132 EEST: SSH1: starting SSH control process
May 30 2017 09:56:32.132 EEST: SSH1: sent protocol version id SSH-2.0-Cisco-1.25
May 30 2017 09:56:32.132 EEST: SSH1: protocol version id is - SSH-2.0-paramiko_2.1.1
May 30 2017 09:56:32.132 EEST: SSH2 1: kexinit sent: encryption algo = aes128-ctr,aes192-ctr,aes256-ctr,aes128-cbc,3des-cbc,aes192-cbc,aes256-cbc
May 30 2017 09:56:32.132 EEST: SSH2 1: kexinit sent: mac algo = hmac-sha1,hmac-sha1-96
May 30 2017 09:56:32.132 EEST: SSH2 1: send:packet of  length 368 (length also includes padlen of 5)
May 30 2017 09:56:32.132 EEST: SSH2 1: SSH2_MSG_KEXINIT sent
May 30 2017 09:56:32.132 EEST: SSH2 1: ssh_receive: 640 bytes received 
May 30 2017 09:56:32.141 EEST: SSH2 1: input: total packet length of 640 bytes
May 30 2017 09:56:32.141 EEST: SSH2 1: partial packet length(block size)8 bytes,needed 632 bytes,
               maclen 0
May 30 2017 09:56:32.141 EEST: SSH2 1: input: padlength 4 bytes
May 30 2017 09:56:32.141 EEST: SSH2 1: SSH2_MSG_KEXINIT received
May 30 2017 09:56:32.141 EEST: SSH2 1: kex: client->server enc:aes128-ctr mac:hmac-sha1-96 
May 30 2017 09:56:32.141 EEST: SSH2 1: kex: server->client enc:aes128-ctr mac:hmac-sha1-96 
May 30 2017 09:56:32.141 EEST: SSH2 1: Using kex_algo = diffie-hellman-group1-sha1
May 30 2017 09:56:32.233 EEST: SSH2 1: expecting SSH2_MSG_KEXDH_INIT
May 30 2017 09:56:32.334 EEST: SSH2 1: ssh_receive: 144 bytes received 
May 30 2017 09:56:32.334 EEST: SSH2 1: input: total packet length of 144 bytes
May 30 2017 09:56:32.334 EEST: SSH2 1: partial packet length(block size)8 bytes,needed 136 bytes,
               maclen 0
May 30 2017 09:56:32.334 EEST: SSH2 1: input: padlength 6 bytes
May 30 2017 09:56:32.334 EEST: SSH2 1: SSH2_MSG_KEXDH_INIT received
May 30 2017 09:56:32.845 EEST: SSH2 0: send:packet of  length 1712 (length also includes padlen of 17)
May 30 2017 09:56:32.845 EEST: SSH2 0: computed MAC for sequence no.#404 type 94
May 30 2017 09:56:32.870 EEST: SSH2 0: send:packet of  length 48 (length also includes padlen of 13)
May 30 2017 09:56:32.879 EEST: SSH2 0: computed MAC for sequence no.#405 type 94
May 30 2017 09:56:33.885 EEST: SSH2 0: send:packet of  length 688 (length also includes padlen of 7)
May 30 2017 09:56:33.885 EEST: SSH2 0: computed MAC for sequence no.#406 type 94
May 30 2017 09:56:33.910 EEST: SSH2 0: send:packet of  length 48 (length also includes padlen of 13)
May 30 2017 09:56:33.910 EEST: SSH2 0: computed MAC for sequence no.#407 type 94
May 30 2017 09:56:34.934 EEST: SSH2 0: send:packet of  length 576 (length also includes padlen of 10)
May 30 2017 09:56:34.934 EEST: SSH2 0: computed MAC for sequence no.#408 type 94
May 30 2017 09:56:34.976 EEST: SSH2 0: send:packet of  length 48 (length also includes padlen of 13)
May 30 2017 09:56:34.976 EEST: SSH2 0: computed MAC for sequence no.#409 type 94
May 30 2017 09:56:36.989 EEST: SSH2 0: send:packet of  length 400 (length also includes padlen of 17)
May 30 2017 09:56:36.989 EEST: SSH2 0: computed MAC for sequence no.#410 type 94
May 30 2017 09:56:37.048 EEST: SSH2 0: send:packet of  length 48 (length also includes padlen of 13)
May 30 2017 09:56:37.048 EEST: SSH2 0: computed MAC for sequence no.#411 type 94
May 30 2017 09:56:39.053 EEST: SSH2 0: send:packet of  length 400 (length also includes padlen of 17)
May 30 2017 09:56:39.053 EEST: SSH2 0: computed MAC for sequence no.#412 type 94
May 30 2017 09:56:39.120 EEST: SSH2 0: send:packet of  length 48 (length also includes padlen of 13)
May 30 2017 09:56:39.120 EEST: SSH2 0: computed MAC for sequence no.#413 type 94
hostname#undebug all 
May 30 2017 09:56:40.672 EEST: SSH2 1: signature length 527
May 30 2017 09:56:40.672 EEST: SSH2 1: send:packet of  length 1216 (length also includes padlen of 7)
May 30 2017 09:56:40.772 EEST: SSH1: Session disconnected - error 0x07
May 30 2017 09:56:41.108 EEST: SSH2 0: send:packet of  length 400 (length also includes padlen of 17)
May 30 2017 09:56:41.108 EEST: SSH2 0: computed MAC for sequence no.#414 type 94
May 30 2017 09:56:41.116 EEST: SSH2 0: send:packet of  length 48 (length also includes padlen of 13)
May 30 2017 09:56:41.116 EEST: SSH2 0: computed MAC for sequence no.#415 type 94
May 30 2017 09:56:41.510 EEST: SSH2 0: ssh_receive: 52 bytes received 
May 30 2017 09:56:41.510 EEST: SSH2 0: input: total packet length of 32 bytes
May 30 2017 09:56:41.510 EEST: SSH2 0: partial packet length(block size)16 bytes,needed 16 bytes,
               maclen 20
May 30 2017 09:56:41.510 EEST: SSH2 0: MAC compared for #210 :ok
May 30 2017 09:56:41.510 EEST: SSH2 0: input: padlength 15 bytes
May 30 2017 09:56:41.510 EEST: SSH2 0: send:packet of  length 32 (length also includes padlen of 6)
May 30 2017 09:56:41.510 EEST: SSH2 0: computed MAC for sequence no.#416 type 94
May 30 2017 09:56:42.106 EEST: SSH2 0: ssh_receive: 52 bytes received 
May 30 2017 09:56:42.106 EEST: SSH2 0: input: total packet length of 32 bytes
hostname#undebug all
All possible debugging has been turned off
hostname#
May 30 2017 09:56:42.106 EEST: SSH2 0: partial packet length(block size)16 bytes,needed 16 bytes,
               maclen 20
May 30 2017 09:56:42.106 EEST: SSH2 0: MAC compared for #211 :ok
May 30 2017 09:56:42.106 EEST: SSH2 0: input: padlength 15 bytes
May 30 2017 09:56:42.106 EEST: SSH2 0: send:packet of  length 48 (length also includes padlen of 9)
May 30 2017 09:56:42.106 EEST: SSH2 0: computed MAC for sequence no.#417 type 94
May 30 2017 09:56:42.123 EEST: SSH2 0: send:packet of  length 1376 (length also includes padlen of 15)
May 30 2017 09:56:42.123 EEST: SSH2 0: computed MAC for sequence no.#418 type 94
May 30 2017 09:56:42.123 EEST: SSH2 0: send:packet of  length 64 (length also includes padlen of 18)
May 30 2017 09:56:42.123 EEST: SSH2 0: computed MAC for sequence no.#419 type 94
May 30 2017 09:56:42.509 EEST: SSH2 0: ssh_receive: 52 bytes received 
May 30 2017 09:56:42.517 EEST: SSH2 0: input: total packet length of 32 bytes
May 30 2017 09:56:42.517 EEST: SSH2 0: partial packet length(block size)16 bytes,needed 16 bytes,
               maclen 20
May 30 2017 09:56:42.517 EEST: SSH2 0: MAC compared for #212 :ok
May 30 2017 09:56:42.517 EEST: SSH2 0: input: padlength 17 bytes
hostname#
May 30 2017 09:56:42.517 EEST: SSH2 0: send:packet of  length 32 (length also includes padlen of 16)
May 30 2017 09:56:42.517 EEST: SSH2 0: computed MAC for sequence no.#420 type 94

[toghraee]

What I found today was that I have the same issue and same debug log as sent by Kaag on my Cisco Catalyst 6500 , running IOS s72033-advipservicesk9-mz.151-2.SY10

However, today I tried to connect a Cisco 3850 running cat3k_caa-universalk9.16.03.02.SPA interestingly it worked on Cisco 3850.

I captured both paramiko logs on 6500 and 3850 to compare :

============6500============ paramiko.transport: Connected (version 2.0, client Cisco-1.25) paramiko.transport: kex algos:[u'diffie-hellman-group-exchange-sha1', u'diffie-hellman-group14-sha1', u'diffie-hellman-group1-sha1'] server key:[u'ssh-rsa'] client encrypt:[u'aes128-cbc', u'3des-cbc', u'aes192-cbc', u'aes256-cbc'] server encrypt:[u'aes128-cbc', u'3des-cbc', u'aes192-cbc', u'aes256-cbc'] client mac:[u'hmac-sha1', u'hmac-sha1-96', u'hmac-md5', u'hmac-md5-96'] server mac:[u'hmac-sha1', u'hmac-sha1-96', u'hmac-md5', u'hmac-md5-96'] client compress:[u'none'] server compress:[u'none'] client lang:[u''] server lang:[u''] kex follows?False paramiko.transport: Kex agreed: diffie-hellman-group1-sha1 paramiko.transport: Cipher agreed: aes128-cbc paramiko.transport: MAC agreed: hmac-md5 paramiko.transport: Compression agreed: none paramiko.transport: kex engine KexGroup1 specified hash_algo paramiko.transport: Switch to new keys ... paramiko.transport: Trying key 542f7f11dcaafae42ec947dbf96bac97 from /root/.ssh/id_rsa paramiko.transport: userauth is OK paramiko.transport: Exception: Illegal info request from server paramiko.transport: Traceback (most recent call last): paramiko.transport: File "/usr/local/lib/python2.7/site-packages/paramiko/transport.py", line 1800, in run paramiko.transport: self.auth_handler._handler_table[ptype](self.auth_handler, m) paramiko.transport: File "/usr/local/lib/python2.7/site-packages/paramiko/auth_handler.py", line 575, in _parse_userauth_info_request paramiko.transport: raise SSHException('Illegal info request from server') paramiko.transport: SSHException: Illegal info request from server paramiko.transport: paramiko.transport: Trying SSH agent key 542f7f11dcaafae42ec947dbf96bac97 paramiko.transport: Trying discovered key 542f7f11dcaafae42ec947dbf96bac97 in /root/.ssh/id_rsa

============3850============ paramiko.transport: starting thread (client mode): 0xdeb37810L paramiko.transport: Local version/idstring: SSH-2.0-paramiko_2.1.2 paramiko.transport: Remote version/idstring: SSH-2.0-Cisco-1.25 paramiko.transport: Connected (version 2.0, client Cisco-1.25) paramiko.transport: kex algos:[u'diffie-hellman-group-exchange-sha1', u'diffie-hellman-group14-sha1'] server key:[u'ssh-rsa'] client encrypt:[u'aes128-ctr', u'aes192-ctr', u'aes256-ctr', u'aes128-cbc', u'3des-cbc', u'aes192-cbc', u'aes256-cbc'] server encrypt:[u'aes128-ctr', u'aes192-ctr', u'aes256-ctr', u'aes128-cbc', u'3des-cbc', u'aes192-cbc', u'aes256-cbc'] client mac:[u'hmac-sha1', u'hmac-sha1-96'] server mac:[u'hmac-sha1', u'hmac-sha1-96'] client compress:[u'none'] server compress:[u'none'] client lang:[u''] server lang:[u''] kex follows?False paramiko.transport: Kex agreed: diffie-hellman-group14-sha1 paramiko.transport: Cipher agreed: aes128-ctr paramiko.transport: MAC agreed: hmac-sha1-96 paramiko.transport: Compression agreed: none paramiko.transport: kex engine KexGroup14 specified hash_algo paramiko.transport: Switch to new keys ... paramiko.transport: Trying key 542f7f11dcaafae42ec947dbf96bac97 from /root/.ssh/id_rsa paramiko.transport: userauth is OK paramiko.transport: Authentication (publickey) successful! paramiko.transport: EOF in transport thread

===============================

The difference between the 2 is :

3850 : Kex agreed: diffie-hellman-group14-sha1 6500: Kex agreed: diffie-hellman-group1-sha1

3850: Cipher agreed: aes128-ctr 6500: Cipher agreed: aes128-cbc

3550: MAC agreed: hmac-sha1-96 6500: MAC agreed: hmac-md5

===========================

This seems to be related to Cisco IOS implementation of SSH on 6500 or 3560

6509#sh ip ssh SSH Enabled - version 2.0 Authentication timeout: 120 secs; Authentication retries: 3 Minimum expected Diffie Hellman key size : 1024 bits

3850#sh ip ssh SSH Enabled - version 2.0 Authentication methods:publickey,keyboard-interactive,password Authentication Publickey Algorithms:x509v3-ssh-rsa,ssh-rsa Hostkey Algorithms:x509v3-ssh-rsa,ssh-rsa Encryption Algorithms:aes128-ctr,aes192-ctr,aes256-ctr,aes128-cbc,3des-cbc,aes192-cbc,aes256-cbc MAC Algorithms:hmac-sha1,hmac-sha1-96 KEX Algorithms:diffie-hellman-group-exchange-sha1,diffie-hellman-group14-sha1 Authentication timeout: 120 secs; Authentication retries: 3 Minimum expected Diffie Hellman key size : 2048 bits

===My Python File ====

root@HOST~ cat test_paramiko_cisco.py import logging import paramiko

logging.getLogger("paramiko").setLevel(logging.DEBUG) ssh = paramiko.SSHClient() ssh.load_system_host_keys('/root/.ssh/known_hosts') ssh.set_missing_host_key_policy(paramiko.WarningPolicy()) paramiko.util.log_to_file("/root/paramiko.log") ssh.connect('SWITCH_IP', username='USERNAME',password='KEY_PASS_PHRASE', key_filename='/root/.ssh/id_rsa', allow_agent='False') remote_conn=ssh.invoke_shell() remote_conn.send("show run\n") output=remote_conn.recv(5000) print output

I'm still looking forward to see how to fix the issue with IOS / 6500 .

Regards Reza Toghraee

[ktbyers]

@toghraee For your issue, can use see what happens if you change to the following:

ssh.connect('SWITCH_IP',  username='USERNAME', password='PASSWORD', 
      use_keys=False, allow_agent=False)

i.e. if you use username/password authentication and not key authentication. Note, both use_keys and allow_agent are set to boolean False.

[ktbyers]

@kaage I wonder if it might be this Cisco bug:

https://bst.cloudapps.cisco.com/bugsearch/bug/CSCsa83601/?referring_site=bugquickviewredir

Symptom:
new SSH sessions fail after changing the hostname

Conditions:
Change the hostname

You can use the following command to see if the key name matches the hostname/domain name:

# show crypto key mypubkey rsa | inc Key name
Key name: TP-self-signed-1429897839
Key name: pynet-rtr1.twb-tech.com
Key name: pynet-rtr1.twb-tech.com.server

[ktbyers]

@toghraee I think your issue is probably a separate issue (so you should re-open a separate issue in Netmiko) as otherwise that will just cause problems to get conflated.

[toghraee]

@ktbyers Thanks Kirk. actually my environment is purely public key authentication based with no password.

this command : ssh.connect('SWITCH_IP', username='USERNAME', password='PASSWORD', use_keys=False, allow_agent=False)

didnt work even on my 3850 switches which works with public key authentication using below command ssh.connect('SWITCH_IP', username='USERNAME',password='KEY_PASS_PHRASE', key_filename='/root/.ssh/id_rsa', allow_agent='False')

This issue seems to be resolved in Cisco IOS 15.5(2) onwards. This IOS supports specifying the ciphers using ip ssh server algorithm encryption command.

http://www.cisco.com/c/en/us/td/docs/ios-xml/ios/security/d1/sec-d1-cr-book/sec-cr-i3.html#wp9081909290

unfortunately this IOS is not supported on 6500.

best regards Reza Toghraee

[kaage]

Hi, my crypto key names are like the ones below:

hostname#show crypto key mypubkey rsa | include Key name
Key name: hostname.domain.com
Key name: hostname.domain.com.server

I don't think this is that bug as I'm able to login via SSH. Also ios_facts module of Ansible works well with this device.

[ktbyers]

@toghraee Yes, I wanted to have you test with username/password as I wanted to isolate the problem (i.e. whether it pertained to key authentication).

I vaguely recall that the error message made me believe it was a key authentication issue (from searching on it)

Note, I am pretty sure people are using Netmiko with Cat6500s.

Like I said, open a separate issue if you want my help (as it is a different issue than the one referenced here). I expect we can get it to work.

[ktbyers]

@kaage Good points...

Let me look at what Ansible is doing (and see if they are even using Paramiko in Ansible 2.3 for ios_facts).

[ktbyers]

@kaage Okay, I don't think Ansible is using Paramiko any more in Ansible 2.3 (so it working there doesn't really tell us much).

It would be interesting to know if ios_facts in Ansible 2.2 works or not (they might still be using Paramiko in Ansible 2.2).

[ktbyers]

@kaage What IOS version are you running?

Can you try re-generating your SSH keys on the router?

Also, can you ensure the new key is in the SSH known-hosts file on the server running netmiko (i.e. manually SSH to the router once from this machine).

[ktbyers]

My reading of these messages...is router closes the SSH session "EOF in transport".

Router says the following:

May 30 2017 09:56:40.772 EEST: SSH1: Session disconnected - error 0x07

Strange it says SSH1 (and says SSH1 earlier). Cisco bug I read yesterday reported this as a documentation bug.

[kaage]

@ktbyers I'm running IOS release 15.2(3)E1. Regenerating SSH keys didn't help. SSH keys are in known-hosts file. I'm wondering if this could be performance / timing related issue as 3560 is old and not very powerfull architecture?

Session disconnected message appeared in the router debug way after Paramiko raised error. As you can see in my log message I was already typed "undebug all" as I was thinking I already got all logs nessessacy. I think router just disconnects as paramiko doesn't talk to it anymore.

[ktbyers]

Yes, Cisco logging generally is not close to real time (i.e. a message will come out a considerable time after it happened especially when there are quite a few messages).

So I still think the router message is the relevant message.

Paramiko just checks whether the session is alive in t.get_remote_server_key() and determines it isn't.

So I still think it is the network device that is probably closing the SSH session.

Not sure where we go from here though...

[kaage]

Can I somehow ensure paramiko is using SSHv2?

[ktbyers]

Paramiko only uses SSHv2; it doesn't use, have support for SSHv1...that is all from Cisco. There is a Cisco bug that states the reporting of SSHv1 here is just a documentation bug (i.e. that it is actually SSH2)

SSH1: Session disconnected - error 0x07

I am not sure I believe it, however, since it was reporting SSHv1 up higher.

[filintod]

I saw this error recently with paramiko, and the problem was not enough time to connect.

Add a timeout to the connection before opening:

device.nemiko_optional_args['timeout'] = 10

For some reason (possible a bug) the timeout is not part of the possible optional_args during get_network_driver.

[ktbyers]

@filintod Netmiko defaults to 8 seconds...so the difference between 8 and 10 seconds mattered in your context?

[filintod]

@ktbyers I was using paramiko straight not via netmiko but was looking for a solution to this problem NoExistingSession (same method get_remote_server_key) and the solution at the end was to increase the timeout when connecting. So I thought that maybe they could try with an extended timeout to see if that also solved their problem.

[princedhadwal]

Hi

Similar kind of issue I am observing, when I am login to routers from single file, I am able to login. But when I am trying to login using import statements I am getting error.

rtr4#

C:\Users\prince.dhadwal\Desktop\py_code\lib>cd ..\testcase

C:\Users\prince.dhadwal\Desktop\py_code\testcase>python TestCase1.py LOGIN FILE CALLED ---------------------py_code.lib.login Device Info file called [{'username': 'ocnos', 'ip': '10.12.29.117', 'password': 'ocnos', 'device_type': 'cisco_ios'}, {'username': 'ocnos', 'ip': '10.12.29.118', 'password': 'ocnos', 'device_type': 'cisco_ios'}] Module imported... Logginig to Device: 10.12.29.117 Traceback (most recent call last): File "TestCase1.py", line 1, in from py_code.lib import login File "C:\Users\prince.dhadwal\Desktop\py_code\lib\login.py", line 42, in <modu le>

File "C:\Users\prince.dhadwal\Desktop\py_code\lib\login.py", line 15, in conne ct_device

net_connect.find_prompt()

File "", line 1, in File "C:\Python27\lib\site-packages\netmiko\ssh_dispatcher.py", line 174, in C onnectHandler return ConnectionClass(*args, kwargs) File "C:\Python27\lib\site-packages\netmiko\base_connection.py", line 187, in init self.establish_connection() File "C:\Python27\lib\site-packages\netmiko\base_connection.py", line 650, in establish_connection self.remote_conn_pre.connect(ssh_connect_params) File "C:\Python27\lib\site-packages\paramiko\client.py", line 391, in connect server_key = t.get_remote_server_key() File "C:\Python27\lib\site-packages\paramiko\transport.py", line 718, in get_r emote_server_key raise SSHException('No existing session') paramiko.ssh_exception.SSHException: No existing session

C:\Users\prince.dhadwal\Desktop\py_code\testcase>

Please help

[ktbyers]

@princedhadwal Please post your code that is working. Please wrap your code in triple backticks so that it doesn't get distorted by the markdown.

[princedhadwal]

@ktbyers Please find the code.

print "LOGIN FILE CALLED"

from py_code.testbed.device_info import devices
from netmiko import ConnectHandler
import time

print devices

def connect_device(devices):

    i=1
    for device in devices:
        print("Logginig to Device: %s" % device['ip'])
        #exec('net_connect_%d  = ConnectHandler(**device)' %i)

        exec('out=net_connect_%d.enable()' %i)
        print out

        exec('out=net_connect_%d.find_prompt()' %i)
        print out

        cmd='terminal length 0'
        exec('output=net_connect_%d.send_command(cmd)' %i)
        i=i+1

if __name__ == '__main__':
    print "--------------Main called for login"
    connect_device(devices)
else :
    print "Module imported..."
    connect_device(devices)

When I am running this file standalone, no error is coming. Here I am logging in two devices, that info I am reading from some other file. But when I am importing this module in some other .py file then error is coming.

[ktbyers]

Okay, and what does your code look like that is failing?

[toghraee]

What device are you connecting to? 

Best regards Reza Toghraee Sent from Smartphone . Forgive the brevity and typos. -------- Original message --------From: princedhadwal notifications@github.com Date: 16/03/2018 18:12 (GMT+00:00) To: ktbyers/netmiko netmiko@noreply.github.com Cc: Reza Toghraee reza@toghraee.com, Mention mention@noreply.github.com Subject: Re: [ktbyers/netmiko] Cisco 3560 SSH problem "paramiko.ssh_exception.SSHException: No existing session" (#467) Hi Similar kind of issue I am observing, when I am login to routers from single file, I am able to login.

But when I am trying to login using import statements I am getting error. rtr4# C:\Users\prince.dhadwal\Desktop\py_code\lib>cd ..\testcase C:\Users\prince.dhadwal\Desktop\py_code\testcase>python TestCase1.py

LOGIN FILE CALLED

---------------------py_code.lib.login

Device Info file called

[{'username': 'ocnos', 'ip': '10.12.29.117', 'password': 'ocnos', 'device_type':

'cisco_ios'}, {'username': 'ocnos', 'ip': '10.12.29.118', 'password': 'ocnos',

'device_type': 'cisco_ios'}]

Module imported...

Logginig to Device: 10.12.29.117

Traceback (most recent call last):

File "TestCase1.py", line 1, in

from py_code.lib import login

File "C:\Users\prince.dhadwal\Desktop\py_code\lib\login.py", line 42, in File "C:\Users\prince.dhadwal\Desktop\py_code\lib\login.py", line 15, in conne

ct_device

net_connect.find_prompt()

File "", line 1, in

File "C:\Python27\lib\site-packages\netmiko\ssh_dispatcher.py", line 174, in C

onnectHandler

return ConnectionClass(*args, **kwargs)

File "C:\Python27\lib\site-packages\netmiko\base_connection.py", line 187, in

init

self.establish_connection()

File "C:\Python27\lib\site-packages\netmiko\base_connection.py", line 650, in

establish_connection

self.remote_conn_pre.connect(**ssh_connect_params)

File "C:\Python27\lib\site-packages\paramiko\client.py", line 391, in connect

server_key = t.get_remote_server_key()

File "C:\Python27\lib\site-packages\paramiko\transport.py", line 718, in get_r

emote_server_key

raise SSHException('No existing session')

paramiko.ssh_exception.SSHException: No existing session C:\Users\prince.dhadwal\Desktop\py_code\testcase>

Please help

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

{"api_version":"1.0","publisher":{"api_key":"05dde50f1d1a384dd78767c55493e4bb","name":"GitHub"},"entity":{"external_key":"github/ktbyers/netmiko","title":"ktbyers/netmiko","subtitle":"GitHub repository","main_image_url":"https://cloud.githubusercontent.com/assets/143418/17495839/a5054eac-5d88-11e6-95fc-7290892c7bb5.png","avatar_image_url":"https://cloud.githubusercontent.com/assets/143418/15842166/7c72db34-2c0b-11e6-9aed-b52498112777.png","action":{"name":"Open in GitHub","url":"https://github.com/ktbyers/netmiko"}},"updates":{"snippets":[{"icon":"PERSON","message":"@princedhadwal in #467: Hi \r\n\r\nSimilar kind of issue I am observing, when I am login to routers from single file, I am able to login.\r\nBut when I am trying to login using import statements I am getting error.\r\n\r\n\r\n\r\nrtr4#\r\n\r\nC:\Users\prince.dhadwal\Desktop\py_code\lib\u003ecd ..\testcase\r\n\r\nC:\Users\prince.dhadwal\Desktop\py_code\testcase\u003epython TestCase1.py\r\nLOGIN FILE CALLED\r\n---------------------py_code.lib.login\r\nDevice Info file called\r\n[{'username': 'ocnos', 'ip': '10.12.29.117', 'password': 'ocnos', 'device_type':\r\n 'cisco_ios'}, {'username': 'ocnos', 'ip': '10.12.29.118', 'password': 'ocnos',\r\n'device_type': 'cisco_ios'}]\r\nModule imported...\r\nLogginig to Device: 10.12.29.117\r\nTraceback (most recent call last):\r\n File \"TestCase1.py\", line 1, in \u003cmodule\u003e\r\n from py_code.lib import login\r\n File \"C:\Users\prince.dhadwal\Desktop\py_code\lib\login.py\", line 42, in \u003cmodu\r\nle\u003e\r\n\r\n File \"C:\Users\prince.dhadwal\Desktop\py_code\lib\login.py\", line 15, in conne\r\nct_device\r\n #net_connect.find_prompt()\r\n File \"\u003cstring\u003e\", line 1, in \u003cmodule\u003e\r\n File \"C:\Python27\lib\site-packages\netmiko\ssh_dispatcher.py\", line 174, in C\r\nonnectHandler\r\n return ConnectionClass(*args, kwargs)\r\n File \"C:\Python27\lib\site-packages\netmiko\base_connection.py\", line 187, in\r\ninit\r\n self.establish_connection()\r\n File \"C:\Python27\lib\site-packages\netmiko\base_connection.py\", line 650, in\r\nestablish_connection\r\n self.remote_conn_pre.connect(ssh_connect_params)\r\n File \"C:\Python27\lib\site-packages\paramiko\client.py\", line 391, in connect\r\n server_key = t.get_remote_server_key()\r\n File \"C:\Python27\lib\site-packages\paramiko\transport.py\", line 718, in get_r\r\nemote_server_key\r\n raise SSHException('No existing session')\r\nparamiko.ssh_exception.SSHException: No existing session\r\n\r\nC:\Users\prince.dhadwal\Desktop\py_code\testcase\u003e\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\nPlease help"}],"action":{"name":"View Issue","url":"https://github.com/ktbyers/netmiko/issues/467#issuecomment-373799913"}}}

[princedhadwal]

@toghraee- I am connecting into Trident 2 boards having Zebos binaries loaded in it. Basic CLI is similar to cisco devices to login and entering into global mode.

[toghraee]

I have tested netmiko with Trident 2 with ONL and Cumulus earlier and that was working perfectly. In general the ssh server which was on ONL and Cumulus was compatible with netmiko & paramiko.

Im not sure IPinfusion Ocnos and Zeboss. Can you get more detailed logs from paramiko?

From: princedhadwal [mailto:notifications@github.com] Sent: 19 March 2018 07:30 To: ktbyers/netmiko netmiko@noreply.github.com Cc: Reza Toghraee reza@toghraee.com; Mention mention@noreply.github.com Subject: Re: [ktbyers/netmiko] Cisco 3560 SSH problem "paramiko.ssh_exception.SSHException: No existing session" (#467)

@toghraee- I am connecting into Trident 2 boards having Zebos binaries loaded in it. Basic CLI is similar to cisco devices to login and entering into global mode.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/ktbyers/netmiko/issues/467#issuecomment-374122982 , or mute the thread https://github.com/notifications/unsubscribe-auth/ACPIp3nbbZbpDpNbMbZRQucRIZTVjykJks5tf16MgaJpZM4NYUr1 .

[princedhadwal]

Below are the paramiko logs

DEBUG:my_log:This message should go to the log file
DEBUG:my_log:This message should go to the log file
DEBUG:my_log:This message should go to the log file
DEBUG:paramiko.transport:starting thread (client mode): 0x30ab250L
DEBUG:paramiko.transport:Local version/idstring: SSH-2.0-paramiko_2.3.1
DEBUG:paramiko.transport:Remote version/idstring: SSH-2.0-OpenSSH_6.7p1 Debian-5+deb8u1
INFO:paramiko.transport:Connected (version 2.0, client OpenSSH_6.7p1)
DEBUG:paramiko.transport:kex algos:[u'curve25519-sha256@libssh.org', u'ecdh-sha2-nistp256', u'ecdh-sha2-nistp384', u'ecdh-sha2-nistp521', u'diffie-hellman-group-exchange-sha256', u'diffie-hellman-group14-sha1'] server key:[u'ssh-rsa', u'ssh-dss', u'ecdsa-sha2-nistp256', u'ssh-ed25519'] client encrypt:[u'aes128-ctr', u'aes192-ctr', u'aes256-ctr', u'aes128-gcm@openssh.com', u'aes256-gcm@openssh.com', u'chacha20-poly1305@openssh.com'] server encrypt:[u'aes128-ctr', u'aes192-ctr', u'aes256-ctr', u'aes128-gcm@openssh.com', u'aes256-gcm@openssh.com', u'chacha20-poly1305@openssh.com'] client mac:[u'umac-64-etm@openssh.com', u'umac-128-etm@openssh.com', u'hmac-sha2-256-etm@openssh.com', u'hmac-sha2-512-etm@openssh.com', u'hmac-sha1-etm@openssh.com', u'umac-64@openssh.com', u'umac-128@openssh.com', u'hmac-sha2-256', u'hmac-sha2-512', u'hmac-sha1'] server mac:[u'umac-64-etm@openssh.com', u'umac-128-etm@openssh.com', u'hmac-sha2-256-etm@openssh.com', u'hmac-sha2-512-etm@openssh.com', u'hmac-sha1-etm@openssh.com', u'umac-64@openssh.com', u'umac-128@openssh.com', u'hmac-sha2-256', u'hmac-sha2-512', u'hmac-sha1'] client compress:[u'none', u'zlib@openssh.com'] server compress:[u'none', u'zlib@openssh.com'] client lang:[u''] server lang:[u''] kex follows?False
DEBUG:paramiko.transport:Kex agreed: ecdh-sha2-nistp256
DEBUG:paramiko.transport:HostKey agreed: ssh-ed25519
DEBUG:paramiko.transport:Cipher agreed: aes128-ctr
DEBUG:paramiko.transport:MAC agreed: hmac-sha2-256
DEBUG:paramiko.transport:Compression agreed: none
DEBUG:paramiko.transport:EOF in transport thread

[toghraee]

Normally after Kex, Cipher , MAC and compression agreement in the log you should see a User authentication (userauth) log. (refer to my logs in this thread) .

Also I noticed that Paramiko is not throwing any exception.

I suggest to trace the parameters sent to paramiko , I think there is something missing, i.e the user authentication paramaters not passed.

Another simple test to isolate the problem will be to use the your program and try to connect to a standard Linux host, check if SSH connection can get established or not.

Best Regards

Reza Toghraee

From: princedhadwal [mailto:notifications@github.com] Sent: 19 March 2018 11:13 To: ktbyers/netmiko netmiko@noreply.github.com Cc: Reza Toghraee reza@toghraee.com; Mention mention@noreply.github.com Subject: Re: [ktbyers/netmiko] Cisco 3560 SSH problem "paramiko.ssh_exception.SSHException: No existing session" (#467)

Below are the paramiko logs

DEBUG:my_log:This message should go to the log file DEBUG:my_log:This message should go to the log file DEBUG:my_log:This message should go to the log file DEBUG:paramiko.transport:starting thread (client mode): 0x30ab250L DEBUG:paramiko.transport:Local version/idstring: SSH-2.0-paramiko_2.3.1 DEBUG:paramiko.transport:Remote version/idstring: SSH-2.0-OpenSSH_6.7p1 Debian-5+deb8u1 INFO:paramiko.transport:Connected (version 2.0, client OpenSSH_6.7p1) DEBUG:paramiko.transport:kex algos:[u'curve25519-sha256@libssh.org', u'ecdh-sha2-nistp256', u'ecdh-sha2-nistp384', u'ecdh-sha2-nistp521', u'diffie-hellman-group-exchange-sha256', u'diffie-hellman-group14-sha1'] server key:[u'ssh-rsa', u'ssh-dss', u'ecdsa-sha2-nistp256', u'ssh-ed25519'] client encrypt:[u'aes128-ctr', u'aes192-ctr', u'aes256-ctr', u'aes128-gcm@openssh.com', u'aes256-gcm@openssh.com', u'chacha20-poly1305@openssh.com'] server encrypt:[u'aes128-ctr', u'aes192-ctr', u'aes256-ctr', u'aes128-gcm@openssh.com', u'aes256-gcm@openssh.com', u'chacha20-poly1305@openssh.com'] client mac:[u'umac-64-etm@openssh.com', u'umac-128-etm@openssh.com', u'hmac-sha2-256-etm@openssh.com', u'hmac-sha2-512-etm@openssh.com', u'hmac-sha1-etm@openssh.com', u'umac-64@openssh.com', u'umac-128@openssh.com', u'hmac-sha2-256', u'hmac-sha2-512', u'hmac-sha1'] server mac:[u'umac-64-etm@openssh.com', u'umac-128-etm@openssh.com', u'hmac-sha2-256-etm@openssh.com', u'hmac-sha2-512-etm@openssh.com', u'hmac-sha1-etm@openssh.com', u'umac-64@openssh.com', u'umac-128@openssh.com', u'hmac-sha2-256', u'hmac-sha2-512', u'hmac-sha1'] client compress:[u'none', u'zlib@openssh.com'] server compress:[u'none', u'zlib@openssh.com'] client lang:[u''] server lang:[u''] kex follows?False DEBUG:paramiko.transport:Kex agreed: ecdh-sha2-nistp256 DEBUG:paramiko.transport:HostKey agreed: ssh-ed25519 DEBUG:paramiko.transport:Cipher agreed: aes128-ctr DEBUG:paramiko.transport:MAC agreed: hmac-sha2-256 DEBUG:paramiko.transport:Compression agreed: none DEBUG:paramiko.transport:EOF in transport thread

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/ktbyers/netmiko/issues/467#issuecomment-374177599 , or mute the thread https://github.com/notifications/unsubscribe-auth/ACPIp3Hqpxioh9JR_c7UTfjGBxW5Ec6qks5tf5LHgaJpZM4NYUr1 .