search

netfishers-onl / Netshot

Network Configuration and Compliance Management
http://www.netfishers.onl/netshot
242 stars 57 forks source link

Unable to backup Cisco NCS-540 (Cisco IOS XR) #315

Open michaelarnauts opened 3 weeks ago

michaelarnauts commented 3 weeks ago

This is the same issue as https://github.com/netfishers-onl/Netshot/issues/250

Hopefully, I can provide more information to troubleshoot this.

Device information: Cisco NCS-540 Cisco IOS XR Software, Version 7.9.2

Telnet used to work fine, but we've disabled telnet access, and SSH doesn't seem to work here.

Job log output from the GUI is:

[INFO] Snapshot task for device bru4-pe1 (xxx.xxx.xxx.xxx).
[INFO] Trying SSH to xxx.xxx.xxx.xxx:0 using credentials DEVICESPECIFIC-a5a04b57-fb45-47c4-91ce-500b9cabf781.
[WARN] Unable to open an SSH socket to xxx.xxx.xxx.xxx:0: Session.connect: java.net.SocketTimeoutException: Read timed out
[INFO] Auto-trying Telnet with credentials TELNET | configbackup.
[WARN] Unable to open a Telnet socket to xxx.xxx.xxx.xxx:0.
[ERROR] Error while taking the snapshot: Couldn't open either SSH or Telnet socket with the device.

Debug log is empty (0 bytes).

Netshot log output is:

2024-08-22 14:11:25,923 WARN  [NetshotRunnerScheduler_Worker-63] TaskJob: Running the task 23444975 of type onl.netfishers.netshot.work.tasks.TakeSnapshotTask
2024-08-22 14:11:25,927 INFO  [NetshotRunnerScheduler_Worker-63] Ssh$JschLogger: Connecting to xxx.xxx.xxx.xxx port 22
2024-08-22 14:11:25,929 INFO  [NetshotRunnerScheduler_Worker-63] Ssh$JschLogger: Connection established
2024-08-22 14:11:26,152 INFO  [NetshotRunnerScheduler_Worker-63] Ssh$JschLogger: Remote version string: SSH-2.0-Cisco-2.0
2024-08-22 14:11:26,152 INFO  [NetshotRunnerScheduler_Worker-63] Ssh$JschLogger: Local version string: SSH-2.0-JSCH_0.2.16
2024-08-22 14:11:26,152 INFO  [NetshotRunnerScheduler_Worker-63] Ssh$JschLogger: CheckCiphers: chacha20-poly1305@openssh.com,aes256-gcm@openssh.com,aes128-gcm@openssh.com,aes256-ctr,aes192-ctr,aes128-ctr,aes256-cbc,aes192-cbc,aes128-cbc,3des-ctr,arcfour,arcfour128,arcfour256
2024-08-22 14:11:26,153 INFO  [NetshotRunnerScheduler_Worker-63] Ssh$JschLogger: CheckMacs: hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha2-256,hmac-sha2-512
2024-08-22 14:11:26,153 INFO  [NetshotRunnerScheduler_Worker-63] Ssh$JschLogger: CheckKexes: diffie-hellman-group14-sha1,diffie-hellman-group14-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521
2024-08-22 14:11:26,263 INFO  [NetshotRunnerScheduler_Worker-63] Ssh$JschLogger: CheckSignatures: rsa-sha2-256,rsa-sha2-512,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,ssh-ed25519,ssh-ed448
2024-08-22 14:11:26,263 DEBUG [NetshotRunnerScheduler_Worker-63] Ssh$JschLogger: server_host_key proposal before known_host reordering is: rsa-sha2-256,rsa-sha2-512,ssh-rsa,ssh-dss,ssh-ed25519
2024-08-22 14:11:26,263 DEBUG [NetshotRunnerScheduler_Worker-63] Ssh$JschLogger: server_host_key proposal after known_host reordering is: rsa-sha2-256,rsa-sha2-512,ssh-rsa,ssh-dss,ssh-ed25519
2024-08-22 14:11:26,263 INFO  [NetshotRunnerScheduler_Worker-63] Ssh$JschLogger: SSH_MSG_KEXINIT sent
2024-08-22 14:11:26,263 INFO  [NetshotRunnerScheduler_Worker-63] Ssh$JschLogger: SSH_MSG_KEXINIT received
2024-08-22 14:11:26,263 INFO  [NetshotRunnerScheduler_Worker-63] Ssh$JschLogger: server proposal: KEX algorithms: ecdh-sha2-nistp521,ecdh-sha2-nistp384,ecdh-sha2-nistp256,diffie-hellman-group14-sha1,curve25519-sha256,diffie-hellman-group14-sha256,diffie-hellman-group16-sha512,curve25519-sha256@libssh.org
2024-08-22 14:11:26,263 INFO  [NetshotRunnerScheduler_Worker-63] Ssh$JschLogger: server proposal: host key algorithms: ecdsa-sha2-nistp521,ecdsa-sha2-nistp384,ecdsa-sha2-nistp256,ssh-dss,rsa-sha2-512,rsa-sha2-256,ssh-rsa
2024-08-22 14:11:26,263 INFO  [NetshotRunnerScheduler_Worker-63] Ssh$JschLogger: server proposal: ciphers c2s: aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com,chacha20-poly1305@openssh.com
2024-08-22 14:11:26,263 INFO  [NetshotRunnerScheduler_Worker-63] Ssh$JschLogger: server proposal: ciphers s2c: aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com,chacha20-poly1305@openssh.com
2024-08-22 14:11:26,263 INFO  [NetshotRunnerScheduler_Worker-63] Ssh$JschLogger: server proposal: MACs c2s: hmac-sha2-512,hmac-sha2-256,hmac-sha1
2024-08-22 14:11:26,263 INFO  [NetshotRunnerScheduler_Worker-63] Ssh$JschLogger: server proposal: MACs s2c: hmac-sha2-512,hmac-sha2-256,hmac-sha1
2024-08-22 14:11:26,263 INFO  [NetshotRunnerScheduler_Worker-63] Ssh$JschLogger: server proposal: compression c2s: none
2024-08-22 14:11:26,263 INFO  [NetshotRunnerScheduler_Worker-63] Ssh$JschLogger: server proposal: compression s2c: none
2024-08-22 14:11:26,263 INFO  [NetshotRunnerScheduler_Worker-63] Ssh$JschLogger: server proposal: languages c2s:
2024-08-22 14:11:26,263 INFO  [NetshotRunnerScheduler_Worker-63] Ssh$JschLogger: server proposal: languages s2c:
2024-08-22 14:11:26,263 INFO  [NetshotRunnerScheduler_Worker-63] Ssh$JschLogger: client proposal: KEX algorithms: diffie-hellman-group18-sha512,diffie-hellman-group16-sha512,diffie-hellman-group14-sha256,diffie-hellman-group14-sha1,diffie-hellman-group-exchange-sha256,diffie-hellman-group-exchange-sha1,diffie-hellman-group1-sha1,ext-info-c,kex-strict-c-v00@openssh.com
2024-08-22 14:11:26,263 INFO  [NetshotRunnerScheduler_Worker-63] Ssh$JschLogger: client proposal: host key algorithms: rsa-sha2-256,rsa-sha2-512,ssh-rsa,ssh-dss,ssh-ed25519
2024-08-22 14:11:26,263 INFO  [NetshotRunnerScheduler_Worker-63] Ssh$JschLogger: client proposal: ciphers c2s: aes128-gcm@openssh.com,aes128-ctr,aes128-cbc,3des-ctr,3des-cbc,blowfish-cbc,aes192-ctr,aes192-cbc,aes256-gcm@openssh.com,aes256-ctr,aes256-cbc
2024-08-22 14:11:26,263 INFO  [NetshotRunnerScheduler_Worker-63] Ssh$JschLogger: client proposal: ciphers s2c: aes128-gcm@openssh.com,aes128-ctr,aes128-cbc,3des-ctr,3des-cbc,blowfish-cbc,aes192-ctr,aes192-cbc,aes256-gcm@openssh.com,aes256-ctr,aes256-cbc
2024-08-22 14:11:26,263 INFO  [NetshotRunnerScheduler_Worker-63] Ssh$JschLogger: client proposal: MACs c2s: hmac-md5-etm@openssh.com,hmac-sha1-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-96-etm@openssh.com,hmac-md5-96-etm@openssh.com,hmac-md5,hmac-sha1,hmac-sha2-256,hmac-sha2-512,hmac-sha1-96,hmac-md5-96
2024-08-22 14:11:26,263 INFO  [NetshotRunnerScheduler_Worker-63] Ssh$JschLogger: client proposal: MACs s2c: hmac-md5-etm@openssh.com,hmac-sha1-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-96-etm@openssh.com,hmac-md5-96-etm@openssh.com,hmac-md5,hmac-sha1,hmac-sha2-256,hmac-sha2-512,hmac-sha1-96,hmac-md5-96
2024-08-22 14:11:26,263 INFO  [NetshotRunnerScheduler_Worker-63] Ssh$JschLogger: client proposal: compression c2s: none
2024-08-22 14:11:26,263 INFO  [NetshotRunnerScheduler_Worker-63] Ssh$JschLogger: client proposal: compression s2c: none
2024-08-22 14:11:26,263 INFO  [NetshotRunnerScheduler_Worker-63] Ssh$JschLogger: client proposal: languages c2s:
2024-08-22 14:11:26,263 INFO  [NetshotRunnerScheduler_Worker-63] Ssh$JschLogger: client proposal: languages s2c:
2024-08-22 14:11:26,263 INFO  [NetshotRunnerScheduler_Worker-63] Ssh$JschLogger: kex: algorithm: diffie-hellman-group16-sha512
2024-08-22 14:11:26,263 INFO  [NetshotRunnerScheduler_Worker-63] Ssh$JschLogger: kex: host key algorithm: rsa-sha2-256
2024-08-22 14:11:26,263 INFO  [NetshotRunnerScheduler_Worker-63] Ssh$JschLogger: kex: server->client cipher: aes128-gcm@openssh.com MAC: <implicit> compression: none
2024-08-22 14:11:26,263 INFO  [NetshotRunnerScheduler_Worker-63] Ssh$JschLogger: kex: client->server cipher: aes128-gcm@openssh.com MAC: <implicit> compression: none
2024-08-22 14:11:26,274 INFO  [NetshotRunnerScheduler_Worker-63] Ssh$JschLogger: SSH_MSG_KEXDH_INIT sent
2024-08-22 14:11:26,274 INFO  [NetshotRunnerScheduler_Worker-63] Ssh$JschLogger: expecting SSH_MSG_KEXDH_REPLY
2024-08-22 14:11:36,282 INFO  [NetshotRunnerScheduler_Worker-63] Ssh$JschLogger: Disconnecting from xxx.xxx.xxx.xxx port 22
2024-08-22 14:11:36,282 WARN  [NetshotRunnerScheduler_Worker-63] CliScript: Unable to open an SSH connection to xxx.xxx.xxx.xxx:0.
2024-08-22 14:11:36,289 WARN  [NetshotRunnerScheduler_Worker-63] CliScript: Unable to open a Telnet connection to xxx.xxx.xxx.xxx:0.
2024-08-22 14:11:36,290 ERROR [NetshotRunnerScheduler_Worker-63] TakeSnapshotTask: Task 23444975. Error while taking the snapshot.
2024-08-22 14:11:36,296 WARN  [NetshotRunnerScheduler_Worker-63] TaskJob: End of task 23444975.
2024-08-22 14:11:37,302 WARN  [NetshotRunnerScheduler_Worker-63] TaskJob: Running the task 23444980 of type onl.netfishers.netshot.work.tasks.RunDiagnosticsTask
michaelarnauts commented 3 weeks ago

It seems older versions of IOS XR don't have this issue:

Ofcourse, in a terminal, just ssh'ing works fine.

SCadilhac commented 3 weeks ago

Hello, can you try to increase the SSH connection timeout in Netshot config?

e.g.

netshot.cli.ssh.connectiontimeout = 30000

It seems that with newer KEX algorithms, NCS540 take quite some time to generate their crypto material.

SIRANYAN commented 2 weeks ago

Hello

I have the same issue (ssh fail connection) with :

Device : Cisco C9200 L-48P-4X version 17.6.7 I have installed netshot version 0.19.4 on Rocky Linux 9.4

Can someone help please? (sorry for my english)

Thank you