ktbyers / netmiko

Multi-vendor library to simplify Paramiko SSH connections to network devices
MIT License
3.61k stars 1.31k forks source link

HP Comware - Display arp - no output #3287

Open netwolk opened 1 year ago

netwolk commented 1 year ago

Hi

I'm trying to do a basic thing: Retreiving the ARP table from a comware 7 switch:

The issue i have is that when running the script, the command get executed, but the output is empty. i have no return value if i check my DEBUG Logs, authentication is fine, the command is performed, but the output is not parsed ?

THx for your help

` from netmiko import ConnectHandler from netmiko import NetMikoTimeoutException, NetMikoAuthenticationException

import logging

Setup logging

LOG_FILENAME = r"C:\Users\Logs\TEST_HP-COMMWARE_Arp_Lookup_log_V2.txt" logging.basicConfig(filename=LOG_FILENAME, level=logging.DEBUG) logger = logging.getLogger("netmiko")

def get_arp_table(ip, username, password):

Define the device type and credentials

hp_comware_device = {
    'device_type': 'hp_comware',
    'ip': ip,
    'username': username,
    'password': password,
    'port': 22,  # Default SSH port
    'timeout': 60,  # Setting a longer timeout in seconds
}

connection = None

# Connect to the device
try:
    connection = ConnectHandler(**hp_comware_device)

    # Identify the device's prompt (e.g., L3S-BE-B45-1>)
    prompt = connection.find_prompt()
    print(prompt)
    # Execute the command to retrieve the ARP table
    output = connection.send_command('display arp', expect_string=prompt, delay_factor=2, read_timeout=100)  # delay_factor here
    print(output)
     # Cleanup: Remove the command and trailing prompt from the output
    output = output.replace('display arp', '').replace(prompt, '').strip()
    print(output)

    # Close the connection
    connection.disconnect()

    return output

except NetMikoTimeoutException:
    print("Connection to the device timed out.")
    return None
except NetMikoAuthenticationException:
    print("Authentication failed. Please check your credentials.")
    return None
except Exception as e:
    print(f"An unexpected error occurred: {e}")
    return None
#finally:
    if connection:
        connection.disconnect()

def write_to_file(filename, content): with open(filename, 'w') as file: file.write(content)

Example usage

ip_address = "10.75.130.1" username = "My_Username" password = "My_Pass" output_file_path = r"C:\Users_ArpLookup.txt"

arp_table = get_arp_table(ip_address, username, password) print(arp_table)

Writing the arp_table to a file

if arp_table: print(arp_table) write_to_file(output_file_path, arp_table) else: print("Failed to retrieve ARP table.") `

when i run the script i get:

` arp_Lookup/Test_HP-COMWARE_Get_ARP_V2.py"

Failed to retrieve ARP table. PS C:\Users\> ` the arp_table variable stays empty. If i check the Debug LOG The command was processed, but i don't get any output > DEBUG:paramiko.transport:starting thread (client mode): 0x8cb78550 DEBUG:paramiko.transport:Local version/idstring: SSH-2.0-paramiko_3.3.1 DEBUG:paramiko.transport:Remote version/idstring: SSH-2.0-Comware-5.20.99 INFO:paramiko.transport:Connected (version 2.0, client Comware-5.20.99) DEBUG:paramiko.transport:=== Key exchange possibilities === DEBUG:paramiko.transport:kex algos: diffie-hellman-group-exchange-sha1, diffie-hellman-group14-sha1, diffie-hellman-group1-sha1 DEBUG:paramiko.transport:server key: ssh-rsa DEBUG:paramiko.transport:client encrypt: aes128-cbc, 3des-cbc, des-cbc DEBUG:paramiko.transport:server encrypt: aes128-cbc, 3des-cbc, des-cbc DEBUG:paramiko.transport:client mac: hmac-sha1, hmac-sha1-96, hmac-md5, hmac-md5-96 DEBUG:paramiko.transport:server mac: hmac-sha1, hmac-sha1-96, hmac-md5, hmac-md5-96 DEBUG:paramiko.transport:client compress: none DEBUG:paramiko.transport:server compress: none DEBUG:paramiko.transport:client lang: DEBUG:paramiko.transport:server lang: DEBUG:paramiko.transport:kex follows: False DEBUG:paramiko.transport:=== Key exchange agreements === DEBUG:paramiko.transport:Kex: diffie-hellman-group-exchange-sha1 DEBUG:paramiko.transport:HostKey: ssh-rsa DEBUG:paramiko.transport:Cipher: aes128-cbc DEBUG:paramiko.transport:MAC: hmac-sha1 DEBUG:paramiko.transport:Compression: none DEBUG:paramiko.transport:=== End of kex handshake === DEBUG:paramiko.transport:Got server p (2048 bits) DEBUG:paramiko.transport:kex engine KexGex specified hash_algo DEBUG:paramiko.transport:Switch to new keys ... DEBUG:paramiko.transport:Adding ssh-rsa host key for 10.75.130.1: b'2399d465540bdf4d09762ddc90bae12e' DEBUG:paramiko.transport:userauth is OK INFO:paramiko.transport:Authentication (password) successful! DEBUG:paramiko.transport:[chan 0] Max packet in: 32768 bytes DEBUG:paramiko.transport:[chan 0] Max packet out: 32496 bytes DEBUG:paramiko.transport:Secsh channel 0 opened. DEBUG:paramiko.transport:[chan 0] Sesch channel 0 request ok DEBUG:paramiko.transport:[chan 0] Sesch channel 0 request ok DEBUG:netmiko:write_channel: b'\n' DEBUG:netmiko:read_channel: ****************************************************************************** * Copyright (c) 2010-2015 Hewlett-Packard Development Company, L.P. * * Without the owner's prior written consent, * * no decompiling or reverse-engineering shall be allowed. * ****************************************************************************** DEBUG:netmiko:read_channel: DEBUG:netmiko:Pattern found: (to continue|[>\]]) ****************************************************************************** * Copyright (c) 2010-2015 Hewlett-Packard Development Company, L.P. * * Without the owner's prior written consent, * * no decompiling or reverse-engineering shall be allowed. * ****************************************************************************** DEBUG:netmiko:read_channel: DEBUG:netmiko:write_channel: b'\n' DEBUG:netmiko:read_channel: DEBUG:netmiko:read_channel: DEBUG:netmiko:read_channel: DEBUG:netmiko: Parenthesis found in pattern. pattern: (>|\]) This can be problemtic when used in read_until_pattern(). You should ensure that you use either non-capture groups i.e. '(?:' or that the parenthesis completely wrap the pattern '(pattern)' DEBUG:netmiko:Pattern found: (>|\]) DEBUG:netmiko:read_channel: DEBUG:netmiko:[find_prompt()]: prompt is DEBUG:netmiko:In disable_paging DEBUG:netmiko:Command: screen-length disable DEBUG:netmiko:write_channel: b'\nscreen-length disable\n' DEBUG:netmiko:read_channel: DEBUG:netmiko:read_channel: DEBUG:netmiko:read_channel: DEBUG:netmiko:read_channel: DEBUG:netmiko:Pattern found: (L3S\-BE\-B45\-1) screen-length disa DEBUG:netmiko:Clear buffer detects data in the channel DEBUG:netmiko:read_channel: ble DEBUG:netmiko:Clear buffer detects data in the channel DEBUG:netmiko:read_channel: DEBUG:netmiko:write_channel: b'\n' DEBUG:netmiko:read_channel: % Screen-length configuration is disabled for current user. DEBUG:netmiko:read_channel: DEBUG:netmiko:[find_prompt()]: prompt is DEBUG:netmiko:write_channel: b'display arp\n' DEBUG:netmiko:read_channel: DEBUG:netmiko:write_channel: b'\n' DEBUG:netmiko:read_channel: display arp Type: S-Static D-Dynamic O-Openflow M-Multiport IP Address MAC Address VLAN ID Interface Aging Type 10.75.131.254 1c3a-6002-5d90 154 BAGG110 20 D 10.76.142.100 009f-1563-3161 500 BAGG111 17 D 10.76.142.39 009f-0763-3661 500 BAGG111 18 D 10.76.142.102 009f-1970-2606 500 BAGG111 17 D 10.76.142.71 009f-6661-3a66 500 BAGG111 17 D DEBUG:netmiko:read_channel: 10.76.142.14 009f-6661-3866 500 BAGG111 19 D 10.76.142.122 009f-0163-5561 500 BAGG111 18 D 10.76.142.49 009f-1461-3c66 500 BAGG113 16 D 10.76.142.117 009f-6661-2f66 500 BAGG113 19 D DEBUG:netmiko:read_channel: DEBUG:netmiko:read_channel: 10.76.142.23 009f-6e61-6066 500 BAGG113 16 D 10.76.142.25 009f-1563-3761 500 BAGG113 18 D 10.76.142.47 009f-6363-5161 500 BAGG113 18 D DEBUG:netmiko:read_channel: DEBUG:netmiko:read_channel: DEBUG:netmiko:write_channel: b'quit\n' DEBUG:paramiko.transport:EOF in transport thread THX
ktbyers commented 1 year ago

The debug posted above doesn't complete (i.e. I don't see the trailing router prompt in the output)?

I see this in the output:

DEBUG:netmiko:read_channel:
% Screen-length configuration is disabled for current user.

You probably need to execute screen-length disable (which Netmiko tries to do automatically).

You might want to look at the Netmiko session_log and see what you see there.

# Where hp_comware_device contains (will create a file named "output.txt" in the script directory).
hp_comware_device["session_log"] = "output.txt"
connection = ConnectHandler(**hp_comware_device)
netwolk commented 1 year ago

hi,

I adjusted the script to write the session log. it gives me the output in the log as expected, And the screen-length disable is sent. `


screen-length disable % Screen-length configuration is disabled for current user. display arp Type: S-Static D-Dynamic O-Openflow M-Multiport IP Address MAC Address VLAN ID Interface Aging Type D 10.75.130.130 9457-a522-e1a0 154 GE1/0/1 16 D 10.75.131.79 1cb9-c431-7dd0 154 GE1/0/3 20 D 10.75.131.65 38ff-3636-4160 154 GE1/0/3 20 D 10.75.138.48 0006-7300-c9db 121 GE1/0/5 10 D 10.75.138.47 0006-7300-b85d 121 GE1/0/5 19 D 10.75.130.237 480f-cf7e-7140 154 GE1/0/5 7 D 10.75.130.135 a01d-4839-6820 154 GE1/0/5 3 D 10.75.148.49 0017-c8a6-faf9 161 GE1/0/6 20 D 10.75.131.69 2cc5-d305-9ed0 154 GE1/0/6 20 D 10.75.131.68 2cc5-d305-a5d0 154 GE1/0/6 20 D 10.75.130.32 0040-9d80-62de 154 GE1/0/6 10 D 10.75.148.254 0661-8042-30b8 161 GE1/0/6 18 D 10.75.148.32 0007-4dcf-c01f 161 GE1/0/6 3 D 10.75.148.18 0080-9189-1501 161 GE1/0/6 17 D 10.75.130.136 9457-a51c-4c00 154 GE1/0/6 3 D 10.75.130.137 b05a-da0a-5ae0 154 GE1/0/7 4 D 10.75.130.36 0040-9d8c-7d28 154 GE1/0/7 9 D 10.75.145.249 045f-b90f-7501 171 GE1/0/7 17 D 10.75.145.252 006c-bca9-1dff 171 GE1/0/7 3 D 10.75.138.40 0006-7300-e1cf 121 GE1/0/7 10 D 10.75.130.35 0040-9d80-63d1 154 GE1/0/7 17 D 10.75.131.73 743e-2b0c-f5a0 154 GE1/0/8 20 D 10.75.131.86 3087-d931-43c0 154 GE1/0/8 20 D .. Not the full output here quit `
netwolk commented 1 year ago

when i run this code:

from netmiko import ConnectHandler
from datetime import datetime
import time

device = {
    'device_type':'hp_comware',
    'ip':"10.75.130.1",
    'username':"Username",
    'password':"Pass",
}

start_time = datetime.now()
print(start_time)

net_connect = ConnectHandler(**device, session_log=r"C:\Users\dellaerb\OneDrive - Katoen Natie\Documents\IT\Scripts\Arp_Lookup\Logs\Comware_Driver_TEST_LOG.txt")
print("connected to device")

# Disable screen-length and set the expected string as the device prompt
net_connect.send_command("screen-length disable")
print("screen-length disabled")

# Send the 'display version' command but don't immediately capture its output
net_connect.write_channel("display arp\n")

# Wait for a few seconds for the device to return the output
time.sleep(5)

# Read the output
output = net_connect.read_channel()

name = net_connect.find_prompt() + device['ip']
net_connect.disconnect()

print(name)
print(output)
print(">>>>>>>>>>>>>>>>>>>>>End<<<<<<<<<<<<<<<<<<<<<<")

end_time = datetime.now()
print(end_time)

i get my ouput as expected, The log file is the same:

screen-length disable % Screen-length configuration is disabled for current user. screen-length disable % Screen-length configuration is disabled for current user. display arp Type: S-Static D-Dynamic O-Openflow M-Multiport IP Address MAC Address VLAN ID Interface Aging Type 10.75.131.254 1c3a-6002-5d90 154 BAGG110 20 D 10.75.130.143 104f-588c-d300 154 BAGG110 5 D 10.76.143.60 a4bb-6dcb-e077 500 BAGG110 20 D 10.76.143.62 a4bb-6dba-fe1e 500 BAGG110 20 D 10.76.143.1 0006-7723-ca16 500 BAGG110 20 D 10.76.143.63 a4bb-6dcb-7c97 500 BAGG110 20 D 10.76.143.65 a4bb-6dba-9a6b 500 BAGG110 20 D 10.76.143.120 c0b1-3c00-0fa7 500 BAGG110 20 D 10.76.143.123 c0b1-3c00-0f84 500 BAGG110 20 D 10.76.143.122 c0b1-3c00-0f86 500 BAGG110 20 D 10.76.143.125 c0b1-3c00-0f81 500 BAGG110 20 D .. still some arp entries, but filtered out ... quit
ktbyers commented 1 year ago

Maybe try adding:

global_cmd_verify=True

As an argument to ConnectHandler (or alternatively add cmd_verify=True to your send_command call). HP Comware has no way of setting the terminal width (as far as I know) and this causes a lot of problems.

If you know of a way to do this on Comware, let me know.

Let me know if the above works.

jiujing commented 1 year ago

Maybe try adding:

global_cmd_verify=True

As an argument to ConnectHandler (or alternatively add cmd_verify=True to your send_command call). HP Comware has no way of setting the terminal width (as far as I know) and this causes a lot of problems.

If you know of a way to do this on Comware, let me know.

Let me know if the above works.

I think it is necessary to remove the extra newline character ("\n") because users may not be familiar with the mechanism of cmd_verify. By using this parameter, users can avoid the issue, but they need to be aware of this mechanism. Netmiko aims to simplify user operations, so removing the newline character ("\n") allows users to avoid incorrect echoing caused by the extra newline character in the "cancel paging" command, regardless of whether the cmd_verify mechanism is enabled.

jiujing commented 1 year ago

You can try setting global_cmd_verify to True when creating the connection in the ConnectHandler function. This can help avoid the bug where the echoing is misaligned with the command. ref link:https://github.com/ktbyers/netmiko/pull/3141

ktbyers commented 1 year ago

@jiujing I doubt the above problem is related to #3141.

I think it is related to global_cmd_verify=False on Comware (due to Comware lacking any way of setting the terminal width).

But I will merge #3141 in and the user can test using the develop branch and see if it helps or not.

jiujing commented 1 year ago

global_cmd_verify

This issue is indeed related to global_cmd_verify, and setting it to False can help avoid this problem.However, the root cause lies in the extra newline character ("\n") before the "cancel paging" command. The "\n" in that piece of code doesn't serve any purpose. You should simply add a carriage return after each command you send, without adding a newline character before the command. Additionally, global_cmd_verify may affect send_config_set. If global_cmd_verify is set to True and cmd_verify is set to False, Netmiko will keep writing data to the channel without reading data. When a large configuration is being pushed, this continuous sending of data may cause the remote network device to close the channel.

ktbyers commented 1 year ago

@jiujing

Yeah, it is a bit hard to tell with the above output, but I don't think the extra caused the issue reported here (looking at the log file output). But #3141 is merged regardless so that has been changed.

Additionally, global_cmd_verify may affect send_config_set. If global_cmd_verify is set to True and cmd_verify 
is set to False, Netmiko will keep writing data to the channel without reading data. When a large configuration 
is being pushed, this continuous sending of data may cause the remote network device to close the channel.

Yes, this is known (i.e. that is what cmd_verify does and why it was implemented in general), but the Comware (as far as I am aware) does not have a way to set the terminal width. Consequently, cmd_verify will break on Comware for commands that exceed a certain length (this is why Comware is one of the very few Netmiko drivers where cmd_verify is disabled by default).

So we either have Comware break for long commands or we have Comware break due to writing commands too fast (or sending too many config commands).

I am somewhat open to switching the default for global_cmd_verify back on Comware, but it will cause the driver to break for other users.

We could add in more logic in the initial session_preparation where we cmd_verify that the output paging disable works properly, before moving on...so that at least this part works reliably.

netwolk commented 1 year ago

thx for al the feedback.

the difference i have found:

on Comware v7 its working with no issues. the issues only occurs on Comware v5

netwolk commented 10 months ago

To Come back to this issue, about the difference i was working on a script to get the LLDP neighbor information.

when using write channel and read channel then i get my output as expected below working code:

`try:

Establishing a connection to the device

    print("start connecting")
    net_connect = ConnectHandler(**device)
    print(f"Connected to {device['ip']} ")

    # Disable screen-length and set the expected string as the device prompt
    net_connect.send_command("screen-length disable")
    print("screen-length disabled")

    # Get hostname
    hostname = get_hostname(net_connect)

    # Send the 'display lldp neighbor-information' command but don't immediately capture its output
    net_connect.write_channel("display lldp neighbor-information\n")

    # Wait for a few seconds for the device to return the output
    time.sleep(5)

    # Read the output
    output = net_connect.read_channel()

    net_connect.disconnect()

    return output,hostname`

when using the send command, i don't get any returned output: below "non" woking code. the ouput is just blank, but i do get a hostname of the device !

` try:

Establishing a connection to the device using Netmiko's ConnectHandler.

    print("start connecting")
    connection = ConnectHandler(**device)
    print(f"Connected to {device['ip']} ")

    # Get Hostname
    hostname = get_hostname(connection)

    # Sending a command to the device to retrieve LLDP neighbor information.
    output = connection.send_command("display lldp neighbor-information")

    # Closing the connection to the device.
    connection.disconnect()

    # Returning the output from the device.
    return output, hostname`

attached the both debugs

LOG_LLDP_NeighborInformation_ComwareV5_LLDP-Output OK.txt LOG_LLDP_NeighborInformation_ComwareV5_LLDP-Output_NOK.txt