Closed mohitbhardwaj5659 closed 2 years ago
In Netmiko 3.3.2 for NX-OS, you need to set both:
fast_cli = False
global_delay_factor = X # where X is your multiplier
Regards,
Kirk
Hi Kirk,
I have tried with the above recommendation but is still throwing this errror. install all nxos bootflash:///nxos.7.0.3.I5.2.bin --- Logging error --- Traceback (most recent call last): File "/home/mohit/NORNIR_POC/nornir-network-automation/upgrade_with_netmiko/nexus_upgrade_final.py", line 278, in install_and_reload installing_and_reloading=net_connect.send_command(install_reload) File "/usr/local/lib/python3.6/site-packages/netmiko/utilities.py", line 429, in wrapper_decorator return func(self, *args, **kwargs) File "/usr/local/lib/python3.6/site-packages/netmiko/base_connection.py", line 1525, in send_command search_pattern.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib64/python3.6/logging/init.py", line 994, in emit
msg = self.format(record)
File "/usr/lib64/python3.6/logging/init.py", line 840, in format
return fmt.format(record)
File "/usr/lib64/python3.6/logging/init.py", line 577, in format
record.message = record.getMessage()
File "/usr/lib64/python3.6/logging/init.py", line 338, in getMessage
msg = msg % self.args
TypeError: not all arguments converted during string formatting
Call stack:
File "calling_nexus_upgrade.py", line 109, in
I have added that in the connecthandler and it is still throwing me the error.
def Connect(self):
device_list=[]
connection =''
for i in range(0,len(self.IP_ADDRESS)):
connection="connection_{}".format(i)
#print(connection)
connection= {
"device_type": self.device_type,
"host": self.IP_ADDRESS[i],
"username":self.username,
"password":self.password,
"global_delay_factor": 4,
"fast_cli": False
}
device_list.append(connection)
return device_list
What command are you sending in send_comand() and how long are you expecting that command to take (to complete)?
It looks like you re trying to reload the switch, you probably should use send_command_timing() to reload the switch. Also does the reload operation prompt you here? If so, you will need to handle the prompting.
Regards, Kirk
This is the ouput which i have ran manually on nexus 9k.
ODIN-S1# install all nxos bootflash:///nxos.7.0.3.I5.2.bin Installer will perform compatibility check first. Please wait. Installer is forced disruptive
Verifying image bootflash:/nxos.7.0.3.I5.2.bin for boot variable "nxos". [####################] 100% -- SUCCESS
Verifying image type. [####################] 100% -- SUCCESS
Preparing "nxos" version info using image bootflash:/nxos.7.0.3.I5.2.bin. [####################] 100% -- SUCCESS
Preparing "bios" version info using image bootflash:/nxos.7.0.3.I5.2.bin. [####################] 100% -- SUCCESS
Warning: the running-config contains configuration that is incompatible with the new image and will be lost (loose incompatibility).
Please run 'show incompatibility-all nxos
Notifying services about system upgrade. [####################] 100% -- SUCCESS
Compatibility check is done: Module bootable Impact Install-type Reason
1 yes disruptive reset Incompatible image for ISSU
27 yes disruptive reset Incompatible image for ISSU
Images will be upgraded according to following table: Module Image Running-Version(pri:alt) New-Version Upg-Required
1 lcn9k 9.3(4) 7.0(3)I5(2) yes
27 nxos 9.3(4) 7.0(3)I5(2) yes
27 bios v07.67(01/30/2020):v07.34(08/11/2015) v07.59(08/26/2016) no
When I ran the command through the code it is getting stuck here and bypasses rest of the step and throws no error .
install all nxos bootflash:///nxos.7.0.3.I5.2.bin Installer will perform compatibility check first. Please wait. Installer is forced disruptive
Verifying image bootflash:/nxos.7.0.3.I5.2.bin for boot variable "nxos". [# ] 0%
def install_and_reload(self,client,net_connect): ''' reloading the switch with the image. ''' try: logger.info("installing image and reloading the device %s",client['host']) install_reload="install all nxos bootflash:///{}".format(self.image) print(install_reload) installing_and_reloading=net_connect.send_command_timing(install_reload) print(installing_and_reloading) except Exception as error: logger.error("Error in reloading the switch",error) sys.exit(0)
i Will handle the prompt with the expect string parameter but it is not going till that point.
This is the debug ouput which i have found from the test log. Hope that helps in narrowing down the problem.
2021-01-11 11:12:50,372 DEBUG starting thread (client mode): 0x34796128
2021-01-11 11:12:50,373 DEBUG Local version/idstring: SSH-2.0-paramiko_2.7.2
2021-01-11 11:12:50,503 DEBUG Remote version/idstring: SSH-2.0-OpenSSH_7.5 PKIX[10.1]
2021-01-11 11:12:50,504 INFO Connected (version 2.0, client OpenSSH_7.5)
2021-01-11 11:12:50,507 DEBUG Kex agreed: ecdh-sha2-nistp256
2021-01-11 11:12:50,507 DEBUG HostKey agreed: ssh-rsa
2021-01-11 11:12:50,507 DEBUG Cipher agreed: aes128-ctr
2021-01-11 11:12:50,507 DEBUG MAC agreed: hmac-sha2-256
2021-01-11 11:12:50,507 DEBUG Compression agreed: none
2021-01-11 11:12:50,518 DEBUG kex engine KexNistp256 specified hash_algo
2021-01-11 11:13:03,516 DEBUG Pattern found: terminal width 511 terminal width 511^M^M
2021-01-11 11:13:03,517 DEBUG In disable_paging 2021-01-11 11:13:03,517 DEBUG Command: terminal length 0
2021-01-11 11:13:03,517 DEBUG write_channel: b'terminal length 0\n' 2021-01-11 11:13:03,517 DEBUG Pattern is: terminal\ length\ 0 2021-01-11 11:13:03,519 DEBUG _read_channel_expect read_data: terminal length 0^M
2021-01-11 11:13:03,519 DEBUG Pattern found: terminal\ length\ 0 terminal length 0^M
2021-01-11 11:13:03,519 DEBUG terminal length 0^M
2021-01-11 11:13:03,519 DEBUG Exiting disable_paging 2021-01-11 11:13:03,920 DEBUG read_channel: ^MODIN-S1# terminal length 0^M^M ^MODIN-S1# 2021-01-11 11:13:03,920 DEBUG Clear buffer detects data in the channel 2021-01-11 11:13:04,722 DEBUG read_channel: 2021-01-11 11:13:04,722 DEBUG write_channel: b'\n' 2021-01-11 11:13:05,123 DEBUG read_channel: ^M^M ^MODIN-S1# 2021-01-11 11:13:05,925 DEBUG read_channel: 2021-01-11 11:13:05,925 DEBUG [find_prompt()]: prompt is ODIN-S1# 2021-01-11 11:13:05,925 INFO connecting to the device 10.192.103.81 2021-01-11 11:13:05,926 INFO installing image and reloading the device 10.192.103.81 2021-01-11 11:13:05,927 DEBUG write_channel: b'install all nxos bootflash:///nxos.7.0.3.I5.2.bin\n' 2021-01-11 11:13:05,927 DEBUG send_command_timing current output: 2021-01-11 11:13:06,328 DEBUG read_channel: install all nxos bootflash:///nxos.7.0.3.I5.2.bin^M^M Installer will perform compatibility check first. Please wait. ^M Installer is forced disruptive^M
2021-01-11 11:13:06,728 DEBUG read_channel: 2021-01-11 11:13:14,736 DEBUG read_channel: ^M Verifying image bootflash:/nxos.7.0.3.I5.2.bin for boot variable "nxos".^M [# ] 0% 2021-01-11 11:13:15,137 DEBUG read_channel: 2021-01-11 11:13:23,145 DEBUG read_channel: 2021-01-11 11:13:23,145 DEBUG send_command_timing final output: Installer will perform compatibility check first. Please wait. Installer is forced disruptive
Verifying image bootflash:/nxos.7.0.3.I5.2.bin for boot variable "nxos". [# ] 0%
There are two items you need to handle here:
Switch will be reloaded for disruptive upgrade.
Do you want to continue with the installation (y/n)? [n] n
Here is an example of handling prompting using the expect_string
argument.
Regards,
Kirk
I have set the global delay factor to 4 in the connecthandler creation paramter to the install and check hardware compatibility function.
def Connect(self): device_list=[] connection ='' for i in range(0,len(self.IPADDRESS)): connection="connection{}".format(i)
connection= {
"device_type": self.device_type,
"host": self.IP_ADDRESS[i],
"username":self.username,
"password":self.password,
"global_delay_factor": 4,
"fast_cli": False
}
device_list.append(connection)
I am passing the net_connet parameter to the below function with the global_delay_factor set but it is still failing.
def install_and_reload(self,client,net_connect): ''' reloading the switch with the image. ''' try: logger.info("installing image and reloading the device %s",client['host']) install_reload="install all nxos bootflash:///{}".format(self.image) print(install_reload) installing_and_reloading=net_connect.send_command_timing(install_reload) print(installing_and_reloading)
# installing_and_reloading += net_connect.send_command_timing("\n")
# #print(installing_and_reloading)
#else:
# print("################################# FAIL TO INSTALL AND RELOAD##############")
print("#########################CHECKING INSTALL AND RELOAD######################")
except Exception as error:
logger.error("Error in reloading the switch",error)
sys.exit(0)
One thing to note i have also use checking hardware compatibilities function and it is showing me the correct result with the same set of parameter.
def checking_hardware_compatibilities(self,client,net_connect):
'''
checking and writing whether software upgrade is disruptive or non-disruptive.
'''
try:
logger.info("checking hardware_compatibilities for switch %s",client['host'])
install_capabilities="show install all impact nxos bootflash:///"+str(self.image)
print(install_capabilities)
checking_install_output=net_connect.send_command(install_capabilities)
#print(net_connect.find_prompt())
print(checking_install_output)
pattern = re.compile("non-disruptive")
result = pattern.findall('non-disruptive')
if result[0] == "non-disruptive":
logging.info("Switch with ip %s will perform a non disruptive upgrade",client['host'])
else:
logging.info("Switch with ip %s will perform a disruptive upgrade",client['host'])
file_checking_install_output=open("hardware_compatibility_result.txt"
+str(client['host']),"w")
file_checking_install_output.write(checking_install_output)
file_checking_install_output.close()
time.sleep(2)
except Exception as error:
logger.error("not able to perform hardware compatibility check exiting the upgrade process",error)
sys.exit(0)
Hello Kirk,
Thanks for your guidance.
Somehow i have managed to run the install command in which was stuck . now i need to handle the prompt for continuation of the upgrade process.
I have used the send_command_timing function to send the install command that doesnt have "expect_string" parameter in that . But for handling the prompt i am using net_connect.send_command() but it is throwing me the error which i have pasted below
def install_and_reload(self,client,net_connect):
'''
reloading the switch with the image.
'''
try:
logger.info("installing image and reloading the device %s",client['host'])
install_reload="install all nxos bootflash:///"+str(self.image)
print(install_reload)
installing_and_reloading=net_connect.send_command_timing(install_reload)
installing_and_reloading += net_connect.send_command_timing("\n")
splitting = installing_and_reloading.split("\n")
print(splitting)
if splitting [-2] == "Do you want to continue with the installation (y/n)? [n] ":
logger.info("Reloading the device %s",client['host'])
installing_and_reloading += net_connect.send_command("y")
installing_and_reloading += net_connect.send_command("\n")
print(installing_and_reloading)
else:
logger.info("FAIL TO RELOAD")
except Exception as error:
logger.error("Error in reloading the switch",error)
sys.exit(0)
2021-01-12 10:32:06,846 INFO Reloading the device 10.192.103.82 Installer will perform compatibility check first. Please wait. Installer is forced disruptive
Verifying image bootflash:/nxos.7.0.3.I5.2.bin for boot variable "nxos". [# ] 0%[####################] 100% -- SUCCESS
Verifying image type. [# ] 0%[####################] 100% -- SUCCESS
Preparing "nxos" version info using image bootflash:/nxos.7.0.3.I5.2.bin. [# ] 0%[####################] 100% -- SUCCESS
Preparing "bios" version info using image bootflash:/nxos.7.0.3.I5.2.bin. [# ] 0%[####################] 100% -- SUCCESS
Warning: the running-config contains configuration that is incompatible with the new image and will be lost (loose incompatibility).
Please run 'show incompatibility-all nxos
Performing module support checks. [####################] 100% -- SUCCESS
Notifying services about system upgrade. [####################] 100% -- SUCCESS
Compatibility check is done: Module bootable Impact Install-type Reason
1 yes disruptive reset Incompatible image for ISSU
Images will be upgraded according to following table: Module Image Running-Version(pri:alt) New-Version Upg-Required
1 nxos 9.3(4) 7.0(3)I5(2) yes
1 bios v07.59(08/26/2016):v07.34(08/11/2015) v07.59(08/26/2016) no
Switch will be reloaded for disruptive upgrade.
Do you want to continue with the installation (y/n)? [n] ^ % Invalid command at '^' marker.
****Do you want to continue with the installation (y/n)? [n]
Hello Kirk ,
Can you please help me handling the prompt .
I am getting stuck while sending the below command. installing_and_reloading += net_connect.send_command("y")
Error Do you want to continue with the installation (y/n)? [n] n ^ % Invalid command at '^' marker.
Vaibhav
Here are some examples on doing that:
And
https://github.com/ktbyers/netmiko/blob/develop/EXAMPLES.md#handling-commands-that-prompt-timing
Regards, Kirk
Hi Kirk : Some command has dynamic output. I have an issue that I want print the output of running a command, however, I only get the beginning of output. Can you tell me how to get all output? Thank you ahead!
@mohitbhardwaj5659 how did you end up fixing this ?
Hi, even I am passing global_delay_factor as well as fast_cli ..my output is not getting completed. but when I pass read_timout to 10000 in send_command its giving whole output but I am unable to send "y" when the user prompt comes..pls help
@adityamum Please open a new issue and include the relevant section of your code and the issue that you are seeing.
Hi Kirk ,
I am trying to send the below command to the nexus 9k switch Command
install all nxos bootflash:///nxos.7.0.3.I5.2.bin Output
Installer will perform compatibility check first. Please wait. Installer is forced disruptive It stucks here and do not show anything .
I am using the same command manually on the switch and i am getting the result
Installer will perform compatibility check first. Please wait. Installer is forced disruptive
Verifying image bootflash:/nxos.7.0.3.I5.2.bin for boot variable "nxos". [####################] 100% -- SUCCESS
Verifying image type. [####################] 100% -- SUCCESS
Preparing "nxos" version info using image bootflash:/nxos.7.0.3.I5.2.bin. [####################] 100% -- SUCCESS
Preparing "bios" version info using image bootflash:/nxos.7.0.3.I5.2.bin. [####################] 100% -- SUCCESS
I am using the below function to send the command. Please suggest if any parameter need to be provided.
def install_and_reload(self,client,net_connect): ''' reloading the switch with the image. ''' try: logger.info("installing image and reloading the device %s",client['host']) install_reload="install all nxos bootflash:///{}".format(self.image) print(install_reload) installing_and_reloading=net_connect.send_command_timing(install_reload,strip_prompt= False,strip_command = False) print(installing_and_reloading) except Exception as error: logger.error("Error in reloading the switch",error) sys.exit(0)