redfish.managers.test_bmc_network_conf.Add_Valid_IPv4_Addres_And_Verify fails

[generatz]

I ran the test_bmc_network_conf suite on romulus in qemu. Here is that piece of the Jenkins log:

16:30:07 Add Valid IPv4 Address And Verify :: Add IPv4 Address via Redfish ... #(EST) 2022/01/11 16:30:07.503704 -    0.285186 - Issuing: cat /usr/share/ipmi-providers/channel_config.json
16:30:07 #(EST) 2022/01/11 16:30:07.802433 -    0.014417 - Executing: get('/redfish/v1/Managers/bmc/EthernetInterfaces/eth0')
16:30:07 #(EST) 2022/01/11 16:30:07.954341 -    0.151908 - Issuing: cat /usr/share/ipmi-providers/channel_config.json
16:30:07 #(EST) 2022/01/11 16:30:08.353297 -    0.069566 - Issuing: /sbin/ip addr | grep eth0
16:30:08 #(EST) 2022/01/11 16:30:08.786122 -    0.075541 - Issuing: cat /usr/share/ipmi-providers/channel_config.json
16:30:08 #(EST) 2022/01/11 16:30:09.112360 -    0.014521 - Executing: get('/redfish/v1/Managers/bmc/EthernetInterfaces/eth0')
16:30:09 #(EST) 2022/01/11 16:30:09.297589 -    0.185229 - Issuing: cat /usr/share/ipmi-providers/channel_config.json
16:30:09 #(EST) 2022/01/11 16:30:09.612861 -    0.015816 - Executing: patch('/redfish/v1/Managers/bmc/EthernetInterfaces/eth0', body={'IPv4StaticAddresses': [{'Address':...
16:30:09 #(EST) 2022/01/11 16:30:30.866985 -   21.254124 - Issuing: cat /usr/share/ipmi-providers/channel_config.json
16:30:30 #(EST) 2022/01/11 16:33:01.459048 -  150.592063 - Issuing: cat /usr/share/ipmi-providers/channel_config.json
16:33:01 #(EST) 2022/01/11 16:33:21.664796 -   20.205748 - Issuing: ping -c 1 -w 2 127.0.0.1
16:33:21 #(EST) 2022/01/11 16:33:21.679188 -    0.014393 - Issuing: BMC Execute Command  read uptime filler 2>/dev/null < /proc/uptime && [ ! -z "\${uptime}" ] && echo \${uptime}  quiet=1  test_mode=0  time_out=5
16:33:21 #(EST) 2022/01/11 16:33:21.684013 -    0.004824 - Issuing: read uptime filler 2>/dev/null < /proc/uptime && [ ! -z "${uptime}" ] && echo ${uptime}
16:33:21 state:
16:41:05   [redfish]:                                      0
16:41:05   [ping]:                                         1
16:41:05   [uptime]:                                       
16:41:05 #(EST) 2022/01/11 16:41:05.339518 -  338.304042 - **ERROR** redfish commands to the BMC are failing.  Terminating FFDC collection.
16:41:05 TEST_HISTORY:                                     /home/ojenkins/workspace/romulus_redfish_managers/logs//test_history.txt
16:41:05 #(EST) 2022/01/11 16:41:05.372384 -    0.032866 - Executing: delete('/redfish/v1/SessionService/Sessions/cSmxVuE0Nh')
16:41:05 | FAIL |
16:46:46 SSHException: No existing session
16:46:46 
16:46:46 Also teardown failed:
16:46:46 Several failures occurred:
16:46:46 
16:46:46 1) SSHException: Error reading SSH protocol banner
16:46:46 
16:46:46 2) Resolving variable '${active_channel_config['1']['name']}' failed: TypeError: 'NoneType' object is not subscriptable
16:46:46 
16:46:46 3) Resolving variable '${resp.dict}' failed: AttributeError: 'NoneType' object has no attribute 'dict'
16:46:46 
16:46:46 4) RetriesExhaustedError
16:46:46 ------------------------------------------------------------------------------

The BMC is subsequently unreachable. It also looks as if it could cause similar problems with real hardware BMCs.

I'd like to see some added documentation and changes to the tests that address:

• an enumeration of tests in this suite that will not run on QEMU
• if warranted, an enumeration of tests in this suite that will not run outside specified ip address constraints (address ranges, netmask, gateway, etc.)
• if warranted, a warning about running the entire suite unchanged with provided ip addresses Better yet, maybe something like this: Disable hazardous tests by default. The hazardous tests would only be enabled if a suite-specific variable, say "test_bmc_network_conf_enable_all" was changed from "false" to "true". A test is added (maybe "Some Tests Disabled") that only depends on the variable "test_bmc_network_conf_enable_all" to alert users that not all tests are enabled.
• as warranted, guidelines about how to modify the variables so that addressing failures on HW can be precluded
• as warranted, tests could check values against network settings (gateway, netmask, etc.) before making risky changes within the test

[gkeishin]

16:33:21 state:
16:41:05   [redfish]:                                      0

looks like redfish call to the BMC failed.

Having said that... @swe12345 when u have time , please check on it latter.

[gkeishin]

@generatz Sweta is out this week, so we will come back next week on this

[generatz]

I've also run this on HW and confirmed that the BMC is subsequently unreachable using IPv4 addressing. I recovered access by using ipmitool with the IPv6 address and restored the needed ipv4 parameters (this was a static ip setup).

Perhaps the use of IPv6 addressing for recovery could be incorporated into the Setup and Teardown.

[swe12345]

@generatz I ran it on BMC. Seems to be working fine

bash-4.2$ curl -k -H "X-Auth-Token: $bmc_token" -X PATCH -D patch.txt -d '{"IPv4StaticAddresses":[{},{"Address": "xx.xx.xx.xx","SubnetMask": "xx.xx.xx.xx","Gateway":"xx.xx.xx.xx"}]}' https://$bmc/redfish/v1/Managers/bmc/EthernetInterfaces/eth0

[generatz]

Thanks, @swe12345. I had problems with my configuration for the BMC (hardware). It is working there now.

The (virtual) romulus bmc isn't conventionally networked (it is running on the same host as the Jenkins agent). I think it is for that reason that the test fails in one way or another, even after I tried to adjust the ip addresses. In the end, after getting the patch to be accepted, the test failed when it attempted cat /usr/share/ipmi-providers/channel_config.json via ssh.

[gkeishin]

Looks resolved.. Closing ticket