We just experienced a hardware issue with one of the systems we're testing such that the DMTF Redfish Usecase test fails as follows:
$ python3 Redfish-Usecase-Checkers/power_control/power_control.py -r xx.xx.xx.xx:443 -u root -p 0penBmcXXX -S Always ; echo "rc=$?"
Found 1 system instances
Resetting system using On
Monitoring power state for system...
ERROR: system was not in the On state after using On as the reset type.
Resetting system using ForceOff
Monitoring power state for system...
Resetting system using ForceOn
Monitoring power state for system...
ERROR: system was not in the On state after using ForceOn as the reset type.
Resetting system using ForceRestart
Monitoring power state for system...
ERROR: system was not in the On state after using ForceRestart as the reset type.
Resetting system using GracefulRestart
Monitoring power state for system...
ERROR: system was not in the On state after using GracefulRestart as the reset type.
Resetting system using GracefulShutdown
Monitoring power state for system...
Resetting system using PowerCycle
Monitoring power state for system...
ERROR: system was not in the On state after using PowerCycle as the reset type.
rc=1
The failed test exposes the following crash in redfish/dmtf_tools/Redfish_Usecase_Checkers.robot, as reported in the log:
Traceback (most recent call last):
File "/home/ojenkins/workspace/icebridge_redfish_dmtf_tools/lib/gen_cmd.py", line 427, in shell_cmd
BuiltIn().fail(err_msg)
File "/home/ojenkins/workspace/icebridge_redfish_dmtf_tools/temp-python/lib/python3.6/site-packages/robot/libraries/BuiltIn.py", line 534, in fail
raise AssertionError(msg) if msg else AssertionError()
As a consequence, no useful information is reported about the actual problem.
Here's my patch to show how I updated redfish/dmtf_tools/Redfish_Usecase_Checkers.robot to work around the crash and to report the test case results, as follows:
$ git diff redfish/dmtf_tools/Redfish_Usecase_Checkers.robot
diff --git a/redfish/dmtf_tools/Redfish_Usecase_Checkers.robot b/redfish/dmtf_tools/Redfish_Usecase_Checkers.robot
index 49304085..fdfa369e 100644
--- a/redfish/dmtf_tools/Redfish_Usecase_Checkers.robot
+++ b/redfish/dmtf_tools/Redfish_Usecase_Checkers.robot
@@ -23,7 +23,7 @@ ${command_account} ${DEFAULT_PYTHON} ${rsv_dir_path}${/}account_managemen
${command_power_control} ${DEFAULT_PYTHON} ${rsv_dir_path}${/}power_control/power_control.py
... -r ${OPENBMC_HOST}:${HTTPS_PORT} -u ${OPENBMC_USERNAME}
-... -p ${OPENBMC_PASSWORD} -S Always
+... -p ${OPENBMC_PASSWORD} -S Always -d ${EXECDIR}${/}logs${/}
${power_on_timeout} 7 mins
${power_off_timeout} 7 mins
@@ -83,6 +83,33 @@ Test Teardown Execution
DMTF Power
[Documentation] Power the BMC machine on via DMTF tools.
- Print Timen Doing "DMTF Power".
+ Print Timen Running "DMTF Power".
+ ${output}= Run DMTF Tool ${rsv_dir_path} ${command_power_control} check_error=1
+ Log ${output}
+
+ ${json}= OperatingSystem.Get File ${EXECDIR}${/}logs${/}results.json
+
+ ${object}= Evaluate json.loads('''${json}''') json
+
+ ${result_list}= Set Variable ${object["TestResults"]}
+
+ @{failed_tc_list}= Create List
+ @{error_messages}= Create List
+
+ Log result: ${result_list}
+
+ FOR ${result} IN @{result_list}
+ ${rc}= evaluate 'ErrorMessages'=='${result}'
+ ${num}= Run Keyword If ${rc} == False Set Variable ${result_list["${result}"]["fail"]}
+ Run Keyword If ${num} != None and ${num} > 0 Append To List ${failed_tc_list} ${result}
+ ${error_messages}= Run Keyword If ${rc} == True Set Variable
+ ... ${result_list["ErrorMessages"]}
+ END
+
+ Log Many ErrorMessages: @{error_messages}
+ Log To Console ErrorMessages:
+ FOR ${msg} IN @{error_messages}
+ Log To Console ${msg}
+ END
- Run DMTF Tool ${rsv_dir_path} ${command_power_control}
+ Should Be Empty ${error_messages} DMTF Power keyword failed.
Here's the resulting modified Keyword:
DMTF Power
[Documentation] Power the BMC machine on via DMTF tools.
Print Timen Running "DMTF Power".
${output}= Run DMTF Tool ${rsv_dir_path} ${command_power_control} check_error=1
Log ${output}
${json}= OperatingSystem.Get File ${EXECDIR}${/}logs${/}results.json
${object}= Evaluate json.loads('''${json}''') json
${result_list}= Set Variable ${object["TestResults"]}
@{failed_tc_list}= Create List
@{error_messages}= Create List
Log result: ${result_list}
FOR ${result} IN @{result_list}
${rc}= evaluate 'ErrorMessages'=='${result}'
${num}= Run Keyword If ${rc} == False Set Variable ${result_list["${result}"]["fail"]}
Run Keyword If ${num} != None and ${num} > 0 Append To List ${failed_tc_list} ${result}
${error_messages}= Run Keyword If ${rc} == True Set Variable
... ${result_list["ErrorMessages"]}
END
Log Many ErrorMessages: @{error_messages}
Log To Console ErrorMessages:
FOR ${msg} IN @{error_messages}
Log To Console ${msg}
END
Should Be Empty ${error_messages} DMTF Power keyword failed.
We just experienced a hardware issue with one of the systems we're testing such that the DMTF Redfish Usecase test fails as follows:
The failed test exposes the following crash in redfish/dmtf_tools/Redfish_Usecase_Checkers.robot, as reported in the log:
As a consequence, no useful information is reported about the actual problem.
Here's my patch to show how I updated redfish/dmtf_tools/Redfish_Usecase_Checkers.robot to work around the crash and to report the test case results, as follows:
Here's the resulting modified Keyword: