mkumatag / openbmc-automation

Test OpenBMC Distribution, moved to - https://github.com/openbmc/openbmc-test-automation
Apache License 2.0
5 stars 11 forks source link

PreTag Set Less Than Minimum PowerCAP - Wrong Implemtation #61

Closed gkeishin closed 8 years ago

gkeishin commented 8 years ago

Set Less Than Minimum PowerCAP [Documentation] Test set powercap with less than min powercap value ${org_power_cap}= Get PowerCap ${min}= Get Minimum PowerCap ${sample_invalid_pcap}= Evaluate ${min}-${100} ${resp}= Set PowerCap ${sample_invalid_pcap} Sleep ${DBUS_POLL_INTERVAL} Should Not Be Equal As Strings ${resp.status_code} ${HTTP_OK} ${power_cap}= Get PowerCap Should Be Equal ${org_power_cap} ${power_cap}

This code isn't making any sense out ... it sets the Power cap and waits for it NOT to be OK ? and then try to get that cap value again..

First of all this code needs to be revamp..

The respond check should be ideally under the keyword Get/Set context not in the test case for ONLY those expects the respond check OK or ERROR..

_Example:_\ Set PowerCap [Arguments] ${powercap_value} @{pcap_list} = Create List ${powercap_value} ${data} = create dictionary data=@{pcap_list} ${resp} = openbmc post request /org/openbmc/sensors/host/PowerCap/action/setValue data=${data} [return] ${resp}

To

Set PowerCap [Arguments] ${powercap_value} @{pcap_list} = Create List ${powercap_value} ${data} = create dictionary data=@{pcap_list} ${resp} = openbmc post request /org/openbmc/sensors/host/PowerCap/action/setValue data=${data} should be equal as strings ${resp.status_code} ${HTTP_OK}

and remove those response check from the test cases

and for Get Maximun/ Minimum PowerCap user define keyword Example: Get Minimum PowerCap ${resp} = OpenBMC Get Request /org/openbmc/sensors/powercap/min_cap ${jsondata}= To Json ${resp.content} [return] ${jsondata["data"]["value"]}

To 

Get Minimum PowerCap ${resp} = OpenBMC Get Request /org/openbmc/sensors/powercap/min_cap should be equal as strings ${resp.status_code} ${HTTP_OK} ${jsondata}= To Json ${resp.content} [return] ${jsondata["data"]["value"]}

mkumatag commented 8 years ago

Set PowerCap is keyword and I intentionally don't want to keep any checks here because I want to use that keyword only as a util function and use it wherever I want in positive or negative testcase.

gkeishin commented 8 years ago

My primary question was on are u expecting this Set to fail ? ${resp}= Set PowerCap ${sample_invalid_pcap} Sleep ${DBUS_POLL_INTERVAL} Should Not Be Equal As Strings ${resp.status_code} ${HTTP_OK} <----- ( Fails if objects are equal after converting them to strings. )

So are we expecting ERROR in this case ?

The remaining comments were on how to enhance the test cases..

mkumatag commented 8 years ago

OpenBMC REST call for setting powercap shouldn't accept any values like more than max or less than min.

What I'm expecting here is testcases should fail if set powercap accepts value less than min. The comparison statement make sure that set powercap does not return 200 OK.

mkumatag commented 8 years ago

FYI - refer https://github.com/openbmc/skeleton/issues/43 for more information

gkeishin commented 8 years ago

agreed .. It was broken earlier.. but now its returning HTTP OK..

Should Not Be Equal As Strings ${resp.status_code}, ${HTTP_OK} Documentation: Fails if objects are equal after converting them to strings. Start / End / Elapsed: 20160530 23:24:19.195 / 20160530 23:24:19.197 / 00:00:00.002 23:24:19.196 INFO Argument types are: <type 'int'> <type 'unicode'>
23:24:19.196 FAIL 200 == 200 <----- HTTP OK

So this test is broken.. This needs to be fix

mkumatag commented 8 years ago

Raise a issue in skeleton component for this.

gkeishin commented 8 years ago

Sure.... My concern was if this is some sort of REST server code bug returning Error even when that PowerCap property existing on the system..

If by design ( openbmc/skeleton#43 ) if its going not to return HTTP OK then, why is this return HTTP OK here on Palmetto..

Example: On Palmetto : openbmc-automation$ curl -c cjar -b cjar -k -H "Content-Type: application/json" -X POST -d "{\"data\": [ 1900 ]}" http://9.3.164.177/org/openbmc/sensors/host/PowerCap/action/setValue { "data": null, "message": "200 OK", "status": "ok" }

On Barrelleye: openbmc-automation$ curl -c cjar -b cjar -k -H "Content-Type: application/json" -X POST-d "{\"data\": [ 1900 ]}" https://9.3.23.24/org/openbmc/sensors/host/PowerCap/action/setValue { "data": { "description": "org.openbmc.objectmapper.Error.NotFound: path or object not found: /org/openbmc/sensors/host/PowerCap" }, "message": "404 Not Found", "status": "error" }

gkeishin commented 8 years ago

Raised ticket on Skeleton: https://github.com/openbmc/skeleton/issues/97

gkeishin commented 8 years ago

Just in case, if the HTTP OK request is confirmed by the REST team.. the code will needs the required changes.. Thanks