redhat-performance / badfish

Vendor-agnostic tool for managing bare-metal systems via the Redfish API
https://quads.dev
GNU General Public License v3.0
93 stars 26 forks source link

badfish not working reliably with default #8

Closed misacek007 closed 5 years ago

misacek007 commented 5 years ago

I'm probably only missing something because it worked once and then it never did.

It's fresh clone of badfish with default config trying to configure boot_order on dell 630 which should be perfectly possible. Maybe there is need to retry something several times before it succeeds? (blind idea)

[root@c10-h30-r630 badfish]# python2 badfish.py -H mgmt-c10-h33-r630.fqdn -u quads -p password -t director -i config/idrac_interfaces.yml
- PASS: PATCH command passed to update boot order
- FAIL: POST command failed to create BIOS config job, status code is 400
{'cookies': <<class 'requests.cookies.RequestsCookieJar'>[]>, '_content': '', 'headers': {'Content-Length': '0', 'Accept-Ranges': 'bytes', 'Keep-Alive': 'timeout=60, max=199', 'Server': 'iDRAC/8', 'Connection': 'Keep-Alive', 'Cache-Control': 'no-cache', 'Date': 'Tue, 18 Dec 2018 21:51:44 GMT', 'OData-Version': '4.0', 'Access-Control-Allow-Origin': '*', 'Content-Type': 'application/json;odata.metadata=minimal;charset=utf-8'}, 'url': u'https://mgmt-c10-h33-r630.fqdn/redfish/v1/Managers/iDRAC.Embedded.1/Jobs', 'status_code': 400, '_content_consumed': True, 'encoding': 'utf-8', 'request': <PreparedRequest [POST]>, 'connection': <requests.adapters.HTTPAdapter object at 0x7fd03465ffd0>, 'elapsed': datetime.timedelta(0, 5, 165930), 'raw': <urllib3.response.HTTPResponse object at 0x7fd034676850>, 'reason': 'Bad Request', '_next': None, 'history': []}
[root@c10-h30-r630 badfish]# curl -k -u quads:password https://mgmt-c10-h31-r630.fqdn/redfish/v1/Managers/iDRAC.Embedded.1/Jobs
{"@odata.context":"/redfish/v1/$metadata#DellJobCollection.DellJobCollection","@odata.id":"/redfish/v1/Managers/iDRAC.Embedded.1/Jobs","@odata.type":"#DellJobCollection.DellJobCollection","Description":"Collection of Job Instances","Id":"JobQueue","Members":[],"Members@odata.count":0,"Name":"JobQueue"}
[root@c10-h30-r630 badfish]# curl -k -u quads:password https://mgmt-c10-h31-r630.fqdn/redfish/v1/Managers/iDRAC.Embedded.1/Jobs | python -m json.tool

{
    "@odata.context": "/redfish/v1/$metadata#DellJobCollection.DellJobCollection",
    "@odata.id": "/redfish/v1/Managers/iDRAC.Embedded.1/Jobs",
    "@odata.type": "#DellJobCollection.DellJobCollection",
    "Description": "Collection of Job Instances",
    "Id": "JobQueue",
    "Members": [],
    "Members@odata.count": 0,
    "Name": "JobQueue"
}
grafuls commented 5 years ago

Can confirm the issue. For some reason the request made by badfish is not being accepted by the redfish API although I am able to do a successful request via curl:

curl -k -X POST -H 'Content-Type: application/json' -d '{"TargetSettingsURI": "/redfish/v1/Systems/System.Embedded.1/Bios/Settings"}' --user quads:password https://mgmt-c10-h33-r630.fqdn/redfish/v1/Managers/iDRAC.Embedded.1/Jobs
grafuls commented 5 years ago

This should have been addressed here: b7c74ce0ab3fe912dcf0456f053ccb830e887272

Now performing a racreset which will trigger the job for the boot source settings change to take effect. This process might take a few minutes to commit.

misacek007 commented 5 years ago

I finally had time to retest with previous comment commit it seems not really fix the issue (or it might be completely different issue).

[root@centos7 badfish]# ~/git/badfish/badfish.py -u user -p password -H mgmt-b08-h03-r620.fqdn --check-boot
Current boot order:
1: NIC.Integrated.1-3-1
2: HardDisk.List.1-1
3: NIC.Slot.2-4
4: NIC.Slot.2-1
5: NIC.Slot.2-2
6: NIC.Slot.2-3
[root@centos7 badfish]# ~/git/badfish/badfish.py -u user -p password -H mgmt-b08-h03-r620.fqdn --boot-to NIC.Slot.2-4
- PASS: Command passed to set BIOS attribute pending values
- PASS: status code 204 returned for POST command to reset iDRAC
- WARNING, iDRAC will now reset and be back online within a few minutes.
- FAIL: POST command failed to create BIOS config job, status code is 401
{'cookies': <<class 'requests.cookies.RequestsCookieJar'>[]>, '_content': '', 'headers': {'transfer-encoding': 'chunked', 'accept-ranges': 'bytes', 'keep-alive': 'timeout=60, max=199', 'connection': 'Keep-Alive', 'date': 'Tue, 29 Jan 2019 18:01:08 GMT', 'www-authenticate': 'Basic realm="RedfishService"'}, 'url': u'https://mgmt-b08-h03-r620.fqdn/redfish/v1/Managers/iDRAC.Embedded.1/Jobs', 'status_code': 401, '_content_consumed': True, 'encoding': None, 'request': <PreparedRequest [POST]>, 'connection': <requests.adapters.HTTPAdapter object at 0x7fcd2300cf90>, 'elapsed': datetime.timedelta(0, 9, 36103), 'raw': <requests.packages.urllib3.response.HTTPResponse object at 0x7fcd2301fd10>, 'reason': 'Unauthorized', 'history': []}
Traceback (most recent call last):
  File "/root/git/badfish/badfish.py", line 449, in <module>
    sys.exit(main())
  File "/root/git/badfish/badfish.py", line 419, in main
    badfish.reboot_server()
  File "/root/git/badfish/badfish.py", line 204, in reboot_server
    data = _response.json()
  File "/usr/lib/python2.7/site-packages/requests/models.py", line 802, in json
    return json.loads(self.text, **kwargs)
  File "/usr/lib64/python2.7/json/__init__.py", line 338, in loads
    return _default_decoder.decode(s)
  File "/usr/lib64/python2.7/json/decoder.py", line 366, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/lib64/python2.7/json/decoder.py", line 384, in raw_decode
    raise ValueError("No JSON object could be decoded")
ValueError: No JSON object could be decoded

It seems to reboot drac but boot order is unchanged once it comes back.