redhat-performance / badfish

Vendor-agnostic tool for managing bare-metal systems via the Redfish API
https://quads.dev
GNU General Public License v3.0
93 stars 26 forks source link

Boot order changing ends with exception. #9

Closed misacek007 closed 5 years ago

misacek007 commented 5 years ago

I'm trying to change boot order on dell r620 but ends up with exception.

[root@centos7 badfish]# ~/git/badfish/badfish.py -u quads -p password -H mgmt-b08-h03-r620.fqdn --check-boot
Current boot order:
1: NIC.Integrated.1-3-1
2: HardDisk.List.1-1
3: NIC.Slot.2-4
4: NIC.Slot.2-1
5: NIC.Slot.2-2
6: NIC.Slot.2-3
[root@centos7 badfish]# ~/git/badfish/badfish.py -u quads -p password -H mgmt-b08-h03-r620.fqdn --boot-to NIC.Slot.2-4
- PASS: Command passed to set BIOS attribute pending values
- PASS: status code 204 returned for POST command to reset iDRAC
- WARNING, iDRAC will now reset and be back online within a few minutes.
- FAIL: POST command failed to create BIOS config job, status code is 401
{'cookies': <<class 'requests.cookies.RequestsCookieJar'>[]>, '_content': '', 'headers': {'transfer-encoding': 'chunked', 'accept-ranges': 'bytes', 'keep-alive': 'timeout=60, max=199', 'connection': 'Keep-Alive', 'date': 'Tue, 29 Jan 2019 18:01:08 GMT', 'www-authenticate': 'Basic realm="RedfishService"'}, 'url': u'https://mgmt-b08-h03-r620.fqdn/redfish/v1/Managers/iDRAC.Embedded.1/Jobs', 'status_code': 401, '_content_consumed': True, 'encoding': None, 'request': <PreparedRequest [POST]>, 'connection': <requests.adapters.HTTPAdapter object at 0x7fcd2300cf90>, 'elapsed': datetime.timedelta(0, 9, 36103), 'raw': <requests.packages.urllib3.response.HTTPResponse object at 0x7fcd2301fd10>, 'reason': 'Unauthorized', 'history': []}
Traceback (most recent call last):
  File "/root/git/badfish/badfish.py", line 449, in <module>
    sys.exit(main())
  File "/root/git/badfish/badfish.py", line 419, in main
    badfish.reboot_server()
  File "/root/git/badfish/badfish.py", line 204, in reboot_server
    data = _response.json()
  File "/usr/lib/python2.7/site-packages/requests/models.py", line 802, in json
    return json.loads(self.text, **kwargs)
  File "/usr/lib64/python2.7/json/__init__.py", line 338, in loads
    return _default_decoder.decode(s)
  File "/usr/lib64/python2.7/json/decoder.py", line 366, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/lib64/python2.7/json/decoder.py", line 384, in raw_decode
    raise ValueError("No JSON object could be decoded")
ValueError: No JSON object could be decoded

It seems to reboot drac but boot order is unchanged once it comes back.

Originally posted by @misacek007 in https://github.com/redhat-performance/badfish/issues/8#issuecomment-458517192

sadsfae commented 5 years ago

Thanks for filing this. After your assignment ends we'll see about recreating this against your host listed and tracking down what's wrong. Often we've seen strangeness in some of the older r620 units when they haven't had a full power drain. So long as the iDRAC firmware is at 2.60.60.60 it should have enough of the Redfish API functionality to work.

grafuls commented 5 years ago

I was able to recreate this. Added some exception handling plus a check for host state in: https://github.com/redhat-performance/badfish/commit/00f8a841da3200d9b118735be1dd7da3b527fb44

misacek007 commented 5 years ago

Thanks. I'll recheck on next occassion I got in the lab (can be months).