dell / iDRAC-Redfish-Scripting

Python and PowerShell scripting for Dell EMC PowerEdge iDRAC REST API with DMTF Redfish
GNU General Public License v2.0
600 stars 278 forks source link

Difference between Physical and Virtual CD #90

Closed kingil closed 4 years ago

kingil commented 4 years ago

Hello, guys.

I stucked with iDRAC and Redfish API. I try to BootNextTime with virtual cd, but my iDRAC want to boot from physical cd.

It's work fine when i try to boot from virtual cd in web version, because i have more options in boot once menu (separately physical and virtual). But in Redfish i have only this:

-Supported values for next server reboot, one time boot: None Pxe Cd Floppy Hdd BiosSetup Utilities UefiTarget SDCard UefiHttp

When i use "Cd", server try to boot from Physical only. What's way to boot iDRAC from virtual cd over Redfish query?

I tried to use:

curl -XPATCH -u ${IPMI_USERNAME}:${IPMI_PASSWORD}  "https://${IPMI_ADDRESS}/redfish/v1/Managers/iDRAC.Embedded.1/Attributes/" --insecure -H "Content-Type: application/json" -d '{ "Attributes": { "ServerBoot.1.FirstBootDevice": "VCD-DVD" }}'
curl -XPATCH -u ${IPMI_USERNAME}:${IPMI_PASSWORD} "https://${IPMI_ADDRESS}/redfish/v1/Managers/iDRAC.Embedded.1/Attributes/" --insecure -H "Content-Type: application/json" -d '{ "Attributes": { "VirtualMedia.1.BootOnce": "Enabled" }}'
curl -XPATCH -u ${IPMI_USERNAME}:${IPMI_PASSWORD}  "https://${IPMI_ADDRESS}/redfish/v1/Managers/iDRAC.Embedded.1/Actions/Oem/EID_674_Manager.ImportSystemConfiguration" --insecure -H "Content-Type: application/json" -d '{"ShareParameters":{"Target":"ALL"},"ImportBuffer":"<SystemConfiguration><Component FQDD=\"iDRAC.Embedded.1\"><Attribute Name=\"ServerBoot.1#BootOnce\">Enabled</Attribute><Attribute Name=\"ServerBoot.1#FirstBootDevice\">VCD-DVD</Attribute></Component></SystemConfiguration>" }'

I am really unhappy, because i can't boot from virtual media in iDRAC 8. It's needed to deploy new servers by clicking Jenkins task with ansible playbook.

texroemer commented 4 years ago

Hi @kingil

DMTF doesn't support next onetime boot to virtual media devices so you will need to use OEM extension which you were able to get working for iDRAC 9 but not on iDRAC 7/8. For iDRAC 7/8, URI for setting individual attributes is not supported, you will need to use action "redfish/v1/Managers/iDRAC.Embedded.1/Actions/Oem/EID_674_Manager.ImportSystemConfiguration". This action will work on iDRAC 7/8 along with iDRAC 9.

Please refer to python script "SetNextOneTimeBootVirtualMediaDeviceOemREDFISH.py" which shows setting next onetime boot to either virtual CD or virtual floppy using "redfish/v1/Managers/iDRAC.Embedded.1/Actions/Oem/EID_674_Manager.ImportSystemConfiguration"

Thanks

kingil commented 4 years ago

I tried this, but still not working:

mac:~ kingil$ python3 SetNextOneTimeBootVirtualMediaDeviceOemREDFISH.py -ip 192.168.36.111 -u ADMIN -p password -d 1 -r y

- WARNING, setting next onetime boot device to Virtual CD

- FAIL: status code 503 returned
- Detailed error information: {'_content': b'', '_content_consumed': True, '_next': None, 'status_code': 503, 'headers': {'Strict-Transport-Security': 'max-age=63072000', 'Vary': 'Accept-Encoding', 'Content-Type': 'application/json; charset=utf-8', 'Server': 'iDRAC/8', 'Date': 'Tue, 28 Jan 2020 15:09:39 GMT', 'Cache-Control': 'no-cache', 'Connection': 'Keep-Alive', 'Transfer-Encoding': 'chunked', 'Accept-Ranges': 'bytes'}, 'raw': <urllib3.response.HTTPResponse object at 0x10a0b0050>, 'url': 'https://192.168.36.111/redfish/v1/Managers/iDRAC.Embedded.1/Actions/Oem/EID_674_Manager.ImportSystemConfiguration', 'encoding': 'utf-8', 'history': [], 'reason': 'Service Unavailable', 'cookies': <RequestsCookieJar[]>, 'elapsed': datetime.timedelta(microseconds=634104), 'request': <PreparedRequest [POST]>, 'connection': <requests.adapters.HTTPAdapter object at 0x10a097510>}

What's wrong? I can't solve this. I had read iDRAC redfish docs, but nothing found. Last 2.70.70 firmware. Tried to reset from web UI, from SSH.

kingil commented 4 years ago

I was rollbacked to 2.40.40 firmware and next, i update to 2.70.70. After that i run script and first time it works fine.

- WARNING, setting next onetime boot device to Virtual CD
- PASS, successfully set next onetime boot device to Virtual CD

- WARNING, Current server power state is: On
- PASS, Command passed to gracefully power OFF server, 204 status code returned
- PASS, GET command passed to verify server is in OFF state
- PASS, Command passed to power ON server, 204 status code returned
- WARNING, System will now reboot and onetime boot to Virtual CD after POST

But second attempt failed:

- WARNING, setting next onetime boot device to Virtual CD

- FAIL: status code 503 returned
- Detailed error information: {'_content': b'', '_content_consumed': True, '_next': None, 'status_code': 503, 'headers': {'Strict-Transport-Security': 'max-age=63072000', 'Vary': 'Accept-Encoding', 'Content-Type': 'application/json; charset=utf-8', 'Server': 'iDRAC/8', 'Date': 'Tue, 28 Jan 2020 20:08:27 GMT', 'Cache-Control': 'no-cache', 'Connection': 'Keep-Alive', 'Transfer-Encoding': 'chunked', 'Accept-Ranges': 'bytes'}, 'raw': <urllib3.response.HTTPResponse object at 0x104d4b250>, 'url': 'https://192.168.36.111/redfish/v1/Managers/iDRAC.Embedded.1/Actions/Oem/EID_674_Manager.ImportSystemConfiguration', 'encoding': 'utf-8', 'history': [], 'reason': 'Service Unavailable', 'cookies': <RequestsCookieJar[]>, 'elapsed': datetime.timedelta(seconds=3, microseconds=704164), 'request': <PreparedRequest [POST]>, 'connection': <requests.adapters.HTTPAdapter object at 0x104d35290>}
texroemer commented 4 years ago

Hi @kingil

Can you give me more details on what you did after executing it the first time before executing the script again?

After it passed the first time, did you verify server rebooted, completed POST and one time boot to the virtual CD, then you attempted the script again? I just tested this workflow my server and didn't see any issues.

Thanks

kingil commented 4 years ago

After first attempt, i see that server reboot, show post, show "Boot from Virtual CD", then boot Virtual ISO which mounted with InsertMedia in redfish (firmware update iso). When process finished, server reboot from hdd and i want to try process again (writing playbook to autoupdate when install new servers).

I use script and it return 503. Maybe you should try 5 or 10 attempt to reproduce that.

kingil commented 4 years ago

Do I understand correctly that for proper import i need to do this:

curl -k -u ADMIN:PASSWORD -XPOST https://iDRACHOST/redfish/v1/Managers/iDRAC.Embedded.1/Actions/Oem/EID_674_Manager.ExportSystemConfiguration -H "Content-Type: application/json" -d '{"ShareParameters":{"Target":"ALL"},"ImportBuffer":"<SystemConfiguration><Component FQDD=\"iDRAC.Embedded.1\"><Attribute Name=\"ServerBoot.1#BootOnce\">Enabled</Attribute><Attribute Name=\"ServerBoot.1#FirstBootDevice\">VCD-DVD</Attribute></Component></SystemConfiguration>"}'

Can I check the correct operation with that query, or do I need to do something else?

texroemer commented 4 years ago

For status code 503, this means service unavailable. From the DMTF spec:

The server is currently unable to handle the request due to temporary overloading or maintenance of the server. A service may use this response to indicate that the request URI is valid, but the service is performing initialization or other maintenance on the resource. It may also use this response to indicate the service itself is undergoing maintenance, such as finishing initialization steps after reboot of the service.

When you hit the issue, did you immediately try the script again and it still fails or did it pass now?

For your CURL example, everything is correct except for the URI. You need to use "/redfish/v1/Managers/iDRAC.Embedded.1/Actions/Oem/EID_674_Manager.ImportSystemConfiguration" for importing or applying the configuration changes.

Thanks

kingil commented 4 years ago

Yes, of course i mean ImportSystemConfiguration, i made a typo.

I think I really understand what's the matter. When I change SCP, I cannot do this a second time or more if the server is not restarted. I mean, if i do Import twice or more, it break iDrac and i have problems, that can solve with reset controller, or change "nextboot" option in web UI.

Normally i need:

  1. Import scp
  2. Restart system and wait when it booted

What i did:

  1. Testing ansible playbook with curl import scp more that twice, and server return 503 for me.
  2. Restart of system is not reset 503 return status. Cold reset of idrac helps only first time.

I think it's problem in iDRAC, not in scripts. Maybe it's a feature (like anti-ddos), but need more informative description in docs.

Sorry for bugging you.

texroemer commented 4 years ago

Hi @kingil

I was able to repro your issue. When you execute SCP import POST operation, it returns a job ID URI in the output. You need to execute GET on this job ID URI, validate it's marked completed successfully before you attempt to execute SCP import POST operation again.

Also the error message returned with status code 503 will give you this information about a job ID already exists.

[root@linux ~]# curl -k -u root:calvin -X POST https://192.168.0.120/redfish/v1/Managers/iD RAC.Embedded.1/Actions/Oem/EID_674_Manager.ImportSystemConfiguration -H "Content-Type: app lication/json" -d '{"ShareParameters":{"Target":"ALL"},"ImportBuffer":"<SystemConfiguratio n><Component FQDD=\"iDRAC.Embedded.1\"><Attribute Name=\"ServerBoot.1#BootOnce\">Enabled</ Attribute><Attribute Name=\"ServerBoot.1#FirstBootDevice\">VCD-DVD "}' -i HTTP/1.1 202 Accepted Date: Mon, 03 Feb 2020 13:42:06 GMT Server: Apache OData-Version: 4.0 Access-Control-Allow-Origin: * Cache-Control: no-cache X-Frame-Options: DENY Strict-Transport-Security: max-age=63072000; includeSubDomains; preload Location: /redfish/v1/TaskService/Tasks/JID_807373267620 Content-Length: 0 Content-Type: application/json;odata.metadata=minimal;charset=utf-8

[root@linux ~]# curl -k -u root:calvin -X POST https://192.168.0.120/redfish/v1/Managers/iD RAC.Embedded.1/Actions/Oem/EID_674_Manager.ImportSystemConfiguration -H "Content-Type: app lication/json" -d '{"ShareParameters":{"Target":"ALL"},"ImportBuffer":"<SystemConfiguratio n><Component FQDD=\"iDRAC.Embedded.1\"><Attribute Name=\"ServerBoot.1#BootOnce\">Enabled</ Attribute><Attribute Name=\"ServerBoot.1#FirstBootDevice\">VCD-DVD "}' -i HTTP/1.1 503 Service Unavailable Date: Mon, 03 Feb 2020 13:42:07 GMT Server: Apache OData-Version: 4.0 Access-Control-Allow-Origin: * Cache-Control: no-cache X-Frame-Options: DENY Strict-Transport-Security: max-age=63072000; includeSubDomains; preload Content-Length: 567 Connection: close Content-Type: application/json;odata.metadata=minimal;charset=utf-8

{"error":{"@Message.ExtendedInfo":[{"Message":"Unable to create a configuration job because an existing configuration job is already in progress.","MessageArgs":[],"MessageArgs@odata.count":0,"MessageId":"IDRAC.1.6.RAC052","RelatedProperties":[],"RelatedProperties@odata.count":0,"Resolution":"Retry the operation after the existing configuration job is complete, or cancel the existing configuration job and retry the operation.","Severity":"Warning"}],"code":"Base.1.2.GeneralError","message":"A general error has occurred. See ExtendedInfo for more information"}}

kingil commented 4 years ago

In my case i don't have description, because body content is not returned for me. I had only 503 header and nothing more. :) Thanks!

rlopez133 commented 4 years ago

@kingil would you mind sharing your ansible task here? I'm running into the same 503, I check racadm queue view and I don't see any jobs but getting the 503.

Was trying something like

- name: Set Boot Override to VirtualMedia
  uri:
    url: "https://192.168.0.2/redfish/v1/Managers/iDRAC.Embedded.1/Actions/Oem/EID_674_Manager.ImportSystemConfiguration"
    validate_certs: no
    method: POST
    user: "ADMIN"
    password: "PASSWORD"
    headers:
      accept: application/json
      content-type: application/json
    body: '{"ShareParameters":{"Target":"ALL"},"ImportBuffer":"<SystemConfiguration><Component FQDD=\"iDRAC.Embedded.1\"><Attribute Name=\"ServerBoot.1#BootOnce\">Enabled</Attribute><Attribute Name=\"ServerBoot.1#FirstBootDevice\">VCD-DVD</Attribute></Component></SystemConfiguration>"}'
TASK [install : Set Boot Override to VirtualMedia] ****************************************************************************
FAILED! => {"accept_ranges": "bytes", "cache_control": "no-cache", "changed": false, "connection": "close", "content": "", "content_type": "application/json; charset=utf-8", "date": "Tue, 23 Jun 2020 21:41:27 GMT", "elapsed": 4, "msg": "Status code was 503 and not [200]: HTTP Error 503: Service Unavailable", "redirected": false, "server": "iDRAC/8", "status": 503, "strict_transport_security": "max-age=63072000", "transfer_encoding": "chunked", "url": "https://192.168.0.2/redfish/v1/Managers/iDRAC.Embedded.1/Actions/Oem/EID_674_Manager.ImportSystemConfiguration", "vary": "Accept-Encoding"}