dell / iDRAC-Redfish-Scripting

Python and PowerShell scripting for Dell EMC PowerEdge iDRAC REST API with DMTF Redfish
GNU General Public License v2.0
598 stars 276 forks source link

VirtualMedia inserted but does not map #236

Closed DBoyleCTDI closed 1 year ago

DBoyleCTDI commented 1 year ago

Hi,

When attempting to inject an OS iso image (RHEL 9.1) and set the OneTimeBoot parameter for VirtualMedia boot I am getting back a successful mount as well as an accepted OneTimeBoot parameter but upon rebooting the iDRAC atttempts a VirtualMedia boot and returns Boot: Virtual Optical Drive failed.

I'm performing these with the below commands: Insert Virtual Media InsertEjectVirtualMediaREDFISH.py -ip 192.168.0.120 -u root -p calvin --action insert --device cd --uripath http://192.168.0.200:8080/RHEL-9.1.0-20221027.3-x86_64-dvd1.iso

OneTimeBoot SetNextOneTimeBootVirtualMediaDeviceOemREDFISH.py -ip 192.168.0.120 -u root -p calvin --device 1

The only difference I can note when doing this manually which is successful via the Remote Console is that you must not only "Connect the Virtual Media" you also have to "Map" the drive. The secondary portion seems to be what is missing from the Redfish functionality as the boot order changes doing this manually vs via Redfish (manually the VirtualDisk boot option is loaded 3rd vs the VirtualDisk boot option loading 1st and failing)

I've confirmed this process works manually so the issue doesn't seem to be the HTTP server but let me know if more information would help.

System Information: System: Dell PowerEdge XR11 iDRAC Version: iDRAC9 version 5.10.50.00 BIOS: 1.6.5 OS: None

Thanks, Dylan Boyle

texroemer commented 1 year ago

Hi @DBoyleCTDI

I setup a XR11 in my lab with same iDRAC version and BIOS as your server, unable to repro the issue you're seeing. When iDRAC attaches the ISO as a virtual CD and once you set onetime boot flag to virtual CD, BIOS now takes control and will attempt to boot to the ISO after POST and if fails to boot, will return the error you're seeing.

Questions:

  1. Did you try setting onetime boot to virtual CD multiple times, same issue?

  2. After attaching the ISO, can you use script SetNextOneTimeBootDeviceREDFISH.py instead to set onetime boot to Cd, see if onetime boot passes or still fails? Example: SetNextOneTimeBootDeviceREDFISH.py -ip 192.168.0.120 -u root -p calvin --device Cd --reboot

  3. If still seeing issues, can you set iDRAC attribute "VirtualMedia.1.Attached" to Attached (example: SetIdracLcSystemAttributesREDFISH.py -ip 192.168.0.120 -u root -p calvin --set idrac --attribute-name VirtualMedia.1.Attached --attribute-value Attached), reboot the server and select F11 Boot Manager, select "Virtual Optical" from one time boot menu and see if the ISO boots.

Thanks Tex

DBoyleCTDI commented 1 year ago

Hi Tex,

Have tried the above solutions but unfortunately no luck. Below are the results from the individual attempts.

  1. Ran the following command "SetNextOneTimeBootVirtualMediaDeviceOemREDFISH.py -ip 192.168.0.120 -u root -p calvin --device 1" a few times in a row with the following results:

    1st Run:

    - INFO, setting next onetime boot device to Virtual CD
    - PASS, successfully set next onetime boot device to Virtual CD
    - INFO, argument --reboot not detected. Onetime boot will be applied on next manual server reboot

    Following Runs:

    - INFO, setting next onetime boot device to Virtual CD
    - INFO, next onetime boot device already set to Virtual CD, no changes applied
    - INFO, argument --reboot not detected. Onetime boot will be applied on next manual server reboot
  2. Attempting this resulted in much of the same as above. I received back the successful messages below but it still is unable to successfully boot the targeted VirtualMedia

- PASS, PATCH command passed to set next boot onetime boot device to "Cd"
- INFO, system will now reboot and boot to onetime boot device "Cd"
- INFO, Current server power state is: Off
- PASS, POST command passed to power ON server
  1. Finally trying this I do receive back a successful POST and the attribute is changed but does not succeed in allowing the VirtualMedia to boot
- INFO, configuring "IDRAC" attributes
 Attribute Name: VirtualMedia.1.Attached, setting new value to: Attached
- PASS, PATCH command passed to successfully set "IDRAC" attribute(s), status code 200 returned
- INFO, getting new attribute current values
- Attribute Name: VirtualMedia.1.Attached, Attribute Value: Attached

Only other thing I've noticed is under Virtual Media in iDRAC 9 GUI is "Connection Status". When I manually map the ISO this is successfully changing to "Connected", anytime I am attempting via scripts the ISO name appears under "Remote File Share" and the "Connection Status" under Virtual Media remains "Not Connected". Not sure if related or just a symptom of the true issue.

I did downgrade the BIOS from 1.7.5 to 1.6.5 (part of the current project) but that is the extent of the changes I've made. I've yet to reset to "Factory Default" settings but can certainly give that a shot if recommended.

Appreciate the above help, just let me know if there is any other information that could assist.

texroemer commented 1 year ago

After setting attribute VirtualMedia.1.Attached to Attached, did you rerun InsertEjectVirtualMediaREDFISH.py to attach the ISO, then reboot the server and enter F11 boot manager to select "Virtual Optical Drive" device, see if the server boots to the ISO or still fails? I just noticed i forgot to mention make sure the ISO is attached before you reboot the server and enter F11.

Workflow:

  1. Set attribute VirtualMedia.1.Attached to Attached.
  2. Attach the ISO using VirtualMedia.InsertMedia action
  3. Reboot the server and press F11 to enter BIOS boot manager
  4. Select one-shot UEFI boot menu
  5. Select Virtual Optical Drive
  6. Confirm if ISO boots

image

image

For the connection status property in the GUI under Virtual Media section, this property status is only tied to attaching virtual media using virtual console. Action VirtualMedia.InsertMedia will map to iDRAC Remote File Share (RFS) feature so this is why when you attach the ISO, you see the image attached under RFS section in the GUI.

If you still see issues can you try latest iDRAC 6.10.00?

https://www.dell.com/support/home/en-us/drivers/driversdetails?driverid=kmyv9&oscode=naa&productcode=poweredge-xr11

Thanks Tex

DBoyleCTDI commented 1 year ago

Awesome, appreciate the info on the RFS that makes total sense.

Did not have encounter success attempting the above option of running the below commands and then booting to the Virtual Optical Drive. Simple returns the same: Booting from Virtual Optical Disk Boot Failed: Virtual Optical Disk

SetIdracLcSystemAttributesREDFISH.py -ip 192.168.0.120 -u root -p calvin --set idrac --attribute-name VirtualMedia.1.Attached --attribute-value Attached

- INFO, configuring "IDRAC" attributes
 Attribute Name: VirtualMedia.1.Attached, setting new value to: Attached
- PASS, PATCH command passed to successfully set "IDRAC" attribute(s), status code 200 returned
- INFO, getting new attribute current values
- Attribute Name: VirtualMedia.1.Attached, Attribute Value: Attached
InsertEjectVirtualMediaREDFISH.py -ip 192.168.0.120 -u root -p calvin --action insert --index 2 --uripath http://192.168.0.200:8080/RHEL-9.1.0-20221027.3-x86_64-dvd1.iso

- PASS, POST command passed to successfully insert virtual media, status code 204 returned

With the above tested I updated my iDRAC firmware to the mentioned 6.10.00 to test that. Unfortunately I've tried each of the solutions mentioned above again with the 6.10.00 firmware and has not resolved the Boot failure and noted those below for clarity:

  1. Set one time boot multiple times
  2. Set one time boot with SetNextOneTimeBootDeviceREDFISH.py
  3. Set attribute VirtualMedia.1.Attached prior to Insert and then booting manually via F11 and selecting Virtual Optical Disk

I can also attempt to update BIOS but neither the 1.7.5 or 1.8.2 seemed to mention any fixes related to this.

Only difference I could note from above was the existence of a "Virtual Network File 2" which I assume is from the updated iDRAC version.

One other thing I noticed was under the Storage tab there is the following from the Storage Log which seems to line up with whenever I'm inserting the media but not necessarily sure why the RAID controller would be served this but when attempting to boot I received a different error of saying "Ensure the media is compatible": 2023-01-27 13:32:17 | The boot media of the Controller RAID Controller in Slot 1 is Disk.Virtual.239:RAID.Slot.1-1.

Bootable_Media

Worth mentioning I preformed the above firmware update utilizing the SimpleUpdate Redfish commands and placed the iDRAC firmware on the HTTP server that also houses the RHEL OS ISO to ensure the connection is stable for the VirtualMedia process/isn't causing the issue here as far as I can tell.

I assume we are running out of options but if you have any other suggestions or need other information please just let me know.

Edit: Another thought I had, I've noticed that the way the SimpleUpdate is preformed requires the firmware images to be uploaded prior to running the update. Haven't found any mention of if but does the VirtualMedia mount require time to "upload" the image? Did not seem like this was a requirement for the Virtual Console but learning not to compare the two for one to one functionality.

texroemer commented 1 year ago

Can you try this? Once you run "InsertEjectVirtualMediaREDFISH.py --attach" to attach the ISO, can you run "InsertEjectVirtualMediaREDFISH.py --get", confirm it reports the ISO attached. Now reboot the server and during POST run "InsertEjectVirtualMediaREDFISH.py --get" multiple times and see if the ISO is still attached. I'm thinking the ISO might be getting auto detached during POST which shouldn't be happening.

Your webserver, is it on the same local network as the iDRAC or on a different network?

What webserver are you using? Apache, Nginx?

Thanks Tex

DBoyleCTDI commented 1 year ago

The webserver was located on the same local network as the iDRAC (running the webserver on the host PC connected to iDRAC on a LAN).

Was using a simple web server application (Rebex) that does not seem to have supported the iDRAC file access despite functioning correctly for the BIOS/iDRAC simple update. Once I was able to log the output in a verbose setting the network connection with the webserver seemed unstable when attempting to mount the ISO.

For the sake of the time on this project I've opted to instead use the USB mount/boot for this process as I've had no issue running the installation this way.

I plan to attempt the VirtualMedia method again once I've setup a more robust web server (most likely Apache as we've had success with that in the past) but everything above seems to indicate an issue with our hastily setup web server.

Appreciate all of the help on this!