search

dell / iDRAC-Redfish-Scripting

Python and PowerShell scripting for Dell EMC PowerEdge iDRAC REST API with DMTF Redfish
GNU General Public License v2.0
598 stars 276 forks source link

Firmware Update #231

Closed MarcPro92 closed 1 year ago

MarcPro92 commented 1 year ago

Hello,

Sometime during iDRAC update, job fail and script exited. It seems that iDrac was in resetting mode before receiving the completed job status



* Name: Firmware Update: iDRAC
* Job Status: Downloading the iDRAC-with-Lifecycle-Controller_Firmware_WPNPP_WN64_5.10.30.00_A00.EXE update package.
* Current job execution time: 0:00:01

- INFO, job ID JID_677257538440 not marked completed, current job information:

* Name: Firmware Update: iDRAC with Lifecycle Controller
* Job Status: Downloading the iDRAC-with-Lifecycle-Controller_Firmware_WPNPP_WN64_5.10.30.00_A00.EXE update package.
* Current job execution time: 0:00:19

- INFO, job ID JID_677257538440 not marked completed, current job information:

* Name: Firmware Update: iDRAC with Lifecycle Controller
* Job Status: Downloading the iDRAC-with-Lifecycle-Controller_Firmware_WPNPP_WN64_5.10.30.00_A00.EXE update package.
* Current job execution time: 0:00:36

- INFO, job ID JID_677257538440 not marked completed, current job information:

* Name: Firmware Update: iDRAC with Lifecycle Controller
* Job Status: Downloading the iDRAC-with-Lifecycle-Controller_Firmware_WPNPP_WN64_5.10.30.00_A00.EXE update package.
* Current job execution time: 0:00:53

- INFO, job ID JID_677257538440 not marked completed, current job information:

* Name: Firmware Update: iDRAC with Lifecycle Controller
* Job Status: Downloading the iDRAC-with-Lifecycle-Controller_Firmware_WPNPP_WN64_5.10.30.00_A00.EXE update package.
* Current job execution time: 0:01:10

- INFO, job ID JID_677257538440 not marked completed, current job information:

* Name: Firmware Update: iDRAC with Lifecycle Controller
* Job Status: Downloading the iDRAC-with-Lifecycle-Controller_Firmware_WPNPP_WN64_5.10.30.00_A00.EXE update package.
* Current job execution time: 0:01:27

- INFO, job ID JID_677257538440 not marked completed, current job information:

* Name: Firmware Update: iDRAC with Lifecycle Controller
* Job Status: Downloading the iDRAC-with-Lifecycle-Controller_Firmware_WPNPP_WN64_5.10.30.00_A00.EXE update package.
* Current job execution time: 0:01:43

- INFO, job ID JID_677257538440 not marked completed, current job information:

* Name: Firmware Update: iDRAC with Lifecycle Controller
* Job Status: Downloading the iDRAC-with-Lifecycle-Controller_Firmware_WPNPP_WN64_5.10.30.00_A00.EXE update package.
* Current job execution time: 0:02:00

- INFO, job ID JID_677257538440 not marked completed, current job information:

* Name: Firmware Update: iDRAC with Lifecycle Controller
* Job Status: Downloading the iDRAC-with-Lifecycle-Controller_Firmware_WPNPP_WN64_5.10.30.00_A00.EXE update package.
* Current job execution time: 0:02:16

- INFO, job ID JID_677257538440 not marked completed, current job information:

* Name: Firmware Update: iDRAC with Lifecycle Controller
* Job Status: Downloading the iDRAC-with-Lifecycle-Controller_Firmware_WPNPP_WN64_5.10.30.00_A00.EXE update package.
* Current job execution time: 0:02:33

- INFO, job ID JID_677257538440 not marked completed, current job information:

* Name: Firmware Update: iDRAC with Lifecycle Controller
* Job Status: Downloading the iDRAC-with-Lifecycle-Controller_Firmware_WPNPP_WN64_5.10.30.00_A00.EXE update package.
* Current job execution time: 0:02:54

- INFO, job ID JID_677257538440 not marked completed, current job information:

* Name: Firmware Update: iDRAC with Lifecycle Controller
* Job Status: Downloading the iDRAC-with-Lifecycle-Controller_Firmware_WPNPP_WN64_5.10.30.00_A00.EXE update package.
* Current job execution time: 0:03:30

- INFO, job ID JID_677257538440 not marked completed, current job information:

* Name: Firmware Update: iDRAC with Lifecycle Controller
* Job Status: Downloading the iDRAC-with-Lifecycle-Controller_Firmware_WPNPP_WN64_5.10.30.00_A00.EXE update package.
* Current job execution time: 0:03:53

- FAIL, requests command failed to GET job status, detailed error information: 
('Connection aborted.', RemoteDisconnected('Remote end closed connection without response',))
- INFO, Script will wait 10 seconds and try to check job status again
- FAIL, requests command failed to GET job status, detailed error information: 
HTTPSConnectionPool(host='xx.xxx.xxx.xxx', port=443): Max retries exceeded with url: /redfish/v1/Managers/iDRAC.Embedded.1/Jobs/JID_677257538440 (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7ff89730b8d0>: Failed to establish a new connection: [Errno 111] Connection refused',))
- INFO, Script will wait 10 seconds and try to check job status again
- FAIL, requests command failed to GET job status, detailed error information: 
HTTPSConnectionPool(host='xx.xxx.xxx.xxx', port=443): Max retries exceeded with url: /redfish/v1/Managers/iDRAC.Embedded.1/Jobs/JID_677257538440 (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7ff8972e2080>: Failed to establish a new connection: [Errno 111] Connection refused',))
- INFO, Script will wait 10 seconds and try to check job status again
- FAIL, requests command failed to GET job status, detailed error information: 
HTTPSConnectionPool(host='xx.xxx.xxx.xxx', port=443): Max retries exceeded with url: /redfish/v1/Managers/iDRAC.Embedded.1/Jobs/JID_677257538440 (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7ff89730b668>: Failed to establish a new connection: [Errno 111] Connection refused',))
- INFO, Script will wait 10 seconds and try to check job status again
- FAIL, requests command failed to GET job status, detailed error information: 
HTTPSConnectionPool(host='xx.xxx.xxx.xxx', port=443): Max retries exceeded with url: /redfish/v1/Managers/iDRAC.Embedded.1/Jobs/JID_677257538440 (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7ff89729c358>: Failed to establish a new connection: [Errno 111] Connection refused',))
- INFO, Script will wait 10 seconds and try to check job status again
- FAIL, unable to get job status after 5 attempts, script will exit```
texroemer commented 1 year ago

Hi @MarcPro92

Can you let me know the name of the script and the script version you're using. Based off the script output you posted for polling job status it looks like you're using an older script version.

Thanks Tex

MarcPro92 commented 1 year ago

Hi @texroemer,

i'm using script : InstallFromRepositoryREDFISH.py in version 14 i known that it is not the last version but i don't see many difference into the "loop_job_status" between version 14 and 16

Rgds, Marc.

texroemer commented 1 year ago

Ok version 16 had updated job message checking and echoing to the screen. I went ahead and updated the script to add a new check if iDRAC update job is detected to poll job status every 1 second. Script should now catch the job getting marked completed before iDRAC reboot.

MarcPro92 commented 1 year ago

Hello @texroemer, i have another issue with this script (but not related to your update). on some servers firmware update stucks, server reboot but for example BIOS are not updated.

here is the script log :

- PASS, POST command passed for method "InstallFromRepository", status code 202 returned
- PASS, repository job ID JID_684275645399 successfully created
- INFO, script will now loop checking the repo update job status

- INFO, repository package successfully downloaded. If version changed detected for any device, update job ID will get created and execute for that device

- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:00:16
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:00:38
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:01:01
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:01:26
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:01:52
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:02:29
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:02:47
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:03:23
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:03:47
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:04:18
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:04:48
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:05:12
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:05:34
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:05:59
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:06:19
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:06:41
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:06:58
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:07:16
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:07:34
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:07:52
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:08:09
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:08:24
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:08:40
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:08:56
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:09:21
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:09:36
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:09:52
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:10:08
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:10:24
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:10:40
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:10:56
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:11:11
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:11:27
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:11:42
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:11:58
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:12:13
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:12:29
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:12:46
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:13:03
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:13:18
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:13:34
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:13:49
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:14:05
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:14:20
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:14:36
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:14:51
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:15:07
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:15:22
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:15:38
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:15:53
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:16:09
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:16:24
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:16:40
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:16:55
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:17:11
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:17:26
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:17:42
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:17:57
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:18:12
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:18:28
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:18:43
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:18:59
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:19:14
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:19:30
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:19:45
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:20:01
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:20:17
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:20:32
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:20:48
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:21:03
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:21:19
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:21:34
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:21:50
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:22:05
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:22:21
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:22:36
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:22:52
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:23:07
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:23:23
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:23:38
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:23:55
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:24:11
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:24:26
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:24:42
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:24:58
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:25:13
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:25:29
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:25:45
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:26:01
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:26:30
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:26:49
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:27:07
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:27:25
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:27:41
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:27:57
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:28:15
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:28:31
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:28:46
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:29:02
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:29:18
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:29:35
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:29:51
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:30:06
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:30:22
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:30:37
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:30:53
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:31:09
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:31:25
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:31:40
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:31:56
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:32:12
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:32:27
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:32:43
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:32:59
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:33:15
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:33:30
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:33:46
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:34:01
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:34:17
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:34:33
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:34:49
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:35:05
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:35:21
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:35:36
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:35:53
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:36:09
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:36:24
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:36:40
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:36:56
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:37:11
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:37:27
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:37:43
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:37:58
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:38:14
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:38:30
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:38:45
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:39:01
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:39:17
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:39:32
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:39:48
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:40:04
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:40:19
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:40:35
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:40:50
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:41:06
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:41:22
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:41:37
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:41:53
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:42:08
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:42:24
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:42:40
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:42:56
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:43:11
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:43:27
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:43:42
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:43:58
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:44:14
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:44:29
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:44:45
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:45:00
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:45:16
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:45:32
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:45:48
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:46:04
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:46:20
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:46:36
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:46:51
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:47:07
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:47:22
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:47:38
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:47:54
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:48:09
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:48:25
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:48:40
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:48:56
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:49:12
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:49:28
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:49:43
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:49:59
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:50:14
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:50:30
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:50:46
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:51:01
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:51:17
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:51:33
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:51:48
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:52:04
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:52:20
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:52:36
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:52:52
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:53:07
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:53:23
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:53:38
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:53:54
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:54:10
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:54:26
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:54:41
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:54:57
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:55:12
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:55:28
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:55:44
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:55:59
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:56:15
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:56:31
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:56:46
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:57:02
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:57:18
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:57:33
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:57:49
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:58:04
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:58:20
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:58:36
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:58:51
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:59:07
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:59:23
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:59:38
- INFO, Package successfully downloaded, JID_684275645399 execution time: 0:59:54
- INFO, Package successfully downloaded, JID_684275645399 execution time: 1:00:09
- INFO, Package successfully downloaded, JID_684275645399 execution time: 1:00:25
- INFO, Package successfully downloaded, JID_684275645399 execution time: 1:00:40
- INFO, Package successfully downloaded, JID_684275645399 execution time: 1:00:56
- INFO, Package successfully downloaded, JID_684275645399 execution time: 1:01:11
- INFO, Package successfully downloaded, JID_684275645399 execution time: 1:01:27
- INFO, Package successfully downloaded, JID_684275645399 execution time: 1:01:43
- INFO, Package successfully downloaded, JID_684275645399 execution time: 1:01:58
- INFO, Package successfully downloaded, JID_684275645399 execution time: 1:02:14
- INFO, Package successfully downloaded, JID_684275645399 execution time: 1:02:30
- INFO, Package successfully downloaded, JID_684275645399 execution time: 1:02:45
- INFO, Package successfully downloaded, JID_684275645399 execution time: 1:03:01
- INFO, Package successfully downloaded, JID_684275645399 execution time: 1:03:16
- INFO, Package successfully downloaded, JID_684275645399 execution time: 1:03:32
- INFO, Package successfully downloaded, JID_684275645399 execution time: 1:03:48
- INFO, Package successfully downloaded, JID_684275645399 execution time: 1:04:03
- INFO, Package successfully downloaded, JID_684275645399 execution time: 1:04:19
- INFO, Package successfully downloaded, JID_684275645399 execution time: 1:04:35
- INFO, Package successfully downloaded, JID_684275645399 execution time: 1:04:50
- INFO, Package successfully downloaded, JID_684275645399 execution time: 1:05:06
- INFO, Package successfully downloaded, JID_684275645399 execution time: 1:05:21
- INFO, Package successfully downloaded, JID_684275645399 execution time: 1:05:37
- INFO, Package successfully downloaded, JID_684275645399 execution time: 1:05:54
- INFO, Package successfully downloaded, JID_684275645399 execution time: 1:06:12
- INFO, Package successfully downloaded, JID_684275645399 execution time: 1:06:29
- INFO, Package successfully downloaded, JID_684275645399 execution time: 1:06:45
- INFO, Package successfully downloaded, JID_684275645399 execution time: 1:07:00
- INFO, Package successfully downloaded, JID_684275645399 execution time: 1:07:17
- INFO, Package successfully downloaded, JID_684275645399 execution time: 1:07:32
- INFO, Package successfully downloaded, JID_684275645399 execution time: 1:07:48
- INFO, Package successfully downloaded, JID_684275645399 execution time: 1:08:04
- INFO, Package successfully downloaded, JID_684275645399 execution time: 1:08:21
- INFO, Package successfully downloaded, JID_684275645399 execution time: 1:08:36
- INFO, Package successfully downloaded, JID_684275645399 execution time: 1:08:52
- INFO, Package successfully downloaded, JID_684275645399 execution time: 1:09:08
- INFO, Package successfully downloaded, JID_684275645399 execution time: 1:09:24
- INFO, Package successfully downloaded, JID_684275645399 execution time: 1:09:39
- INFO, Package successfully downloaded, JID_684275645399 execution time: 1:09:55
- INFO, Package successfully downloaded, JID_684275645399 execution time: 1:10:10
- INFO, Package successfully downloaded, JID_684275645399 execution time: 1:10:26
- INFO, Package successfully downloaded, JID_684275645399 execution time: 1:10:42
- INFO, Package successfully downloaded, JID_684275645399 execution time: 1:10:57
- INFO, Package successfully downloaded, JID_684275645399 execution time: 1:11:13
- INFO, Package successfully downloaded, JID_684275645399 execution time: 1:11:28
- INFO, Package successfully downloaded, JID_684275645399 execution time: 1:11:44
- INFO, Package successfully downloaded, JID_684275645399 execution time: 1:12:00
- INFO, Package successfully downloaded, JID_684275645399 execution time: 1:12:16
- INFO, Package successfully downloaded, JID_684275645399 execution time: 1:12:31
- INFO, Package successfully downloaded, JID_684275645399 execution time: 1:12:47
- INFO, Package successfully downloaded, JID_684275645399 execution time: 1:13:02
- INFO, Package successfully downloaded, JID_684275645399 execution time: 1:13:18
- INFO, Package successfully downloaded, JID_684275645399 execution time: 1:13:34
- INFO, Package successfully downloaded, JID_684275645399 execution time: 1:13:49
- INFO, Package successfully downloaded, JID_684275645399 execution time: 1:14:05
- INFO, Package successfully downloaded, JID_684275645399 execution time: 1:14:20
- INFO, Package successfully downloaded, JID_684275645399 execution time: 1:14:36
- INFO, Package successfully downloaded, JID_684275645399 execution time: 1:14:52
- INFO, Package successfully downloaded, JID_684275645399 execution time: 1:15:07
- INFO, Package successfully downloaded, JID_684275645399 execution time: 1:15:23
- INFO, Package successfully downloaded, JID_684275645399 execution time: 1:15:39
- INFO, Package successfully downloaded, JID_684275645399 execution time: 1:15:54
- INFO, Package successfully downloaded, JID_684275645399 execution time: 1:16:10
- INFO, Package successfully downloaded, JID_684275645399 execution time: 1:16:25
- INFO, Package successfully downloaded, JID_684275645399 execution time: 1:16:41
- INFO, Package successfully downloaded, JID_684275645399 execution time: 1:16:56
- INFO, Package successfully downloaded, JID_684275645399 execution time: 1:17:12
- INFO, Package successfully downloaded, JID_684275645399 execution time: 1:17:28
- INFO, Package successfully downloaded, JID_684275645399 execution time: 1:17:44
- INFO, Package successfully downloaded, JID_684275645399 execution time: 1:17:59
- INFO, Package successfully downloaded, JID_684275645399 execution time: 1:18:15
- INFO, Package successfully downloaded, JID_684275645399 execution time: 1:18:30
- INFO, Package successfully downloaded, JID_684275645399 execution time: 1:18:46
- INFO, Package successfully downloaded, JID_684275645399 execution time: 1:19:01
- INFO, Package successfully downloaded, JID_684275645399 execution time: 1:19:17
- INFO, Package successfully downloaded, JID_684275645399 execution time: 1:19:33
- INFO, Package successfully downloaded, JID_684275645399 execution time: 1:19:48
- INFO, Package successfully downloaded, JID_684275645399 execution time: 1:20:04
- INFO, Package successfully downloaded, JID_684275645399 execution time: 1:20:20
- INFO, Package successfully downloaded, JID_684275645399 execution time: 1:20:35
- INFO, Package successfully downloaded, JID_684275645399 execution time: 1:20:51
- INFO, Package successfully downloaded, JID_684275645399 execution time: 1:21:06
- INFO, Package successfully downloaded, JID_684275645399 execution time: 1:21:22
- INFO, Package successfully downloaded, JID_684275645399 execution time: 1:21:37
- INFO, Package successfully downloaded, JID_684275645399 execution time: 1:21:53
- INFO, Package successfully downloaded, JID_684275645399 execution time: 1:22:09
- INFO, Package successfully downloaded, JID_684275645399 execution time: 1:22:24
- INFO, Package successfully downloaded, JID_684275645399 execution time: 1:22:40
- INFO, Package successfully downloaded, JID_684275645399 execution time: 1:22:56
- INFO, Package successfully downloaded, JID_684275645399 execution time: 1:23:12
- INFO, Package successfully downloaded, JID_684275645399 execution time: 1:23:27
- INFO, Package successfully downloaded, JID_684275645399 execution time: 1:23:43
- INFO, Package successfully downloaded, JID_684275645399 execution time: 1:23:59
- INFO, Package successfully downloaded, JID_684275645399 execution time: 1:24:16
- INFO, Package successfully downloaded, JID_684275645399 execution time: 1:24:32
- INFO, Package successfully downloaded, JID_684275645399 execution time: 1:24:48
- INFO, Package successfully downloaded, JID_684275645399 execution time: 1:25:03
- INFO, Package successfully downloaded, JID_684275645399 execution time: 1:25:19
- INFO, Package successfully downloaded, JID_684275645399 execution time: 1:25:34
- INFO, Package successfully downloaded, JID_684275645399 execution time: 1:25:50
- INFO, Package successfully downloaded, JID_684275645399 execution time: 1:26:06
- INFO, Package successfully downloaded, JID_684275645399 execution time: 1:26:21
- INFO, Package successfully downloaded, JID_684275645399 execution time: 1:26:37
- INFO, Package successfully downloaded, JID_684275645399 execution time: 1:26:53
- INFO, Package successfully downloaded, JID_684275645399 execution time: 1:27:09
- INFO, Package successfully downloaded, JID_684275645399 execution time: 1:27:24
- INFO, Package successfully downloaded, JID_684275645399 execution time: 1:27:40
- INFO, Package successfully downloaded, JID_684275645399 execution time: 1:27:56
- INFO, Package successfully downloaded, JID_684275645399 execution time: 1:28:12
- INFO, Package successfully downloaded, JID_684275645399 execution time: 1:28:28
- INFO, Package successfully downloaded, JID_684275645399 execution time: 1:28:43
- INFO, Package successfully downloaded, JID_684275645399 execution time: 1:28:59
- INFO, Package successfully downloaded, JID_684275645399 execution time: 1:29:14
- INFO, Package successfully downloaded, JID_684275645399 execution time: 1:29:30
- INFO, Package successfully downloaded, JID_684275645399 execution time: 1:29:46

- FAIL: Timeout of 1h30 has been reached, script stopped

in iDRAC GUI i see: JID_684275645399 : Repository Update Downloaded JID_684278969328 : Firmware Update: BIOS Running(10%)

i have reduce the timeout from 2:00 to 1:30 :-)

texroemer commented 1 year ago

Hi @MarcPro92

There is a known bug in iDRAC code where the BIOS update job will intermittently get stuck in running state and not get marked completed successfully. Now when a job is stuck in running state, it will not allow you to create another BIOS update job so you will either have to wait for iDRAC internal timeout of 6 hours to elapse which will mark the job as failed for timeout or you can clear the job queue using force option from either RACADM or Redfish.

See script example below to clear the job queue using the force option. Also for the iDRAC bug, this will be fixed in 6.10.00 release which is scheduled to be posted next month.

C:\Python310>python DeleteJobQueueREDFISH.py -ip 192.168.0.120 -u root -p calvin --clear-restart

- PASS: DeleteJobQueue action passed to clear the job queue and restart Lifecycle Controller services, status code 200 returned

- INFO, Lifecycle Controller services restarted. Script will loop checking the status of Lifecycle Controller until Ready state
- INFO, Lifecycle Controller services not in ready state, polling again
- INFO, Lifecycle Controller services not in ready state, polling again
- INFO, Lifecycle Controller services not in ready state, polling again

- PASS, Lifecycle Controller services are in ready state