Commifreak / unraid-appdata.backup

UNRAID AppData backup plugin
19 stars 1 forks source link

Is it possible to honour the docker autostart delay? #31

Open lonzodelorana opened 1 month ago

lonzodelorana commented 1 month ago

Updating docker containers will stop/update/start the container immediately and then proceed to stop/update/start the next, however on older or low power hardware this can end up with the containers updating but failing to start.

The Unraid docker implementation has an autostart delay built in, used to slow down starting of containers after a reboot for just this issue, is it possible to honour this delay before starting the next update procedure, so containers can update without manual intervention on this sort of setup please?

Commifreak commented 1 month ago

on older or low power hardware this can end up with the containers updating but failing to start.

How? These actions happens in order. I dont know why this should fail - and with what error message? The container delay is being respected at container start - but AFTER it started.

If there are race conditions, it must be an issue with docker itself. Please provide a error message (syslog?)

I could add a static 3 second wait after internal docker update. Should hurt nobody.

lonzodelorana commented 1 month ago

Actually, it looks like I made a huge assumption over why the containers were not starting and it looks like the docker autostart delays are indeed honoured by appdata.backup. My sincere apologies.

I believe what is happening is a container which is being used as the networking for other containers is being updated and it looks like as the dependant containers are recreated due to the networking change, appdata.backup can't find the containers to start them, log snippet below.

[24.06.2024 03:02:55][ℹ️][network-container] Should NOT backup external volumes, sanitizing them... [24.06.2024 03:02:55][ℹ️][network-container] Calculated volumes to back up: /mnt/user/appdata/network-container [24.06.2024 03:02:55][ℹ️][network-container] Backing up network-container... [24.06.2024 03:02:55][ℹ️][network-container] Backup created without issues [24.06.2024 03:02:55][ℹ️][network-container] Verifying backup... [24.06.2024 03:02:55][ℹ️][network-container] Installing planned update for network-container... [24.06.2024 03:03:04][ℹ️][Main] Set containers to previous state [24.06.2024 03:03:04][ℹ️][network-container] Starting network-container... (try #1) done! [24.06.2024 03:03:08][ℹ️][container1] Starting container1... (try #1) Container did not started! - Code: No such container [24.06.2024 03:03:13][ℹ️][container1] Starting container1... (try #2) Container did not started! - Code: No such container [24.06.2024 03:03:18][ℹ️][container1] Starting container1... (try #3) Container did not started! - Code: No such container [24.06.2024 03:03:18][❌][container1] Container did not started after multiple tries, skipping. [24.06.2024 03:03:22][ℹ️][container1] Starting container1... (try #1) Container did not started! - Code: No such container [24.06.2024 03:03:27][ℹ️][container1] Starting container1... (try #2) Container did not started! - Code: No such container [24.06.2024 03:03:32][ℹ️][container1] Starting container1... (try #3) Container did not started! - Code: No such container [24.06.2024 03:03:32][❌][container1] Container did not started after multiple tries, skipping. [24.06.2024 03:03:35][ℹ️][container1] The container has a delay set, waiting 30 seconds before carrying on [24.06.2024 03:04:05][ℹ️][container1] Starting container1... (try #1) Container did not started! - Code: No such container [24.06.2024 03:04:10][ℹ️][container1] Starting container1... (try #2) Container did not started! - Code: No such container [24.06.2024 03:04:15][ℹ️][container1] Starting container1... (try #3) Container did not started! - Code: No such container [24.06.2024 03:04:15][❌][container1] Container did not started after multiple tries, skipping. [24.06.2024 03:04:17][ℹ️][container1] The container has a delay set, waiting 90 seconds before carrying on [24.06.2024 03:05:47][ℹ️][container1] Starting container1... (try #1) Container did not started! - Code: No such container [24.06.2024 03:05:52][ℹ️][container1] Starting container1... (try #2) Container did not started! - Code: No such container [24.06.2024 03:05:57][ℹ️][container1] Starting container1... (try #3) Container did not started! - Code: No such container [24.06.2024 03:05:57][❌][container1] Container did not started after multiple tries, skipping. [24.06.2024 03:06:00][ℹ️][container1] The container has a delay set, waiting 60 seconds before carrying on

When checking back in the unraid GUI in the morning, all containers are present, just not started

Again, sorry for the assumptions!

Commifreak commented 1 month ago

Thats weird. But the names do not change, doesnt it?

lonzodelorana commented 1 month ago

nope, the names are the same, rebuild only takes 30 -60 seconds or so for all of the dependant containers.

The container ID does change though for all of the dependant containers

Commifreak commented 1 month ago

The container ID is not being used for those actions. Please submit a debug log and share its ID. There are some internal debugs in that case.

lonzodelorana commented 1 month ago

Hi,

Sorry for the delay, the most recent debug log didn't have the issue, but it reoccurred last night, so I've submitted the log and the ID is

24d6ffd8-670d-4af5-a111-ae1b129dccfb

Commifreak commented 1 month ago

I need to adjust my debug things inside the plugin to get a deeper look. Stay tuned.