garethgeorge / backrest

Backrest is a web UI and orchestrator for restic backup.
GNU General Public License v3.0
1.71k stars 49 forks source link

Recovery Attempt Configuration #525

Open lookup82 opened 1 month ago

lookup82 commented 1 month ago

I'm experiencing an issue where the backup recovery attempt fails at boot because my PC takes some time to establish a network connection. While I appreciate the recovery feature, it would be helpful to have a configuration option to modify the recovery behavior or to implement a timeout before the recovery attempt.

Thank you!

garethgeorge commented 1 month ago

Common issue— there probably needs to be a cookbook for this in the backrest docs :)

For now a good work around is to add a command hook on CONDITION_BACKUP_START which runs a health check e.g. checks that the network is available with curl I.e. just “curl google.com” will do this.

Set the error behavior to on error retry with exponential backoff, which will keep the task waiting until the hook succeeds.

Edit: I use the following --

image
lookup82 commented 1 month ago

Thank you @garethgeorge ! In Windows, I tried this PowerShell script, but strangely the exit code was not caught correctly:

$pingResult = Test-Connection -ComputerName google.com -Count 1 -Quiet

if ($pingResult) {
    Write-Output "Internet connection is up"
    exit 0
} else {
    Write-Output "Internet connection is down"
    exit 1
}

I ended up making a simpler version that just waits for 5 minutes: Start-Sleep -Seconds 300

garethgeorge commented 2 weeks ago

Hmmm, that's pretty odd re: exit code support on windows -- there's definitely some room for how windows scripts are handled today however. Just to check, it wasn't being masked by the retry behavior? e.g. if you set the error behavior to something simpler like ON_ERROR_FATAL, you still didn't see it captured?

Regardless, glad you were able to work around with the sleep. May steal that idea for a simple edit to my workflows too -- I've seen the same issue infrequently on my machine :)

garethgeorge commented 2 weeks ago

Just tested -- on my Windows install actually, I am seeing an exit status captured from image