search

kmwoley / restic-windows-backup

Powershell scripts to run Restic backups on Windows
MIT License
321 stars 67 forks source link

rclone config with hetzner storage box append only #95

Open demlak opened 6 months ago

demlak commented 6 months ago

Hey there.. we are using the script with restic-rest-server since a year.. everything works good..

now i want to change the way the script uploads data..

we use a hetzner storage box, which is able to listen via ssh and that could be forced to use restic in this way: https://forum.restic.net/t/hetzner-storage-box-supports-rclone-serve-restic-stdio/5943/6

on my linux clients, everything went fine with the new way.

since i am not a windows pro, i don't understand exactly how to change the scripts for using it like this: restic -o rclone.program='ssh -p23 uXXXXX@uXXXXX.your-storagebox.de forced-command' -r rclone: init

i was following the instructions for installing ssh key.. like described in the README.md.. and checking via .\PsExec.exe -s -i powershell.exe + ssh user@server was working

could you explain, what exactly i have to update in config.ps1 and the other files, to use it in that way via the task scheduler?

thx a lot

EDIT: Workaround here: https://github.com/kmwoley/restic-windows-backup/issues/95#issuecomment-1986989063

SeeJayEmm commented 6 months ago

I think you need to do the following:

In config.ps1 set the $AdditionalBackupParameters to include your -o parameter. In secrets.ps1 set the RESTIC_REPOSITORY env variable to "rclone:" and set the password to whatever your repo password is.

This is all guessing since I've never tried it.

demlak commented 6 months ago

thx a lot.. your hint leeds me to update my installation.. but still problems:

following works perfect: .\restic.exe -o rclone.program='ssh -p 23 uxxxx@uxxxx-sub3.your-storagebox.de forced-command' -r rclone: check

Since my installation is old.. i just updated: backup.ps1, install.ps1, and windows.exclude. I also checked config for changes between the versions.. but $AdditionalBackupParameters seems to be the only new variable

This in config.ps1:

$AdditionalBackupParameters = @("--exclude-if-present", ".nobackup", "-o", "rclone.program='ssh -p 23 uxxxx@uxxxx.your-storagebox.de forced-command'")

And this in secrets.ps1:

$Env:RESTIC_REPOSITORY='rclone: '

Results to this:

PS C:\restic> . .\config.ps1; . .\secrets.ps1; & $ResticExe check

using temporary cache in C:\WINDOWS\TEMP\restic-check-cache-1769111303
rclone: 2024/03/09 08:48:36 NOTICE: Config file "C:\\WINDOWS\\system32\\config\\systemprofile\\AppData\\Roaming\\rclone\\rclone.conf" not found - using defaults
Fatal: unable to open config file: <config/> does not exist
Is there a repository at the following location?
rclone:

as far as i understand the sourcecode of backup.ps1, the $AdditionalBackupParameters is only used on backup, so i also tried this:

PS C:\restic> . .\config.ps1; . .\secrets.ps1; & $ResticExe $AdditionalBackupParameters check

using temporary cache in C:\WINDOWS\TEMP\restic-check-cache-1624378344
Fatal: unable to open repository at rclone: : cmd.Start: exec: "ssh -p 23 uxxxx@uxxxx.your-storagebox.de forced-command": executable file not found in %PATH%

There seems to be something wrong with my syntax

SeeJayEmm commented 6 months ago

Do you have OpenSSH installed on your Windows machine? If you try to SSH from the command line does it work?

demlak commented 6 months ago

Hey.. ssh from commandline works.. like described in the first part of my last post..

finaly it seems i found the problem(s)..

it is about the brackets.. i changed:

$AdditionalBackupParameters = @("-o", "rclone.program='ssh -p 23 uxxxx@uxxxx.your-storagebox.de forced-command'")

to

$AdditionalBackupParameters = @("-o", 'rclone.program="ssh -p 23 uxxxx@uxxxx.your-storagebox.de forced-command"')

and now, this works:

. .\config.ps1; . .\secrets.ps1; & $ResticExe $AdditionalBackupParameters check

but well.. as stated in my last posting, the variable $AdditionalBackupParameters is used only on backup-command.. so everything else in the backup-script is not working.. for example says log after starting task in task-scheduler:

restic.exe : rclone: 2024/03/09 21:50:13 NOTICE: Config file 
"C:\\WINDOWS\\system32\\config\\systemprofile\\AppData\\Roaming\\rclone\\rclone.conf" not found - using defaults
In C:\restic\backup.ps1:82 Zeichen:14
+     $locks = & $ResticExe list locks --no-lock -q 3>&1 2>> $ErrorLog
+              ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (rclone: 2024/03... using defaults:String) [], RemoteException
    + FullyQualifiedErrorId : NativeCommandError

Fatal: unable to open config file: <config/> does not exist
Is there a repository at the following location?
rclone:

(But new snapshot was generated)

As far as i understand, i need to change the config.ps1 in that way, that $ResticExe is has the config-parameters included. kinda like

$rcloneconnect = @("-o", 'rclone.program="ssh -p 23 uxxxx@uxxxx.your-storagebox.de forced-command"')
$ResticExe = "$ResticExe" + $rcloneconnect

(i don't know about powershell-syntax) or i have to change backup.ps1 for adding the config-parameters at every use of $ResticExe? It seems, default it is not possible to use restic-windows-backup in the way i want to use it.

demlak commented 6 months ago

Ok.. finaly i got it working.. here is a workaround:

  1. Download and install/copy rclone.exe to c:\windows\system32. So it is reachable for SYSTEM-user
  2. Configure and test your SSH, like described in the README.md at Topic "Backup over SFTP".
  3. Add a new Variable to config.ps1 and replace the uxxxx with your credentials: 
    $RcloneConnect = @("-o", 'rclone.program="ssh -p 23 uxxxx@uxxxx.your-storagebox.de forced-command"')
  4. Edit backup.ps1: After every $ResticExe add an $RcloneConnect. For Example Line 82: before: 
    $locks = & $ResticExe list locks --no-lock -q 3>&1 2>> $ErrorLog

    after:

    $locks = & $ResticExe $RcloneConnect list locks --no-lock -q 3>&1 2>> $ErrorLog
  5. In secrets.ps1 use rclone: as your repo

i won't call this a "solution" but a workaround, because we have to alter the backup.ps1. Ofcourse those changes are gone, after next update.

@kmwoley maybe you want to reflect this kind of solution/way to use restic in your great powershell-scripts? this is a great way to use restic append only via ssh, without restic rest-server

my skillz in powershell and git are too low to provide a good solution/pull-request, sorry =(

kmwoley commented 6 months ago

If someone could work up a pull request, I'll happily review and integrate it. I'm due for my annual updates to the script. :D

On Sat, Mar 9, 2024 at 1:48 PM demlak @.***> wrote:

Ok.. here is a workaround.

  1. Configure and test your SSH, like described in the README.md at Topic "Backup over SFTP".
  2. Add a new Variable to config.ps1 and replace the uxxxx with your credentials:

$RcloneConnect = @("-o", 'rclone.program="ssh -p 23 @.*** forced-command"')

  1. Edit backup.ps1: After every $ResticExe add an $RcloneConnect. For Example Line 82: before:

$locks = & $ResticExe list locks --no-lock -q 3>&1 2>> $ErrorLog

after:

$locks = & $ResticExe $RcloneConnect list locks --no-lock -q 3>&1 2>> $ErrorLog

In secrets.ps1 use rclone: as your repo

i won't call this a "solution" but a workaround, because we have to alter the backup.ps1. Ofcourse those changes are gone, after next update.

@kmwoley https://github.com/kmwoley maybe you want to reflect this kind of solution/way to use restic in your great powershell-scripts? this is a great way to use restic append only via ssh, without restic rest-server

— Reply to this email directly, view it on GitHub https://github.com/kmwoley/restic-windows-backup/issues/95#issuecomment-1986989063, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABOWG6BOOQLABAOCBNCCMULYXN7RBAVCNFSM6AAAAABENCCIAOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSOBWHE4DSMBWGM . You are receiving this because you were mentioned.Message ID: @.***>

woelfisch commented 6 months ago

If someone could work up a pull request, I'll happily review and integrate it. I'm due for my annual updates to the script. :D

There you go: https://github.com/kmwoley/restic-windows-backup/pull/96

Three additional options