search

wolfv6 / rclone_jobber

An rclone script to perform backups
263 stars 51 forks source link

User error with setting up a remote host? #22

Open DarrenPIngram opened 2 years ago

DarrenPIngram commented 2 years ago

May I possibly get a nudge in the right direction? I'm having problems getting the script to run using a remote rclone connection.

I have the following elements in the script:

source="/home/x/storage"            #the directory to back up (without a trailing slash)
dest="${dropbox_crypt_WS1bkp}:"              #the directory to back up to (without a trailing slash or "last_snapshot") destination=$dest/last_snapshot

With the objective to backup /home/x/storage/* (all from this point and downwards) to Dropbox location (config made in rclone) dropbox_crypt_WS1bkp. It passes the following command

Back up in progress 2021-12-08_11:52:12 WS1bkp_storage
rclone sync /home/x/storage :/last_snapshot --backup-dir=:/archive/2021/2021-12-08_11:52:12 --log-file=/home/x/scripts/rclone_jobber/WS1backup.log --log-level=INFO
ERROR: WS1bkp_storage failed.  rclone exit_code=1

but I am not getting sense into my head. I've tried adding the source without the dollar sign, without the curly brackets as well (different parts of the script/docs give the impression that they are left out, adding to my confusion).

Any guidance and pointers are welcome.

NoLooseEnds commented 2 years ago

I think you are confusing some variables?

In your script put this variable the top, but below /bin/bash line: remote=<your rclone remote name here from rclone config file>

Then you write (that pulls the name from the variable above): dest="${remote}:"

DarrenPIngram commented 2 years ago

Thank you. I think I got distracted and some of the examples (from memory) didn't show that variable.

I've left it running doing "something" but no progress is shown (nor any errors). Away now until Saturday or Sunday with only my tablet for company. I can tail the log at least!

Appreciate the help in getting me on with this interesting script!

Darren

On Wed, 8 Dec 2021 at 15:00, NoLooseEnds @.***> wrote:

I think you are confusing some variables?

In your script put at the top, but below /bin/bash line: remote=

Then you write: dest="${remote}:"

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/wolfv6/rclone_jobber/issues/22#issuecomment-988794205, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABV5MAEP4WI6SWZKSPEGQX3UP5JFTANCNFSM5JTWRCXQ . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

NoLooseEnds commented 2 years ago

Happy to help. :)

You'll need to follow the log to see the progress. How much info you get in the log can you set using the --loglevel flag in the rclone command.

I have it hooked up to a slack webhook, so I get notified there when it succeeds or fails. Also have a monitoring service that triggers and sends me an email if it does not complete within a given timeframe.

DarrenPIngram commented 2 years ago

I've set a cronjob http service and will see how that works. The backup is still running which is good (its only about 32Tb). May I ask, whilst I have your attention, maybe a few other questions as I finalise the first job of many?

Is there any way (planned other backup) to have several sources linked into one backup job? I don't see anything in the documentation or examples in any case? Even if I make a file of paths and use source=file name?

NoLooseEnds commented 2 years ago

32 TB is quite a bit, and would probably take quite a while.

You should look more into the basics of rclone. I.e filter_rules, you could add multiple sources there. Or you could make symlinks to one directory if that is easier.

This script is basicly just automating the backup part, while you should look into rclone for all the rest.

DarrenPIngram commented 2 years ago

Ah, thanks. Yes 32Tb is probably large, but it was a dataset on the server that needed to be backed up anyway...

I will look deeper into the script. I am not the world's best scripter and am worried about breaking stuff, especially important stuff. I will look how to make the argument within the script.

d

On Sat, 11 Dec 2021 at 18:14, NoLooseEnds @.***> wrote:

32 TB is quite a bit, and would probably take quite a while.

You should look more into the basics of rclone. I.e filter_rules, you could add multiple sources there. Or you could make symlinks to one directory if that is easier.

This script is basicly just automating the backup part, while you should look into rclone for all the rest.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/wolfv6/rclone_jobber/issues/22#issuecomment-991698217, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABV5MAG3BUNJNBFDNKBAB2TUQN2HDANCNFSM5JTWRCXQ . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.