Questions on restoring data

[mphelpsmd]

I just did a test with backing up about 1 GB of test data both locally and to my Backblaze B2 account, and have some questions on the restore operation.

Here are the steps I took to initialize the repository and storage locations:

[root@ares duplicacy_test_data]# pwd
/srv/fileserver/temp/duplicacy_test_data

[root@ares temp]# duplicacy init test_data /backups/local/encrypted_rotating_backup_1/test1/

(Changed preferences file to replace “default” with “rotating_backup_1”)

[root@ares duplicacy_test_data]# duplicacy backup -storage rotating_backup_1
Storage set to /backups/local/encrypted_rotating_backup_1/test1/
No previous backup found
Indexing /srv/fileserver/temp/duplicacy_test_data
Loaded 1 include/exclude pattern(s)
(lots of “Packed data_dir…” lines)

[root@ares duplicacy_test_data]# export DUPLICACY_BACKBLAZE_B2_TEST_B2_ID="xxx"
[root@ares duplicacy_test_data]# export DUPLICACY_BACKBLAZE_B2_TEST_B2_KEY="xxx"

[root@ares duplicacy_test_data]# duplicacy add backblaze_b2_test test_data b2://my-B2-backup-bucket
Enter Backblaze Account ID: xxx
Enter Backblaze Application Key: xxx
/srv/fileserver/temp/duplicacy_test_data will be backed up to b2://my-B2-backup-bucket with id test_data

[root@ares duplicacy_test_data]# duplicacy backup -storage backblaze_b2_test
Storage set to b2://my-B2-backup-bucket
Enter Backblaze Account ID: xxx
Enter Backblaze Application Key: xxx
No previous backup found
Indexing /srv/fileserver/temp/duplicacy_test_data
Loaded 1 include/exclude pattern(s)

To test a restore:

[root@ares duplicacy_test_data]# cd ..
[root@ares temp]# mkdir temp_removed_duplicacy_test_data
[root@ares temp]# cd duplicacy_test_data/
[root@ares duplicacy_test_data]# mv data_dir_* ../temp_removed_duplicacy_test_data/

[root@ares duplicacy_test_data]# duplicacy restore -r 2 backblaze_b2_test
Storage set to /backups/local/encrypted_rotating_backup_1/test1/
Loaded 1 include/exclude pattern(s)
Restoring /srv/fileserver/temp/duplicacy_test_data to revision 2
Restored /srv/fileserver/temp/duplicacy_test_data to revision 2
Total running time: 00:00:01

But this didn't work in restoring anything, although it gave no error. Adding the "-storage" parameter, however, seemed to do the trick:

[root@ares duplicacy_test_data]# duplicacy restore -r 2 -storage backblaze_b2_test
Storage set to b2://my-B2-backup-bucket
Loaded 1 include/exclude pattern(s)
Restoring /srv/fileserver/temp/duplicacy_test_data to revision 2
(Lots of “Downloaded data_dir…” lines)

So, the main questions about the restore operation:

1) It seems that the revision (-r) parameter is required. How do I have it use the most recent revision available? I had just created this backup and hadn't done much with it, so I just experimented with different "-r" numbers to determine the highest. I'm assuming there's a much more straightforward way, but I haven't come across it in the documentation.

2) Why did everything work fine with the "-storage backblaze_b2_test" option, but omitting "-storage" resulted in no error (but nothing accomplished)? Is this intended behavior? Why is the term "-storage" even necessary, as the current directory has the repository information, and I've specified the storage location on the command line? Or, if it is necessary, why doesn't omitting it give an error?

And a couple minor questions not related to the main topic (let me know if you'd prefer me to start a new thread):

1) Despite exporting DUPLICACY__B2ID and DUPLICACY_B2_KEY, (in upper case, as indicated in the wiki at https://github.com/gilbertchen/duplicacy/wiki/Managing-Passwords), duplicacy still asked me for the B2 ID and key. Why is this? 2) If the encryption (-e) option is not used, is the data still being encrypted when transmitting to/from Backblaze (but not on Backblaze's servers), or is everything sent via cleartext unless the -e option is used?

Thanks so much in advance,

Mike

[theincogtion]

I guess with the list command you will see the availabe revisions: https://github.com/gilbertchen/duplicacy/wiki/list

I am currently backing up on sftp and I will do a restore test afterwards.

The point with the -e option would be interessting for me too. If I got it right in case of a total data loss (also .duplicacy folder vanished), I have to use the init command again to recreate the connection to the storage correct? And afterwards running the restore. What if I forget the storage id? No restore possible anymore? edit: just saw that the storage id is stored in cleal text (not encrypted) on the backup device. so in case of data loss it can be found again.

[gboudreau]

1. Answered by theincogtion above; use duplicacy list to get a list of all the revisions available.
2. The syntax you used, without -storage, is valid. You tried to restore, from the default storage (Storage set to /backups/local/encrypted_rotating_backup_1/test1/) a file name backblaze_b2_test.
3. Use -background for duplicacy to use the env variables: duplicacy -background restore ...

   -background             read passwords, tokens, or keys only from keychain/keyring or env

4. Without -e, the data is not encrypted by Duplicacy. But since the storage type you are using requires HTTPS, your data is encrypted while traveling, and then decrypted, to be stored unencrypted, by the receiving server. Same would be true if you used SFTP. I am pretty sure all remote protocols supporter (except FTP) would travel encrypted.

[mphelpsmd]

Thanks so much for the responses.

duplicacy list did work for me for identifying the latest revision to restore. Perfect. Still, I think there should be some option for restoring the most recent revision (I think it should simply be the default if someone omits the -r parameter).

I wasn't aware of the -background option. I was under the impression that if there was an environment variable set, then it used it.

So it looks like I should back up the data to B2, and back up the .duplicacy directory to my 1password account, as that would ensure that all of the information needed to restore the files are located on cloud services.

Thanks again for the information,

Mike

[gilbertchen]

back up the .duplicacy directory to my 1password account

I think you only need to back up the .duplicacy/preferences file. All other files can be disposable.

[TowerBR]

I think you only need to back up the .duplicacy/preferences file. All other files can be disposable.

Exactly, that's what I do, and without the cache subfolder.

[mphelpsmd]

OK, I think that just backing up the preferencesfile should be adequate for most situations, since if you lose the source data and restore then it won't have files that would have been excluded from the patterns in the filtersfile. So subsequent backups should be OK.

I'd imagine, though, that the problem would become if you started to re-create files in, say, a temp directory that you'd excluded from the backups using a filtersfile. Now these files will be backed up to the storage location, as well. This wouldn't have happened if you had saved the entire .duplicacy directory and restored it in its entirety (including the filters file), right?