search

kopia / kopia

Cross-platform backup tool for Windows, macOS & Linux with fast, incremental backups, client-side end-to-end encryption, compression and data deduplication. CLI and GUI included.
https://kopia.io
Apache License 2.0
7.36k stars 379 forks source link

Kopia stalls when uploading to Proton Drive with rclone #4015

Open whacka opened 1 month ago

whacka commented 1 month ago

Hi,

I'm currently running an initial snapshot using Kopia.

I'm a bit concerned with the performance, does anyone know if this is normal or a config issue?

The snapshot has been running for 3 days, the job seems to have slowed down over time.

The total size is about 100GB, the first 75GB uploaded within a few hours. Over the past 2 days the job has only uploaded a further 15GB.

My network doesn't seem to be having any issues. My repository is using rclone with Proton Drive.

The snapshot log for the past few days looks like the following (I've replace hashes etc. with ###):

16:10:55.509 keeping 2024-07-31 16:10:33.789317889 retention: [incomplete] pins: []
17:48:15.018 checkpointed root ###
17:48:15.069 deleting ###
17:48:15.069 deleting ###
17:48:15.069 keeping 2024-07-31 14:16:12.445115011 retention: [incomplete] pins: []
17:48:15.069 keeping 2024-07-31 14:44:03.136059422 retention: [incomplete] pins: []
17:48:15.069 keeping 2024-07-31 15:41:27.918713013 retention: [incomplete] pins: []
17:48:15.069 keeping 2024-07-31 16:10:33.789317889 retention: [incomplete] pins: []
17:48:15.069 keeping 2024-07-31 17:48:15.021791429 retention: [incomplete] pins: []
20:55:29.687 checkpointed root ###
21:58:37.671 deleting ###
21:58:37.671 deleting ###
21:58:37.671 deleting ###
21:58:37.671 keeping 2024-07-31 16:10:33.789317889 retention: [incomplete] pins: []
21:58:37.671 keeping 2024-07-31 17:48:15.021791429 retention: [incomplete] pins: []
21:58:37.671 keeping 2024-07-31 20:55:29.687515312 retention: [incomplete] pins: []
whacka commented 1 month ago

Hi, To follow up on this, I ended up testing the backup locally, this finished within 5 minutes. There were about 550 files created in the Proton Drive, and only about 300 locally. There were also a lot of items in Proton Drive's "trash", which might indicate the upload was repeatedly failing. After a bit of consideration I decided to pay for a different service and avoid rclone altogether, this backup ran quicker and finished normally. I'm happy with the result, but wanted to note that there seems to be an issue with Proton Drive and rclone.

tuxruffian commented 3 weeks ago

@whacka FWIW, I have been using Rustic with Proton Drive (Also via rclone) and it works perfectly. No GUI though.