search

spersson / Kup

A backup scheduler for KDE's Plasma desktop
https://www.linux-apps.com/p/1127689/
131 stars 14 forks source link

Slow backup transfer #69

Open spoontex opened 5 years ago

spoontex commented 5 years ago

I've just installed Kup Backup in Kde Neon. but I only have speeds about 20-40Mb/s to other computer via NFS server

If I just copy files with dolhpin I get speeds about 90-100 Mb/s.

Any idea?

Thanks.

spersson commented 5 years ago

I assume this is with versioned (bup) backups?

https://groups.google.com/forum/#!searchin/bup-list/NFS%7Csort:date/bup-list/qzsu78JPQzQ/M2b4MUbLCAAJ

Those messages don't prove it, but other people are also thinking that putting the bupindex and git index files on NFS will be slow. It will access those files to figure out what needs to be copied. I have done it like that in kup in order to avoid storing local files. It's a tradeoff. The alternative is to call "bup save -r" which will keep local index files but store the backup contents to a different path. I didn't like having to create a git repo locally just to store those cache files there. If backing up to local USB drive then there is not much effect on speed, and that is the main targeted usage scenario for kup. At least you're only copying what is necessary for incremental backup.

I am considering adding better support for samba shares. Perhaps in connection with that I will re-evaluate this decision. Again.

Actually, since this is not easy for me to test... Perhaps you could test for me. It would be insteresting to see how much speed difference there actually is. In that case I think it's best to measure time to complete "bup index" and "bup save" commands in two scenarios:

  1. Run "bup init" to create a local git repo for cache files to be stored in your home folder. Then run "time bup index" with the files you are backing up. Then "time bup save -r /path/to/dest".
  2. Do the same as what Kup does: "time bup -d /path/to/dest index" followed by "bup -d /path/to/dest save"
tmtravolta commented 3 years ago

I have kup-backup 0.8.0 on Ubuntu 21.04 with following setup: rsync-based kup backup, NFS storage. Copying files manually with rsync gives transfer speed 10x faster than letting kup handle the transfer. So it's not only bup that might make NFS storage slow?

spersson commented 3 years ago

@tmt-xx Thanks for that information! Can you look at the log file from Kup, the rsync parameters are recorded there. Then compare with how you run rsync, see if you can figure out which parameter is making the difference?