search

bit-team / backintime

Back In Time - An easy-to-use backup tool for GNU/Linux using rsync in the back
https://backintime.readthedocs.io
GNU General Public License v2.0
2.06k stars 203 forks source link

BackInTime Slow For 1+ backups. #1671

Open hickscorp opened 7 months ago

hickscorp commented 7 months ago

First backup is fast - usually between 130 and 300 MB/s. Second backup and subsequent stays around 12MB/s.

There also seems to be a problem with how hardlinks are handled - often times, BackInTime reports "No space left on device" while only very few changes have been made to the original files.

A quick question - as I noticed the 2nd issue maybe popping since more recently - I use BackInTime on two computers which are "mirrored" in terms of storage. It seems to me that because BackInTime is used on both computers using the same external storage,, it might not be able to hardlink properly, as the original files are from two different sources... Could it be the case?

emtiu commented 3 months ago

Hi there, sorry for reacting so late and thanks for your report.

You are most likely experiencing our oldest-and-worst bugs #988 and #994. We are working on fixing them, but it's slow going. The reason behind these problems is that rsync fails to handle identical files with different permissions. This is probably also the reason why unnecessary copies are made when you write to the same backup location from two computers: because permissions (or, more likely, file ownership) are different between the two source machines.

A workaround that often helps is to pass --no-perms --no-group --no-owner to rsync, which you can do in the expert settings of your backup profile.

Please report back if this helps, if you are still using BackInTime.

hickscorp commented 3 months ago

Hi @emtiu and thanks a lot for your answer. I apologize as I couldn't answer before. I will be trying to use the --no-perms --no-group --no-owner options. Just so we're on the same page - I didn't find any UI option to enable them, so I'm adding them to the command line additional options, is that correct (See screenshot)?

Thanks again! image

emtiu commented 3 months ago

It's good that you ask! :)

Yes, that is exactly the way I've been applying this workaround for the last few years:

Screenshot_20240720_232911

hickscorp commented 3 months ago

Thanks @emtiu !

A quick question. I can see that your screenshot has Preserve ACLs unticked. Do you think that having it ticked could give contrary directives to RSync along with the manual options we are using now?

A suggestion would be to allow BiT to have popover little bubbles which could give the command line flags behind the tick box? So that one more advanced user could understand better the combination of all "UI" flags and know if the "custom" ones are in contradiction?

Anyway - minor change :) Thanks for taking the time and I will report back.

Keep up the good work! I love BiT.

emtiu commented 3 months ago

A quick question. I can see that your screenshot has Preserve ACLs unticked. Do you think that having it ticked could give contrary directives to RSync along with the manual options we are using now?

Possibly, yes – but we haven't tested the effect of ACLs on bugs #988 and #994 as far as I know. Unless you have a good reason to preserve ACLs, it's probably best to untick the option.

A suggestion would be to allow BiT to have popover little bubbles which could give the command line flags behind the tick box? So that one more advanced user could understand better the combination of all "UI" flags and know if the "custom" ones are in contradiction?

That's a good idea. It gets even more complex, because rsync has a rule to allow the last(?) option to override any previous(?) option if they are in contradiction. So the effect of any specific rsync command also depends on the order in which backintime assembles the different options into a command.

We'll be dealing a lot more with this once we finally get to fixing #988 and #994, so thank you for your input on this :)