elemental-lf / benji

Benji Backup: A block based deduplicating backup software for Ceph RBD images, iSCSI targets, image files and block devices
https://benji-backup.me
Other
136 stars 43 forks source link

Benji Read Performance #57

Open adambmedent opened 5 years ago

adambmedent commented 5 years ago

We have been doing some more testing with Benji and seem to be running into a odd performance issue.

We are pointing Benji at a logical volume. It reads the logical volume at around 200MB/s. I know my storage has considerably more performance. For example, if I have 2 benji backups running at the same time to the same storage, I can hit 400MB/s. Is there anything I can do so that a single backups can achieve those same speeds?

I have tried messing with the simultaneousReads and simultaneousWrites options but they don't seem to have an effect. With those set to 15 I hit 200MB/s, set to 30 and I still hit 200MB/s.

The storage has almost no latency and is under very little load.

elemental-lf commented 5 years ago

It's hard to say without some profiling to find the bottleneck. Benji has only seen limited benchmarking and performance tuning and my use case has relatively small volumes and the change rate is small too, so it isn't a priority for me at the moment. If someone with experience in Python profiling would look into this, I'd be willing to help interpret the results and work on a solution. You could also try to look at the debugging output to see if you can deduce something from it. The reading and writing of blocks should output some time measurements.

adambmedent commented 5 years ago

Ok, ill see what I can figure out, but this is definitely a bit over my head.

adambmedent commented 4 years ago

On a side note, is it possible to use a "hints" file with LVM, similar to Ceph/RBD? I see it mentioned in the documentation as possible, just not sure how it would work? I don't think I can point rbd-diff to a lvm volume?