search

mholt / photobak

Back up your content from Google Photos - DEPRECATED: use Timeliner
https://github.com/mholt/timeliner
307 stars 29 forks source link

Add an option to skip checking of items that already exist in the database #10

Closed KonishchevDmitry closed 7 years ago

KonishchevDmitry commented 7 years ago

At this time photobak always checks all items stored on filesystem: it checks that they are exist and recalculates their checksums which leads to high CPU and disk I/O usage and too excessive in the case when you run photobak every day in append-only mode when there is no reason for files to be corrupted or missing. So I added this option to be able to run photobak at least most of the times without these checks.

The result on my 11GB repository:

$ time photobak -v -concurrency 2 ...
real  8m34.654s
user  2m34.612s
sys   0m9.788s

$ time photobak -v -concurrency 2 -skiprepochecks ...
real  3m45.256s
user  0m5.716s
sys   0m0.848s

I/O utilization in atop is even more representative.

mholt commented 7 years ago

Thanks! I'm gonna compile and use this now. Really appreciate it!