search

graysky2 / profile-sync-daemon

Symlinks and syncs browser profile dirs to RAM thus reducing HDD/SDD calls and speeding-up browsers.
https://wiki.archlinux.org/index.php/Profile-sync-daemon
Other
910 stars 88 forks source link

Use .tar for backup so loading to memory is faster #36

Closed protobits closed 11 years ago

protobits commented 11 years ago

Maybe this would be better than actually reading every file to tmpfs

graysky2 commented 11 years ago

Sorry, I don't understand the request... you want psd to tar the profile?

protobits commented 11 years ago

Hi, sorry, yes. Instead of storing the profile-backup as-is, store it as tar. Therefore, untaring to tmpfs would be much faster since it will be read on disk as a big chunk of sequential blocks (specially if it is defragmented and maybe even preloaded into disk cache). Also, tmpfs->disk would be faster I think.

On Mon, May 6, 2013 at 9:04 PM, graysky notifications@github.com wrote:

Sorry, I don't understand the request... you want psd to tar the profile?

— Reply to this email directly or view it on GitHubhttps://github.com/graysky2/profile-sync-daemon/issues/36#issuecomment-17515793 .

hasufell commented 11 years ago

taring will take some time too and every time you re-sync from tmpfs to disc you would have to tar again.

protobits commented 11 years ago

I think you are using rsync. I thought you simply copied everything.

In the case you make a .tar once (first time psd is run), extracting the tar to tmpfs will be much faster. Also, taring inside tmpfs and simply copying over this .tar to the .tar on the HDD will also be faster than writing a whole bunch of files. Maybe even GZIP compressing the .tar file will be faster to read, but the compression time may not be neglible in some cases.

On Tue, May 7, 2013 at 9:48 AM, Julian Ospald notifications@github.comwrote:

taring will take some time too and every time you re-sync from tmpfs to disc you would have to tar again.

— Reply to this email directly or view it on GitHubhttps://github.com/graysky2/profile-sync-daemon/issues/36#issuecomment-17539912 .

hasufell commented 11 years ago

I doubt that. You are copying from tmpfs TO disk. That does not speed up anything, because the disk is the bottleneck. For a large profile that might be 500mb up to 1GB copying for every resync. When you use rsync, it will be faster depending on how many files have changed. That's what rsync was designed for. If you add compression, taring time will increase and use cpu power.

protobits commented 11 years ago

Ok, feel free to close this then.

On Tue, May 7, 2013 at 11:40 AM, Julian Ospald notifications@github.comwrote:

I doubt that. You are copying from tmpfs TO disk. That does not speed up anything, because the disk is the bottleneck. For a large profile that might be 500mb up to 1GB copying for every resync. When you use rsync, it will be faster depending on how many files have changed. That's what rsync was designed for. If you add compression, taring time will increase and use cpu power.

— Reply to this email directly or view it on GitHubhttps://github.com/graysky2/profile-sync-daemon/issues/36#issuecomment-17546349 .

graysky2 commented 11 years ago

A major advantage of rsync is that it only syncs [resyncs] what changed since the last sync. Very efficient and very fast. Thanks for the interest.