overlayfs hack

[axet]

Hello!

I want to share an idea how you can reduce memory footprint from using profile-sync-daemon. Currently it requires to put whole browser profile into tmpfs disk which equivalent full profiles size. Using overlayfs you can reduce profile size to the changes size only.

I wrote simple script which does the trick for firefox:

• https://gitlab.com/axet/homebin/-/blob/debian/homebin.d/firefox

[Konfekt]

Hello,

This has become https://gitlab.com/axet/homebin/-/blob/e20238e97b18c9b9258170dbf1cb3bae43b3ab63/homebin.d/firefox-tmpfs . How does this compare to the use of overlayfs by psd?

[axet]

As I said above, I think, using fuse-overlayfs saves memory, since it does not keep entier profile in memory. Only profiles changes.

If profile is 300M, changes will be 50-100M. Saves RAM.

PSD copy entire profile into memory (300M), which can be not optimal (300Mvs100M).

[Konfekt]

Maybe https://wiki.archlinux.org/title/Profile-sync-daemon#Overlayfs_mode is relevant

[axet]

overlayfs requires root, fuse-overlayfs does not.

[Konfekt]

Okay. You initially described the advantage of overlayfs that it

... can reduce profile size to the changes size only.

which psd also achieves:

The magic is in how the overlay mount only writes out data that has changed rather than the entire profile.

Instead, the advantage of fuse-overlayfs is not requiring root. Then I suspect a pull request would be most welcome (though @graysky2 knows best).

[graysky2]

Worth looking into...

[axet]

Keep in mind. It fuse-overlayfs may have kernel freeze related issue:

https://github.com/containers/fuse-overlayfs/issues/386

[graysky2]

That is not good. PSD is rock solid for me and 100s of others as it stands.

[graysky2]

I created an initial attempt at using fuse-overlayfs to manage mount/umounts. It works without systemd triggering the sync. I need more time to debug why starting with systemd does not work. Please have a look. If you have a suggestion why starting it with systemd fails, please send a PR or discuss here.

https://github.com/graysky2/profile-sync-daemon/tree/fuse-overlayfs

BACKUP your browser profiles first!

Why would systemd calling the script in user mode cause fuse-overlayfs to fail silently? There is no evidence that the mount is setup (ie grep mount|grep over returns nothing). Bug?

https://github.com/containers/fuse-overlayfs/issues/388

[graysky2]

Ah! Seems as though RemainAfterExit=yes needs to be set...

@axet - Will you please try the latest commit from my https://github.com/graysky2/profile-sync-daemon/tree/fuse-overlayfs branch? Does it behave as you expect? I too am running it and look for inconsistencies in behavior.

I would like to avoid the need for root to be in the mix at all, https://github.com/graysky2/profile-sync-daemon/issues/286 and on the surface, this implementation seems to achieve that.

For Arch Linux users, you can install this dev version with this AUR package.

[axet]

Can't test it right now. I think I will stuck with my own firefox-tmpfs. I love it. It much lighter, has no systemd dependency. People will appreciate fuse-overlayfs I'm sure of it.

[graysky2]

I am seeing slow downs using fuse-overlayfs vs kernel mount. Others? https://github.com/containers/fuse-overlayfs/issues/390

[graysky2]

Closing for now due to enormous speed hit using FUSE introduces.