Some questions - Githubissues

segator commented 6 years ago

I have some questions, can you limit max cache size? if the cache is full, do you delete older file accesed on the cache drive? If you umount and remount the cache persist? How mcachefs check if the cache is still in sync with the backend? (think about sshfs) you cache a file but modified from remote server.

Thanks!

segator commented 6 years ago

Other question, do you cache the entire file or only accessed blocks?

Doloops commented 6 years ago

Hi segator,

Yes, the cache is persistent upon umount/remount. There is no max cache size capability yet, but that's a planned short-term feature. The quota should be set on a per-directory or per-directory-hierarchy level, with least-recently-accessed deletion policy (mcachefs stores atime). Complexity here is how users can configure per-directory quotas...

Each file is cached entirely, so not by accessed blocks. This could be an upcoming feature, but has the downside of putting partial files in the local cache filesystem, which means they can no longer be accessed directly. Again, complexity here is how we can configure partial fetching...

However, keeping in sync with the backend is much harder business. In roadmap as well, but with a longer timeframe... Basically, mcachefs was designed at first with the strong hypothesis that the backend will only be modified through mcachefs itself, and only when applying journal (because most of the time, the backend is offline).

How it should work :

resync should only happen after applying local journal
we should at least refresh the metadata cache (list of files/folders, ...)
we should delete local cache files that have been deleted or updated directly at backend side (ie not through mcachefs)
we could optionally re-fetch updated files locally.

Thanks for your interest, and if you wish to contribute, do not hesitate !

segator commented 6 years ago

So supose I have a FS with lot of large files and mcachefs.

if I try to access 1M of the file, what exactly do the mcachefs? mcachefs download the full file to local and them release the asked data?

So in this hypothesis if I have an application that read the first 4M of all the files, mcachefs will try to copy all the files?

Doloops commented 6 years ago

Hi segator,

yes, the whole file will be copied to local FS in the background, even if a few bytes have been read locally.

Best regards,

Doloops / mcachefs

Some questions #3