kahing / catfs

Cache AnyThing filesystem written in Rust
Apache License 2.0
846 stars 54 forks source link

Hello! and thanks for the response over at pcachefs #21

Open Jieiku opened 5 years ago

Jieiku commented 5 years ago

catfs was actually one of the very first ones I tried!

I am running ubuntu 18.04.1 and here is how I installed catfs:

sudo apt-get install cargo git libfuse-dev fuse python-fuse
git clone https://github.com/kahing/catfs.git
cd catfs
sudo cargo install catfs

Here is how I tried it out using a sshfs mount /media/remote with catfs :

sshfs -o reconnect,noatime,ServerAliveInterval=15,ServerAliveCountMax=3,cache_timeout=14400,entry_timeout=1200,large_read,max_read=65536,cache=yes,kernel_cache,workaround=nodelaysrv,cache_stat_timeout=600 admin007@myremotefs:/ /media/remote

~/.cargo/bin/catfs /media/remote /media/cache /media/video

and then launched kodi media player (which is configured to read from /media/video), kodi just gave me a spinning progress indicator while trying to play a file from the catfs /media/video mount, and I could see monitoring bandwidth that the file was transferring, so it seemed to be trying to transfer the entire file before playback. I did not have this issue with mcachefs or pcachefs.

Currently I am using mcachefs because of the ones I tried it performed the fastest.

Is there anything else I should try? maybe I am just missing a small option or setting to get this party going :)

Jieiku commented 5 years ago

Also I just did another test to make sure this was still happening.

One thing that I did note is that catfs appears to be transferring the file at about 17 Mb/s which is the same speed I am seeing with mcachefs, so I think if I could get it start playing back without waiting for the entire file, then I could start using catfs

EDIT: Another thing I just noticed, if you just let kodi wait for the entire file to get finished, and let the progress indicator spin, once the file is done completely downloading/caching, instead of playing it back kodi stops playback completely. THEN if you play the file back another time, it will load the file from cache (which is of course fast now that it is stored locally.)

Jieiku commented 5 years ago

I have carefully read over the catfs home page, I see user_xattr must be used, I am not sure if this is enabled on my system or not. I am currently investigating how to make sure its enabled.

kahing commented 5 years ago

I suspect the problem is that the application does not read the file sequentially. While catfs can stream from remote, if the application reads at offset n catfs does need to read from 0-n first

Jieiku commented 5 years ago

ah, well darn. So you do not suspect that user_xattr has anything to do with it then, if so I will stop chasing that rabbit. I just installed xattr on both the remote and local systems, and was getting ready to test that it works.

I am wondering if there is a way that I can check if kodi is reading the file sequentially or not, possibly from the ubuntu commandline

Jieiku commented 5 years ago

I made a post about it over in the Kodi development forums: https://forum.kodi.tv/showthread.php?tid=336660

I would really LOVE to use catfs with Kodi if that were ever possible.

Jieiku commented 5 years ago

So I got feedback and unfortunately Kodi does not read the file sequentially, so you were right.

Is there a setting that will alllow catfs to work on files non sequentially? it seemed like pcachefs was able to do it.

Please let me know either way, if not and no plans to then maybe if I get some free time I will try converting pcachefs to rust or c/c++ : https://github.com/ibizaman/pcachefs/issues/5

Thanks.