nschlia / ffmpegfs

FUSE-based transcoding filesystem with video support from many formats to FLAC, MP4, TS, WebM, OGG, MP3, HLS, and others.
https://nschlia.github.io/ffmpegfs/
GNU General Public License v3.0
196 stars 14 forks source link

It seems possible for original files to be copied to cache #127

Closed Martin2112 closed 2 years ago

Martin2112 commented 2 years ago

Not sure about this one yet as I just failed to reproduce it on a different machine. However I noticed my cache was bigger than the input.

/dev/mapper/system-cdrip     329G   91G  222G  30% /export/cdrip
/dev/mapper/system-ffmpegfs  299G  122G  164G  43% /var/cache/ffmpegfs

And it seems that the original large FLAC files are present in the cache along with the transcoded track files:

ls -lh /var/cache/ffmpegfs/export/cdrip/flactrack/export/cdrip/flac/Music/Rush/Rush\ -\ Presto.flac.*
-rw-r--r--. 1 martin martin 334M Mar 18 21:13 '/var/cache/ffmpegfs/export/cdrip/flactrack/export/cdrip/flac/Music/Rush/Rush - Presto.flac.cache.flac'

'/var/cache/ffmpegfs/export/cdrip/flactrack/export/cdrip/flac/Music/Rush/Rush - Presto.flac.tracks':
total 336M
-rw-r--r--. 1 martin martin 32M Mar 18 21:13 '01. Rush - Show Donât Tell [05-01.426].flac.cache.flac'
-rw-r--r--. 1 martin martin 30M Mar 18 21:13 '02. Rush - Chain Lightning [04-33.200].flac.cache.flac'
-rw-r--r--. 1 martin martin 29M Mar 18 21:13 '03. Rush - The Pass [04-50.706].flac.cache.flac'
-rw-r--r--. 1 martin martin 36M Mar 18 21:13 '04. Rush - War Paint [05-24.466].flac.cache.flac'
-rw-r--r--. 1 martin martin 26M Mar 18 21:13 '05. Rush - Scars [04-07.266].flac.cache.flac'
-rw-r--r--. 1 martin martin 39M Mar 18 21:13 '06. Rush - Presto [05-46.026].flac.cache.flac'
-rw-r--r--. 1 martin martin 33M Mar 18 21:13 '07. Rush - Superconductor [04-47.266].flac.cache.flac'
-rw-r--r--. 1 martin martin 26M Mar 18 21:13 '08. Rush - Anagram (for Mongo) [03-59.666].flac.cache.flac'
-rw-r--r--. 1 martin martin 29M Mar 18 21:13 '09. Rush - Red Tide [04-30.466].flac.cache.flac'
-rw-r--r--. 1 martin martin 28M Mar 18 21:13 '10. Rush - Hand Over Fist [04-10.240].flac.cache.flac'
-rw-r--r--. 1 martin martin 32M Mar 18 21:13 '11. Rush - Available Light [05-03.360].flac.cache.flac'

This was run with:

/usr/local/bin/ffmpegfs \
        -f \
        --log_stderr \
        --log_maxlevel=DEBUG \
        --autocopy=OFF \
        --recodesame=YES \
        --prebuffer_size=1M \
        --cachepath=/var/cache \
        --expiry_time=24w \
        --min_diskspace=5G \
        /export/cdrip/flac \
        /export/cdrip/flactrack \
        -o allow_other,ro,desttype=flac
nschlia commented 2 years ago

This is no bug, the original (large) files are also cached, you can see them in the outbound directory.

Example

Input files:

-rwxr-xr-x 1 norbert norbert 1,2K  9. Feb 2021  'Subway to Sally - Nord Nord Ost.cue'
-rw-r--r-- 1 norbert norbert 315M 28. Jan 2021  'Subway to Sally - Nord Nord Ost.flac'
-rwxr-xr-x 1 norbert norbert 1,5K 13. Feb 2021  'Мельница - Дорога сна.cue'
-rwxr-xr-x 1 norbert norbert 258M 12. Feb 2021  'Мельница - Дорога сна.flac'

Output files:

insgesamt 581M
-rwxr-xr-x 1 norbert norbert 1,2K  9. Feb 2021  'Subway to Sally - Nord Nord Ost.cue'
-rw-r--r-- 1 norbert norbert 316M 28. Jan 2021  'Subway to Sally - Nord Nord Ost.flac'
drwxr-xr-x 2 norbert norbert 4,0K  9. Feb 2021  'Subway to Sally - Nord Nord Ost.flac.tracks'
-rwxr-xr-x 1 norbert norbert 1,5K 13. Feb 2021  'Мельница - Дорога сна.cue'
-rwxr-xr-x 1 norbert norbert 265M 12. Feb 2021  'Мельница - Дорога сна.flac'
drwxr-xr-x 2 norbert norbert 4,0K 13. Feb 2021  'Мельница - Дорога сна.flac.tracks'

'out/Subway to Sally - Nord Nord Ost.flac.tracks':
insgesamt 316M
-rwxr-xr-x 1 norbert norbert 6,1M  9. Feb 2021  '01. Subway to Sally - Sarabande de noir [00-55.400].flac'
-rwxr-xr-x 1 norbert norbert  46M  9. Feb 2021  '02. Subway to Sally - Schneekönigin [05-46.106].flac'
-rwxr-xr-x 1 norbert norbert  34M  9. Feb 2021  '03. Subway to Sally - Feuerland [04-07.053].flac'
-rwxr-xr-x 1 norbert norbert  27M  9. Feb 2021  '04. Subway to Sally - Sieben [03-22.626].flac'
-rwxr-xr-x 1 norbert norbert 8,7M  9. Feb 2021  '05. Subway to Sally - Lacrimae '\''74 [01-45.906].flac'
-rwxr-xr-x 1 norbert norbert  43M  9. Feb 2021  '06. Subway to Sally - Feuerkind [06-06.280].flac'
-rwxr-xr-x 1 norbert norbert  34M  9. Feb 2021  '07. Subway to Sally - Das Rätsel II [04-21.933].flac'
-rwxr-xr-x 1 norbert norbert  47M  9. Feb 2021  '08. Subway to Sally - S.O.S. [05-56.253].flac'
-rwxr-xr-x 1 norbert norbert  35M  9. Feb 2021  '09. Subway to Sally - Eisblumen [04-32.386].flac'
-rwxr-xr-x 1 norbert norbert  39M  9. Feb 2021  '10. Subway to Sally - Seemannslied [05-17.680].flac'

'out/Мельница - Дорога сна.flac.tracks':
insgesamt 265M
-rwxr-xr-x 1 norbert norbert 19M 13. Feb 2021  '01. Мельница - Дорога сна [03-13.213].flac'
-rwxr-xr-x 1 norbert norbert 21M 13. Feb 2021  '02. Мельница - Горец [03-24.880].flac'
-rwxr-xr-x 1 norbert norbert 30M 13. Feb 2021  '03. Мельница - Ольга [04-39.413].flac'
-rwxr-xr-x 1 norbert norbert 12M 13. Feb 2021  '04. Мельница - Лорд Грегори [02-20.253].flac'
-rwxr-xr-x 1 norbert norbert 29M 13. Feb 2021  '05. Мельница - Змей [05-52.733].flac'
-rwxr-xr-x 1 norbert norbert 11M 13. Feb 2021  '06. Мельница - Тарантелла [01-56.440].flac'
-rwxr-xr-x 1 norbert norbert 22M 13. Feb 2021  '07. Мельница - Зима [04-20.946].flac'
-rwxr-xr-x 1 norbert norbert 36M 13. Feb 2021  '08. Мельница - Оборотень [05-51.253].flac'
-rwxr-xr-x 1 norbert norbert 16M 13. Feb 2021  '09. Мельница - Рапунцель [02-41.866].flac'
-rwxr-xr-x 1 norbert norbert 29M 13. Feb 2021  '10. Мельница - Воин вереска [05-52.053].flac'
-rwxr-xr-x 1 norbert norbert 27M 13. Feb 2021  '11. Мельница - Мора [04-11.520].flac'
-rwxr-xr-x 1 norbert norbert 21M 13. Feb 2021  '12. Мельница - На север [03-34.346].flac'

Resulting sizes:

1,2G    cache
572M    in
316M    out/Subway to Sally - Nord Nord Ost.flac.tracks
265M    out/Мельница - Дорога сна.flac.tracks
1,2G    out

You can see, the cache size is about the size or the output files. The tracked files are bout the size of the corresponding large source file.

Martin2112 commented 2 years ago

OK. Wasn't sure why they'd be cached when the original file is available.

nschlia commented 2 years ago

OK. Wasn't sure why they'd be cached when the original file is available.

It's the recoded version. More obvious when using mp4 or so, much smaller results :)