nschlia / ffmpegfs

FUSE-based transcoding filesystem with video support from many formats to FLAC, MP4, TS, WebM, OGG, MP3, HLS, and others.
https://nschlia.github.io/ffmpegfs/
GNU General Public License v3.0
196 stars 14 forks source link

Files Remain Zero Size When Previously Transcoded? #129

Closed Martin2112 closed 2 years ago

Martin2112 commented 2 years ago

Sorry, this could be me misunderstanding how things should work but something seems odd when tracks were transcoded in previous sessions.

[martin@zen ~]$ ls -l /export/cdrip/flactrack/Music/Electric\ Light\ Orchestra/Out\ of\ the\ Blue/Electric\ Light\ Orchestra\ -\ Out\ of\ the\ Blue.flac.tracks/01.\ Electric\ Light\ Orchestra\ -\ Turn\ to\ Stone\ \[03-46.893\].flac 
-rwxr--r--. 1 martin martin 0 Mar 12 17:31 '/export/cdrip/flactrack/Music/Electric Light Orchestra/Out of the Blue/Electric Light Orchestra - Out of the Blue.flac.tracks/01. Electric Light Orchestra - Turn to Stone [03-46.893].flac'

This file has previously been transcoded . There's a cache entry for it and a row in the sqlite db with finished = 2.

[martin@zen ~]$ ls -l /var/cache/ffmpegfs/export/cdrip/flactrack/export/cdrip/flac/Music/Electric\ Light\ Orchestra/Out\ of\ the\ Blue/Electric\ Light\ Orchestra\ -\ Out\ of\ the\ Blue.flac.tracks/01.\ Electric\ Light\ Orchestra\ -\ Turn\ to\ Stone\ \[03-46.893\].flac.cache.flac 
-rw-r--r--. 1 martin martin 27992970 Mar 20 09:33 '/var/cache/ffmpegfs/export/cdrip/flactrack/export/cdrip/flac/Music/Electric Light Orchestra/Out of the Blue/Electric Light Orchestra - Out of the Blue.flac.tracks/01. Electric Light Orchestra - Turn to Stone [03-46.893].flac.cache.flac'

If I access it then it's not really zero bytes.

[martin@zen ~]$ strings /export/cdrip/flactrack/Music/Electric\ Light\ Orchestra/Out\ of\ the\ Blue/Electric\ Light\ Orchestra\ -\ Out\ of\ the\ Blue.flac.tracks/01.\ Electric\ Light\ Orchestra\ -\ Turn\ to\ Stone\ \[03-46.893\].flac | wc
 385202  407282 2172444

I can see it being read from the cache in the log file. E.g:

2022-03-20 09:33:05 TRACE  : [/var/cache/ffmpegfs//export/cdrip/flactrack//export/cdrip/flac/Music/Electric Light Orchestra/Out of the Blue/Electric Light Orchestra - Out of the Blue.flac.tracks/01. Electric Light Orchestra - Turn to Stone [03-46.893].flac.cache.flac] Closing cache file.

It's still zero bytes via getattr:

[martin@zen ~]$ ls -l /export/cdrip/flactrack/Music/Electric\ Light\ Orchestra/Out\ of\ the\ Blue/Electric\ Light\ Orchestra\ -\ Out\ of\ the\ Blue.flac.tracks/01.\ Electric\ Light\ Orchestra\ -\ Turn\ to\ Stone\ \[03-46.893\].flac 
-rwxr--r--. 1 martin martin 0 Mar 12 17:31 '/export/cdrip/flactrack/Music/Electric Light Orchestra/Out of the Blue/Electric Light Orchestra - Out of the Blue.flac.tracks/01. Electric Light Orchestra - Turn to Stone [03-46.893].flac'

If I try to play it in VLC then it doesn't work. I'm assuming that's because it's still seeing the file as zero byte size.

If I run up ffmpegfs on a dev machine and the files are transcoded in the same session then getattr returns the correct size immediately afterwards.

nschlia commented 2 years ago

There seems to be something wrong with the file size prediction, I just recreated the problem:

  1. Start ffmpegfs, directly do ls:

    $ ls -l '/home/norbert/test/out/Subway to Sally - Nord Nord Ost.flac.tracks/'
    insgesamt 0
    -rwxr-xr-x 1 norbert norbert 0  9. Feb 2021  '01. Subway to Sally - Sarabande de noir [00-55.400].flac'
    -rwxr-xr-x 1 norbert norbert 0  9. Feb 2021  '02. Subway to Sally - Schneekönigin [05-46.106].flac'
    -rwxr-xr-x 1 norbert norbert 0  9. Feb 2021  '03. Subway to Sally - Feuerland [04-07.053].flac'
    -rwxr-xr-x 1 norbert norbert 0  9. Feb 2021  '04. Subway to Sally - Sieben [03-22.626].flac'
    -rwxr-xr-x 1 norbert norbert 0  9. Feb 2021  '05. Subway to Sally - Lacrimae '\''74 [01-45.906].flac'
    -rwxr-xr-x 1 norbert norbert 0  9. Feb 2021  '06. Subway to Sally - Feuerkind [06-06.280].flac'
    -rwxr-xr-x 1 norbert norbert 0  9. Feb 2021  '07. Subway to Sally - Das Rätsel II [04-21.933].flac'
    -rwxr-xr-x 1 norbert norbert 0  9. Feb 2021  '08. Subway to Sally - S.O.S. [05-56.253].flac'
    -rwxr-xr-x 1 norbert norbert 0  9. Feb 2021  '09. Subway to Sally - Eisblumen [04-32.386].flac'
    -rwxr-xr-x 1 norbert norbert 0  9. Feb 2021  '10. Subway to Sally - Seemannslied [05-17.680].flac'

    All files report at size 0!

  2. Now copy one file to trigger transcoding, then repeat ls:

    
    $ cp '/home/norbert/test/out/Subway to Sally - Nord Nord Ost.flac.tracks/01. Subway to Sally - Sarabande de noir [00-55.400].flac' . -v
    '/home/norbert/test/out/Subway to Sally - Nord Nord Ost.flac.tracks/01. Subway to Sally - Sarabande de noir [00-55.400].flac' -> './01. Subway to Sally - Sarabande de noir [00-55.400].flac'

$ ls -l '/home/norbert/test/out/Subway to Sally - Nord Nord Ost.flac.tracks/'insgesamt 52695 -rwxr-xr-x 1 norbert norbert 6296911 9. Feb 2021 '01. Subway to Sally - Sarabande de noir [00-55.400].flac' -rwxr-xr-x 1 norbert norbert 47662101 9. Feb 2021 '02. Subway to Sally - Schneekönigin [05-46.106].flac' -rwxr-xr-x 1 norbert norbert 0 9. Feb 2021 '03. Subway to Sally - Feuerland [04-07.053].flac' -rwxr-xr-x 1 norbert norbert 0 9. Feb 2021 '04. Subway to Sally - Sieben [03-22.626].flac' -rwxr-xr-x 1 norbert norbert 0 9. Feb 2021 '05. Subway to Sally - Lacrimae '\''74 [01-45.906].flac' -rwxr-xr-x 1 norbert norbert 0 9. Feb 2021 '06. Subway to Sally - Feuerkind [06-06.280].flac' -rwxr-xr-x 1 norbert norbert 0 9. Feb 2021 '07. Subway to Sally - Das Rätsel II [04-21.933].flac' -rwxr-xr-x 1 norbert norbert 0 9. Feb 2021 '08. Subway to Sally - S.O.S. [05-56.253].flac' -rwxr-xr-x 1 norbert norbert 0 9. Feb 2021 '09. Subway to Sally - Eisblumen [04-32.386].flac' -rwxr-xr-x 1 norbert norbert 0 9. Feb 2021 '10. Subway to Sally - Seemannslied [05-17.680].flac'

That file just copied has a size. The copy is intact and playable, BTW. All other files are reported zero sized.

3. Next stop and restart ffmpegfs, do an ls, copy again, and do another ls after copying the file:

$ ls -l '/home/norbert/test/out/Subway to Sally - Nord Nord Ost.flac.tracks/' insgesamt 0 -rwxr-xr-x 1 norbert norbert 0 9. Feb 2021 '01. Subway to Sally - Sarabande de noir [00-55.400].flac' -rwxr-xr-x 1 norbert norbert 0 9. Feb 2021 '02. Subway to Sally - Schneekönigin [05-46.106].flac' -rwxr-xr-x 1 norbert norbert 0 9. Feb 2021 '03. Subway to Sally - Feuerland [04-07.053].flac' -rwxr-xr-x 1 norbert norbert 0 9. Feb 2021 '04. Subway to Sally - Sieben [03-22.626].flac' -rwxr-xr-x 1 norbert norbert 0 9. Feb 2021 '05. Subway to Sally - Lacrimae '\''74 [01-45.906].flac' -rwxr-xr-x 1 norbert norbert 0 9. Feb 2021 '06. Subway to Sally - Feuerkind [06-06.280].flac' -rwxr-xr-x 1 norbert norbert 0 9. Feb 2021 '07. Subway to Sally - Das Rätsel II [04-21.933].flac' -rwxr-xr-x 1 norbert norbert 0 9. Feb 2021 '08. Subway to Sally - S.O.S. [05-56.253].flac' -rwxr-xr-x 1 norbert norbert 0 9. Feb 2021 '09. Subway to Sally - Eisblumen [04-32.386].flac' -rwxr-xr-x 1 norbert norbert 0 9. Feb 2021 '10. Subway to Sally - Seemannslied [05-17.680].flac'

$ cp '/home/norbert/test/out/Subway to Sally - Nord Nord Ost.flac.tracks/01. Subway to Sally - Sarabande de noir [00-55.400].flac' . -v '/home/norbert/test/out/Subway to Sally - Nord Nord Ost.flac.tracks/01. Subway to Sally - Sarabande de noir [00-55.400].flac' -> './01. Subway to Sally - Sarabande de noir [00-55.400].flac'

norbert@flachmops:~/test$ ls -l '/home/norbert/test/out/Subway to Sally - Nord Nord Ost.flac.tracks/'insgesamt 0 -rwxr-xr-x 1 norbert norbert 0 9. Feb 2021 '01. Subway to Sally - Sarabande de noir [00-55.400].flac' -rwxr-xr-x 1 norbert norbert 0 9. Feb 2021 '02. Subway to Sally - Schneekönigin [05-46.106].flac' -rwxr-xr-x 1 norbert norbert 0 9. Feb 2021 '03. Subway to Sally - Feuerland [04-07.053].flac' -rwxr-xr-x 1 norbert norbert 0 9. Feb 2021 '04. Subway to Sally - Sieben [03-22.626].flac' -rwxr-xr-x 1 norbert norbert 0 9. Feb 2021 '05. Subway to Sally - Lacrimae '\''74 [01-45.906].flac' -rwxr-xr-x 1 norbert norbert 0 9. Feb 2021 '06. Subway to Sally - Feuerkind [06-06.280].flac' -rwxr-xr-x 1 norbert norbert 0 9. Feb 2021 '07. Subway to Sally - Das Rätsel II [04-21.933].flac' -rwxr-xr-x 1 norbert norbert 0 9. Feb 2021 '08. Subway to Sally - S.O.S. [05-56.253].flac' -rwxr-xr-x 1 norbert norbert 0 9. Feb 2021 '09. Subway to Sally - Eisblumen [04-32.386].flac' -rwxr-xr-x 1 norbert norbert 0 9. Feb 2021 '10. Subway to Sally - Seemannslied [05-17.680].flac'


The copy, again, is intact, has a size greater than 0 and is playable. But all files are reported with size 0.

Obviously the file size prediction goes berserk in the first place, and the size stored inside the database gets lost.

Bug confirmed, I will fix it.
nschlia commented 2 years ago

Bug confirmed in 2.9 release as well :(

nschlia commented 2 years ago

For some formats like FLAC number of channels and sample rate were required. Added check to determine these values.

Martin2112 commented 2 years ago

OK thanks. I'll try it out when I get chance.

Martin2112 commented 2 years ago

Do I need to clear the cache before trying this?

nschlia commented 2 years ago

Do I need to clear the cache before trying this?

No.

Martin2112 commented 2 years ago

It's definitely working better and I can play the tracks that wouldn't open before. I'd say this is fixed.