search

Eyevinn / mp4ff

Library and tools for parsing and writing MP4 files including video, audio and subtitles. The focus is on fragmented files. Includes mp4ff-info, mp4ff-encrypt, mp4ff-decrypt and other tools.
MIT License
444 stars 81 forks source link

File from Unified Streaming packager #274

Closed PhilippeR closed 11 months ago

PhilippeR commented 11 months ago

I would like to use mp4ff-wvttlister to extract webvtt from mp4 packaged by Unified Streaming packager. It fails

mp4ff-wvttlister test.mp4 Track 1, timescale = 1000 [vttC] size=14

goroutine 1 [running]: github.com/Eyevinn/mp4ff/mp4.(TfdtBox).BaseMediaDecodeTime(...) /home/ubuntu/go/pkg/mod/github.com/!eyevinn/mp4ff@v0.38.1/mp4/tfdt.go:76 github.com/Eyevinn/mp4ff/mp4.(Fragment).GetFullSamples(0xc00006a3e0?, 0xc000016120) /home/ubuntu/go/pkg/mod/github.com/!eyevinn/mp4ff@v0.38.1/mp4/fragment.go:130 +0x8a main.parseFragmentedMp4(0xc000128000?, 0xe038?, 0xffffffffffffffff) /home/ubuntu/go/pkg/mod/github.com/!eyevinn/mp4ff@v0.38.1/cmd/mp4ff-wvttlister/main.go:170 +0x42c main.main() /home/ubuntu/go/pkg/mod/github.com/!eyevinn/mp4ff@v0.38.1/cmd/mp4ff-wvttlister/main.go:67 +0x2db

https://github.com/Eyevinn/mp4ff/assets/5978776/2a42de1d-533e-4550-b49d-95a1cd4a222a

Is it a bug or a bad file ? test.zip

tobbee commented 11 months ago

Hi @PhilippeR, thanks for using mp4ff.

The crash happens because the tool expects to get timing information from an tfdt box. That is the usual way to get it in DASH/CMAF. This file is a SmooothStream file, so the corresponding information is only in a tfra box at the end.

The combination of using wvtt and SmoothStreaming is not in the Microsoft specification, and I haven't seen that before, but the content should still be adressable.

I tried to get beyond this crash and ran into another problem. That is that the offset for the mdat box is relative to the moof preceding it, but not signaled the normal way. This is not allowed from iso5 and forward, but this file signals the archaic iso2 format, so it is allowed.

Anyway, I think it would be good if mp4ff-wvttlister could handle this file, so I will try to add the necessary changes. Is it OK to include this file as a test file with the code to ensure that the content remains decodable? If not, could you provide another file that could be included?

I did a quick hack so that I could extract the content, but with the timing reset in each segment (due to not extracting it from tfra). Do you want me to send you the extract, or post it here?

PhilippeR commented 11 months ago

Hi @tobbee , I knew that it was an unusual file. Please find attached a sample that you can use. sample.zip

tobbee commented 11 months ago

@PhilippeR It took a little effort, but now it should be possible to look at the content of this type of files. Please try out #275 to see if it works for you,

Thanks for the sample file. I shortened it to only two segments and included it in a new test.

PhilippeR commented 11 months ago

hi @tobbee. Looks good.