FFMS / ffms2

An FFmpeg based source library and Avisynth/VapourSynth plugin for easy frame accurate access
Other
574 stars 104 forks source link

Various fixes to seeking and decoding #437

Closed arch1t3cht closed 2 months ago

arch1t3cht commented 3 months ago

This consists of various bigger refactors which were necessary to fix certain parts of seeking, but it can still mostly be read commit for commit. Some of the later commits do make some of the earlier commits partially obsolete, so let me know if you'd like any of this squashed.

I sent a patch to ffmpeg to fix 70a7c1b619105fcc6bf4ff77cc4ea34853651cd9 but didn't get a response yet.

Running seek tests with this on the doom9 sample file collection shows no regressions: Before (the format being filename,number of errors):

00006.mkv,0
132518447-659caa63-ce2d-4120-9f76-5700cfc7fcb6.mov,0
3d-parrot.mkv,546
3d-parrot.MTS,547
amarec(20200806-1406)_sample.avi,0
apple_trailers.mov,0
a_test.mkv,0
AV1 Summer.webm,0
AVC ES.264,0
avidv.avi,0
bars601.mkv,0
bars601.ts,224
batman.v.superman.mkv,0
BD 21 grammi.mkv,0
BoatsAtLahaina_too_few_timecodes.mp4,0
camcorder_25i_4-3.mkv,0
CANON.MXF,0
Chimera-AV1-10bit-1280x720-2380kbps.mp4,0
crowd-x265.mp4,0
dvd.mkv,0
dvd.mpeg,0
ffms2_seeking_issue.mkv,0
ffms2_seeking_issue.mp4,224
flpyoj.mkv,0
h263.3gp,0
interlaced_h264.mkv,139
interlaced_h264.mp4,196
Letter - SHE'S.mkv,0
MainconceptLogo_Blu-ray_MPEG2_1920x1080_LPCM.mkv,0
MainconceptLogo_Blu-ray_MPEG2_1920x1080_LPCM.mpg,28
MainconceptLogo_MPEG2_DVD_720x576.mkv,0
MainconceptLogo_MPEG2_DVD_720x576.mpg,88
MC TMB.mov,0
mpeg1.mov,Error
mpeg1.mpg,155
MPEGSolution_stuart.mkv,0
MPEGSolution_stuart.mp4,8634
Nagi no Asukara 2013 - EP01 [BD 1920x1080 23.976fps AVC-yuv444p10 FLACx3 Chap] - mawen1250.mkv,0
NTSC_720p_MPEG_XDCAM-EX_colorbar.mxf,0
parkrun1280_12mbps.mkv,0
parkrun1280_12mbps.ts,473
rav1e_b_4600.ivf,0
rav1e_q_150.mkv,0
seek-test.mkv,0
snow.MTS,437
Sunset.m2ts,0
Sunset.mkv,0
VC1.mpg,98
vc1_sample.mkv,579
VP9.mkv,0
VTS_01_1.demuxed.m2v,0
yVY.webm,0
Zenit.mkv,4487

After: (Don't pay too close attention to the actual numbers, for this second seek test I had to clip all videos to 200 frames to make the tests faster. What's important is the presence or absence of errors. If necessary I can also run a full seek test, though.)

00006.mkv,0
132518447-659caa63-ce2d-4120-9f76-5700cfc7fcb6.mov,0
3d-parrot.mkv,0
3d-parrot.MTS,50
amarec(20200806-1406)_sample.avi,0
apple_trailers.mov,0
a_test.mkv,0
AV1 Summer.webm,0
AVC ES.264,0
avidv.avi,0
bars601.mkv,0
bars601.ts,41
batman.v.superman.mkv,0
BD 21 grammi.mkv,0
BoatsAtLahaina_too_few_timecodes.mp4,0
camcorder_25i_4-3.mkv,0
CANON.MXF,0
Chimera-AV1-10bit-1280x720-2380kbps.mp4,0
crowd-x265.mp4,0
dvd.mkv,0
dvd.mpeg,0
ffms2_seeking_issue.mkv,0
ffms2_seeking_issue.mp4,0
flpyoj.mkv,0
h263.3gp,0
interlaced_h264.mkv,0
interlaced_h264.mp4,0
Letter - SHE'S.mkv,0
MainconceptLogo_Blu-ray_MPEG2_1920x1080_LPCM.mkv,0
MainconceptLogo_Blu-ray_MPEG2_1920x1080_LPCM.mpg,20
MainconceptLogo_MPEG2_DVD_720x576.mkv,0
MainconceptLogo_MPEG2_DVD_720x576.mpg,84
MC TMB.mov,0
mpeg1.mov,Error
mpeg1.mpg,125
MPEGSolution_stuart.mkv,0
MPEGSolution_stuart.mp4,374
Nagi no Asukara 2013 - EP01 [BD 1920x1080 23.976fps AVC-yuv444p10 FLACx3 Chap] - mawen1250.mkv,0
NTSC_720p_MPEG_XDCAM-EX_colorbar.mxf,0
parkrun1280_12mbps.mkv,0
parkrun1280_12mbps.ts,41
rav1e_b_4600.ivf,0
rav1e_q_150.mkv,0
seek-test.mkv,0
snow.MTS,1
Sunset.m2ts,0
Sunset.mkv,0
VC1.mpg,242
vc1_sample.mkv,47
VP9.mkv,0
VTS_01_1.demuxed.m2v,0
yVY.webm,0
Zenit.mkv,0

Some background on some specific files affected by this PR:

So after this PR all mkv and mp4 files have consistent seeking, except for:

dwbuiten commented 3 months ago

@arch1t3cht Thanks! This is a lot, so I'll start digging into it later today - might take a day or three.

(I'll run it through $dayjob's internal tests as well, in the future.)

arch1t3cht commented 2 months ago

Pushed another few commits (sorry for the added work...), to try and fix most of the remaining mpegts files.

Seek test results (capped to 500 frames this time, except for capping CANON.MXF to 100 because that file is huge)

3d-parrot.mkv,0
3d-parrot.MTS,0
00006.mkv,0
132518447-659caa63-ce2d-4120-9f76-5700cfc7fcb6.mov,0
a_test.mkv,0
amarec(20200806-1406)_sample.avi,0
apple_trailers.mov,0
AV1 Summer.webm,0
AVC ES.264,0
avidv.avi,0
bars601.mkv,0
bars601.ts,0
batman.v.superman.mkv,0
BD 21 grammi.mkv,0
BoatsAtLahaina_too_few_timecodes.mp4,0
camcorder_25i_4-3.mkv,0
CANON.MXF,0
Chimera-AV1-10bit-1280x720-2380kbps.mp4,0
crowd-x265.mp4,0
dvd.mkv,0
dvd.mpeg,0
ffms2_seeking_issue.mkv,0
ffms2_seeking_issue.mp4,0
flpyoj.mkv,0
h263.3gp,0
interlaced_h264.mkv,0
interlaced_h264.mp4,0
Letter - SHE'S.mkv,0
MainconceptLogo_Blu-ray_MPEG2_1920x1080_LPCM.mkv,0
MainconceptLogo_Blu-ray_MPEG2_1920x1080_LPCM.mpg,0
MainconceptLogo_MPEG2_DVD_720x576.mkv,0
MainconceptLogo_MPEG2_DVD_720x576.mpg,0
MC TMB.mov,0
mpeg1.mov,Error
mpeg1.mpg,150
MPEGSolution_stuart.mkv,0
MPEGSolution_stuart.mp4,966
Nagi no Asukara 2013 - EP01 [BD 1920x1080 23.976fps AVC-yuv444p10 FLACx3 Chap] - mawen1250.mkv,0
NTSC_720p_MPEG_XDCAM-EX_colorbar.mxf,0
parkrun1280_12mbps.mkv,0
parkrun1280_12mbps.ts,0
rav1e_b_4600.ivf,0
rav1e_q_150.mkv,0
seek-test.mkv,0
snow.MTS,2
Sunset.m2ts,0
Sunset.mkv,0
VC1.mpg,417
vc1_sample.mkv,154
VP9.mkv,0
VTS_01_1.demuxed.m2v,0
yVY.webm,0
Zenit.mkv,0

(Of course, if you have other files that regress with one of these commits, I can see if I can fix that).

With the last two commits here it gets a bit more ugly since it's starting to become a game of whack-a-mole to work around certain files while not breaking others (e.g. most mpegts files here seem to behave better when UseDTS=false, except for dvd.mpeg where avformat does spit out some invalid timestamps), but I tried my best to keep it somewhat organized.

dwbuiten commented 2 months ago

Starting to go through them. A few commits look like I can pull them out and merge immeiately, some I think require some more explanation (like under what circumstances you have seen keyframes with PTS != DTS, or how using PTS in DecodePacket() can work on e.g. MPEG-TS with timestamp wraparound (non-monotonic)).

I have dropped you a PM on IRC, as you seem to be active there (?), since real time communication, at least initially, may be easier.

(I wish GitHub was less bad at per-commit review...)