search

FFMS / ffms2

An FFmpeg based source library and Avisynth/VapourSynth plugin for easy frame accurate access
Other
574 stars 104 forks source link

Fix AV1/VP9 (and send/receive API use in general) #428

Closed dwbuiten closed 4 months ago

dwbuiten commented 8 months ago

This PR does two things:

Since this makes changes to the core decode loop(s), it needs some good testing before merging. Particularily to confiirm AV1 and VP9 are, in fact, fixed (seeking, and linear).

Should (probably) fix #319, and makes #323 obsolete, I think. May or may not fix #352.

CC: @tdaede @jamrial @wangqr

myrsloik commented 8 months ago

Looks reasonable, will try on some random av1 files later

CoffeeFlux commented 7 months ago

Anything I can do to help move this along? I'm looking at getting a new Aegisub stable out but I'd like to have proper VP9/AV1 support, and don't want to build against a fork.

dwbuiten commented 7 months ago

Anything I can do to help move this along? I'm looking at getting a new Aegisub stable out but I'd like to have proper VP9/AV1 support, and don't want to build against a fork.

Mostly it is a function of my time (of which I have little at the moment...) - the current PR seems to have issues, which I need to debug.

L4cache commented 6 months ago

How about using libvpx to decode vp9? It seems more multi-thread capable than native one (but maybe slightly slower in single-thread) and with support of alpha channel in addition.

It seems not suffer form https://github.com/FFMS/ffms2/issues/352. Let me list in detail:

  1. ffms2 current HEAD, ffmpeg branch release/6.1, native vp9 decoder: broken
  2. ffms2 current HEAD, ffmpeg branch release/6.1, libvpx decoder: good
  3. ffms2 with quietvoid's patch , ffmpeg branch release/6.1, native vp9 decoder: looks ok with regularly skipped(duplicated) frames
  4. ffms2 with quietvoid's patch , ffmpeg branch release/6.1, libvpx decoder: good
  5. ffms2 tag 2.40, ffmpeg branch release/4.3, native vp9 decoder: broken
  6. ffms2 tag 2.40, ffmpeg branch release/4.3, libvpx decoder: looks ok but frame count and fps are off
  7. This PR, ffmpeg branch release/6.1, native vp9 decoder: good
  8. This PR, ffmpeg branch release/6.1, libvpx decoder: good

All above are built with msys2 mingw-w64 (as used by media-autobuild_suite), decoded data from all "good" results are exactly the same

L4cache commented 6 months ago

I encountered this error opening files with video+audio with several combination of codecs after this PR. vapoursynth.Error: Source: Insanity detected: decoder returned an empty frame

But not all file of problematic combination will cause error, for example vp9+opus produced by ffmpeg fails but some files downloaded from YouTube don't. If audio is removed then the decoding will be successful.

dwbuiten commented 4 months ago

@L4cache I am revisiting this PR currently, can you provide an example file?

L4cache commented 4 months ago

@L4cache I am revisiting this PR currently, can you provide an example file?

https://transfer.sh/85oyj7LBrm/sample.zip

I forgot to mention, the way I apply this pr is https://github.com/L4cache/ffms2/tree/test-3 Many more video+audio codec combinations are affected, like, basically anything but h264 or h265 + commonly used lossy audio encoders (libmp3lame is broken but libshine is ok so I say encoder)

dwbuiten commented 4 months ago

@L4cache Thanks, I was able to reproduce the issue using your sample. I will look into it.

dwbuiten commented 4 months ago

@L4cache I found the bug. I have updated the PR.

dwbuiten commented 4 months ago

Merged, let the testing begin!

Note thay CI failed for unrelated reasons. Tests pass locally, aside from HDR metadata which is being fixed upstream in FFmpeg right now.