Closed LigH-de closed 2 months ago
Goodness me. It build just a few days ago (that news, of course, helps no one).
Whilst good to fix, it seems as if it may not be the end of the world.
I gather people (including me) now use vspipe
to pipe frames to ffmpeg.
My current understanding is that piping does not require ffmpeg to be built with vapoursynth.
Indeed, there was discussion over at videohelp outlining decent performance gains from using vspipe vs vapoursynth inside ffmpeg, due to some anomaly although I am unsure if that info is still current.
Looks like the problem is in "alternative vapoursynth demuxer"
3 days after, I reported ffmpeg trac ticket 11169.
I don't know if it's appropriate to submit this one to upstream as I believe vapoursynth_alt is from a patch from mabs https://github.com/m-ab-s/media-autobuild_suite/blob/master/build/media-suite_compile.sh#L2229
Aha ... well. Either they close it for not being their fault or they have an idea.
I think my patch that was accepted in ffmpeg a few days ago is the reason for that: VapourSynth support in ffmpeg is rarely compiled in most distributed binaries, because it required VapourSynth to be installed for all users, no matter if they use VapourSynth or not. I updated the VapourSynth demuxer to the V4 API and made it load the VapourSynth library at runtime, just the way AviSynth works in ffmpeg. This way you compile with VapourSynth support enabled, but the resulting binary will work fine even if VapourSynth is not installed (of course if you want to open a VapourSynth script and it cannot find the VapourSynth dll/so/dylib it will fail). I'm not sure why you what the vapoursynth_alt patches are, but they probably need to be updated.
Avoiding hard requirement for vapoursynth dynamic library is really a good news.
I think my patch that was accepted in ffmpeg a few days ago
Thanks ! I see the patches: https://git.videolan.org/?p=ffmpeg.git;a=commit;h=d42cd5b75bd7eeeba8ef7f433edf61b31f6f2858 https://git.videolan.org/?p=ffmpeg.git;a=commit;h=eac611f1a4c494e5e3efe61e7867517e65706534
I'm not sure why you what the vapoursynth_alt patches are, but they probably need to be updated.
OK, I'm a tad confused. I do not know the implications of what this means. Could you please clarify what works and what doesn't ?
PS does it still work without vapoursynth being installed - i.e. will it work with the portable version of x64 vapoursynth with ffmpeg in the same folder, where no vapoursynth registry keys exist ?
Thank you.
OK, I'm a tad confused. I do not know the implications of what this means. Could you please clarify what works and what doesn't ?
ffmpeg will not link against VapourSynth anymore, as it loads it dynamically at runtime when needed. As the vapoursynth_alt demuxer does not do that, you'll get linking errors. You basically have three options: Change the configure to link against VapourSynth again, update vapoursynth_alt, so that it also loads VapourSynth at runtime or to use the default VapourSynth demuxer.
PS does it still work without vapoursynth being installed - i.e. will it work with the portable version of x64 vapoursynth with ffmpeg in the same folder, where no vapoursynth registry keys exist ?
Yes. It will look for VapourSynth in this order (on Windows): User Install, System Install, portable (dll in the same directory as ffmpeg). On other Linux/Mac it will just try to load the library, as those systems manage library locations.
What would really interest me: Why is there an alternative VapourSynth demuxer and what are its benefits? I found one mention that it is faster when using many filters. Is this still true? ffmpeg has updated it's design to be fully multi-threaded since then.
If the "alternative VapourSynth demuxer" is really obsolete now, then the easiest solution may be to remove its call from the compile script but still enable the linking of the library. In the source of the patch I could not spot any explicit handling of dynamic library loading, it rather contains management functions of the script and clip environment and auxiliary things like color spaces.
I'll try building one without 0001-Add-Alternative-VapourSynth-demuxer.patch and ask around to get it tested and compared against older builds. I am quite sure that @Selur has some experience due to his work on his Hybrid video converter which does support VapourSynth and ffmpeg...
If the "alternative VapourSynth demuxer" is really obsolete now, then the easiest solution may be to remove its call from the compile script but still enable the linking of the library. In the source of the patch I could not spot any explicit handling of dynamic library loading, it rather contains management functions of the script and clip environment and auxiliary things like color spaces.
Yes, I had a quick look, in fact it looks very similar to the standard VapourSynth demuxer, but there could be some detail that makes it faster
I'll try building one without 0001-Add-Alternative-VapourSynth-demuxer.patch and ask around to get it tested and compared against older builds. I am quite sure that @Selur has some experience due to his work on his Hybrid video converter which does support VapourSynth and ffmpeg...
That's a good idea. I have never tested the alternative VapourSynth demuxer, in fact I accidentally stumbled over this bug report and this is the first time I heard about it. If the alternative VapourSynth demuxer is still faster, it would be great if those improvements could be integrated in the standard VapourSynth demuxer and would be integrated in ffmpeg.
ffmpeg will not link against VapourSynth anymore, as it loads it dynamically at runtime when needed. As the vapoursynth_alt demuxer does not do that, you'll get linking errors. You basically have three options: Change the configure to link against VapourSynth again, update vapoursynth_alt, so that it also loads VapourSynth at runtime or to use the default VapourSynth demuxer.
Thanks !
Yes. It will look for VapourSynth in this order (on Windows): User Install, System Install, portable (dll in the same directory as ffmpeg). On other Linux/Mac it will just try to load the library, as those systems manage library locations.
Great !
What would really interest me: Why is there an alternative VapourSynth demuxer and what are its benefits? I found one mention that it is faster when using many filters. Is this still true? ffmpeg has updated it's design to be fully multi-threaded since then.
OK, I found a couple of old links containing some test results, which are no doubt out of date.
At this link downward: https://forum.videohelp.com/threads/397728-ffmpeg-accepting-vapoursynth-vpy-input-directly-and-gpu-accelerated-speed#post2679822
Here in points 2. and 3. are test results I had forgotten about (getting old, that happens a lot now) indicating significant slowness using the non _alt demuxer when compared to the vspipe approach: https://forum.videohelp.com/threads/397728-ffmpeg-accepting-vapoursynth-vpy-input-directly-and-gpu-accelerated-speed?p=2679915&viewfull=1#post2679915
Cheers
I only tested it when it was new and back then it did improve performance (a bit) in some cases, it wasn't really that much, so I didn't care to keep track of what happened to it.
Uploaded 3 of my last binaries, the latest has the AltVPYDemuxer removed. May include non GPL content (non-free license), not for distribution, only for testing, will be removed soon. Requires included libopenh264-7.dll in each case.
Hello. Not sure what's happening, so just a note to say the original error is still occurring. Cheers
Nothing happened yet, apparently. I gave you an archive of 3 binaries (the latest was built by me after removing the patch to add the alternative demuxer, the two older ones were built with alternative demuxer when that worked without errors) to test the VapourSynth script loading if you ever use VapourSynth at all (I do not, so I cannot test). Nobody tested yet.
I'm not using Windows, I'll see if the alternative demuxer builds on Linux and check if there are any speed differences. Not sure how that relates to Windows speed differences
I gave you an archive of 3 binaries (the latest was built by me after removing the patch to add the alternative demuxer, the two older ones were built with alternative demuxer when that worked without errors) to test the VapourSynth script loading if you ever use VapourSynth at all (I do not, so I cannot test). Nobody tested yet.
Thank you for your good work !
All of my workflows had been converted to use piping via vspipe into ffmpeg. I use vapoursynth every day. Perhaps, since nobody is testing, consideration could be given to discarding vapoursynth integration from MABS ? Although I would probably not recommend it on the basis it is a great fallback position should piping somehow break.
I will mention it in the doom9 forums, hoping there are more different users willing to test and benchmark.
The alternative demuxer is actually closer to what the AviSynth demuxer does in regard to how it gives the video stream to FFmpeg. The demuxer that got upstreamed uses wrapped_avframe, whereas the alternative demuxer serves it in as rawvideo. That is the major difference between them (and if you look back at the discussion on FFmpeg-devel when the vapoursynth demuxer got committed, there were questions about why wrapped_avframe was used and the alternative demuxer patch¹ was brought up explicitly, I believe), but there was also something about how filepaths inside the script were handled; I mentioned this on Doom9 when I documented how to statically link VapourSynth into FFmpeg, back in early 2019.
¹which at that time was not 'vapoursynth_alt', they both provided 'vapoursynth'. The _alt designation was something I added in to enable both at the same time, dating from before the wrapped_avframe one got upstreamed so that they could be compared.
https://github.com/qyot27/FFmpeg/commit/0123eaeb1ca962a6a9f16436c30ed73c454a1c81.patch
(he.who.shall.not.be.named = sekrit-twc; it was one of those grab-it-before-they-delete-it things that posted on their corresponding Doom9 account, although I think the patch itself was actually hosted on pastebin)
http://git.videolan.org/?p=ffmpeg.git;a=commit;h=7074a7ccd9a4d4e445252780fd182aa0b3778b79
In the wake of the patches to dynamically load VapourSynth, I did try to see if I could adapt the _alt demuxer to the same principles, but my knowledge of VapourSynth's API internals is essentially non-existent (most of the patches on the vapoursynth_alt branch generally showed up because something had immediately broken, but they were all FFmpeg-side, not VapourSynth-side). So yes, the appropriate solution would be to update the vapoursynth_alt demuxer, or to rewrite enough of the upstream demuxer to serve in as rawvideo instead of wrapped_avframe, and hopefully therefore close the performance gap. And if there is still file path weirdness inside vpy scripts with the upstream demuxer, fix that too.
EDIT: I do find it somewhat hilarious that the link to the tedious crosscompile guide in that old Doom9 post now points to the AviSynth+ instructions because the size of the guide and order of the individual dependencies has changed significantly since 5 years ago (line 4396 was near the end of the guide back then, after the encoders section instead of before). The VapourSynth instructions are directly after it as of 2024-09-12, on line 4504.
Nobody tested yet.
By simply relacing the ffmpeg exe and loading a .vpy file that works with vspipe from Vapoursynth R70, none of the three versions you provided work for me - ffmpeg error "Error opening input: Invalid data found when processing input".
Going forward, the question is how many people use ffmpeg filters that aren't provided by vapoursynth, and what the downside is of putting vspipe in front of ffmpeg (I didn't try that yet, I either use vapoursynth or ffmpeg to pipe to an external encoder).#
Edit: continuing on doom9...
When ffmpeg was compiled with VapourSynth support, it still needs to know where to find vapoursynth.dll and vsscript.dll as they are loaded dynamically to actually handle the *.vpy script as input.
Did you use the -f
option in your command line before you specified the input? You have to use ffmpeg -f vapoursynth -i myscript.vpy ...
(or vapoursynth_alt
), otherwise the file will not be loaded.
ffmpeg will not auto-detect VapourSynth on purpose: As VapourSynth is Python code you can do very powerful stuff, which can very useful, but also dangerous: Imagine ffmpeg would autodetect VapourSynth, someone could just send you a "video", but it does contain Python code that will infect your system, automatically executed by ffmpeg. Therefore you have to use the -f
option.
I just tested in my Windows 10 VM: All binaries provided by @LigH-de worked just fine. I used a small example script that generates some patterns and compressed it with ffv1.
I ran each benchmark 3 times and averaged the values, the difference was less than 5 fps in all cases. The CPU load is read from the task manager (graph averaged, load was pretty constant). As one can clearly see, the vapoursynth_alt demuxer is no improvement at all, it generates more load while being a lot slower. The vspipe option is still the fastest. Note that this is just one script tested on a virtual machine. Could be very different for another script on a native machine.
One important difference in the vapoursynth_alt
demuxer seems to be that it reads the frames asynchronously. Since ffmpeg is completely multithreaded since 7.0 (input/compression/output is separated into different threads) this behavior could be contra-productive.
vspipe always have been the best solution (so far)
This performance ratio looks very convincing that we can safely drop the alt-demuxer patch. I'll close this issue now and make a PR later. Thank you, @Stefan-Olt
Did you use the
-f
option in your command line before you specified the input? You have to useffmpeg -f vapoursynth -i myscript.vpy ...
(orvapoursynth_alt
), otherwise the file will not be loaded.
Right, that was the mistake - sorry for the confusion, I didn't use VapourSynth with ffmpeg before...
ffmpeg-git/build-static-64bit/ab-suite.make.log
logs.zip