Are vulkan hwaccel decoders enabled for the ffmpeg 6.1 build?

[streamingdv]

Hi,

this is just a question, I saw that the ffmpeg dependencies got upgraded to 6.1, are vulkan hwaccel decoders enabled for the ffmpeg build (H264/ HEVC/ AV1)? Still not sure where exactly the build configs are located. Thanks for the information.

[saudet]

The builds are here: https://github.com/bytedeco/javacpp-presets/blob/master/ffmpeg/cppbuild.sh

[streamingdv]

@saudet so does that mean that vulkan is disabled?

https://github.com/bytedeco/javacpp-presets/blob/6e36cac407af28bfac011cd0b99352f16adf3bee/ffmpeg/cppbuild.sh#L10C1-L10C8

--disable-vulkan

[streamingdv]

@saudet okay I see, I guess this should be enabled then as these features got introduced in ffmpeg 6.1

https://www.phoronix.com/news/FFmpeg-6.1-Released

Or is there any specific reason why vulkan is disabled?

I think the the decoders can be enabled via

--enable-hwaccel=h264_vulkan --enable-hwaccel=hevc_vulkan --enable-hwaccel=av1_vulkan

And probably remove the --disable-vulkan. Not sure if any other dependencies need to be included or build before in order to enable the vulkan hwaccel decoder support.

[saudet]

You can give it a try if you want

[streamingdv]

Okay I will check it out

[streamingdv]

@saudet good news, it actually works. I modified the build options like this

https://github.com/streamingdv/javacpp-presets/commit/59e9364e9b04ba43bd1485c5feb6a7fa4c27ca68

I've build javacpp-presets locally and

final AVCodecHWConfig config = avcodec_get_hw_config(decoder, index);

will return now a Vulkan hardware decoder which can be used.

However, the thing is that it only works when there are ffmpeg headers available during the build process otherwise the build will fail with

ERROR: vulkan requested but not found

I have a test build repo available for ffmpeg (independant from javacv) and in order to build it locally with vulkan support I needed to install following libs via msys2/ mingw

        mingw-w64-${{ matrix.env }}-vulkan-headers
        mingw-w64-${{ matrix.env }}-vulkan-loader

I also did this on my local build machine where I build javacv/ javacpp-presets and the build was successful. I only compiled it for Windows x86_64 though and as the build process is quite complex for javacpp-preset I don't know what is the proper way to include these vulkan libs/ headers in the build process so that builds for other architectures will pass as well? Maybe you can give me some adives, I'm more than happy to do that as this feature will be highly requested in the future I guess. Furthermore, is there an easy way to test the build to check if every architecture was build correctly for javacpp-presets/ ffmpeg? I don't have these many test machines to verify every architecture? Using GitHub actions maybe?

[saudet]

Yes, that's what GitHub Actions is for. Please update the scripts for that

[streamingdv]

Yeah as I thought, the github actions will try to publish to maven but this will fail of course but as long as the main build passes I guess that's enough for verify that builds went through, right? So I guess the github action script I'm interested in is

ffmpeg.yml

correct? And are there any similar example which in the action script already which is installing some required libs/ header I can look as an example?

[saudet]

The scripts are here: https://github.com/bytedeco/javacpp-presets/tree/master/.github/actions

[streamingdv]

Thanks, I will check it out. As soon as all the builds are passing I will make a pull request. I'm in contact with the Chiaki devs to get some feedback, they have already a working vulkan rendering engine so I guess they know what is needed maybe for other architectures.

[streamingdv]

@saudet I made some progress and to me it looks like the builds are going through now

https://github.com/streamingdv/javacpp-presets/actions/runs/7380740283

All GPL builds however are failing, but I think it's not related to my changes? Seems to be some Sonatype issue but not sure?

Will still need to test it a bit more.

[saudet]

Please add --enable-vulkan to the ENABLE list

[streamingdv]

@saudet yes I will, this was just because of the testing. It is currently enabled in specific builds but I will move it to the general section

[streamingdv]

I enabled Vulkan now for all builds but I have still problems to build the ffmpeg libs for Android, I always get the error

ERROR: vulkan requested but not found

even though I installed the SDK in the Ubuntu environment (and it usually works for other Linux builds), I have no idea if I have to do something else for gradle in order to make it work. However recenetly I have another problem, all of a sudden certain linux packages can't be downloaded anymore. E.g. linux-arm64 and linux-ppc64le

https://github.com/streamingdv/javacpp-presets/actions/runs/7390359065/job/20105096160 https://github.com/streamingdv/javacpp-presets/actions/runs/7390359065/job/20105095254

Maybe it is a temporary issue, or is this already a known issue?

[saudet]

linux-arm64 and linux-ppc64le have temporary network issues yes

[saudet]

I don't know what we have to do for Android

[streamingdv]

@saudet okay, will try to figure out what needs to be done on Android

[streamingdv]

@saudet I did some research and it seems you have to upgrade to at least NDK 25 in order to work with Vulkan

https://developer.android.com/games/optimize/vulkan-get-started?fbclid=IwAR10IN8euE8UOxrF5d4Xrk3Yq4r0eVkeJ3NN86RYHWNoLtr8zGPaDFmo4VE#vulkan-api

Unfortunetly when I ugrade to NDK 25 building OPENSSL will fail with following errors

https://github.com/streamingdv/javacpp-presets/actions/runs/7482282035/job/20365542677

It seems there is a openssl-android.patch could it be that this patch must be updated? I don't know what else to do at the moment, all other builds seem to work.

[streamingdv]

@saudet I'm running out of ideas unfortunetly, the Android builds are not able to find vulkan, I tried various things now but I can't find a solution. All other builds seem to pass. Can I at least make a pull request which enables vulkan for all builds except Android? I know that's not ideal but not sure what else to try and at least we have vulkan then on linux, windows and mac?

[saudet]

Sure, you can open a pull request as WIP

[streamingdv]

Okay great, what is the preferred way then to include these enable args

--enable-vulkan --enable-hwaccel=h264_vulkan --enable-hwaccel=hevc_vulkan --enable-hwaccel=av1_vulkan

Should I add an extra variable like ENABLE_VULKAN below the existing ENABLE variable in ffmpeg\cppbuild.sh? This variable will then get passes to all other ffmpeg configure args except Android builds?

[saudet]

It doesn't really matter, it's not going to get merged until that works anyway

[streamingdv]

Well that was the whole point of the question ^^ If it doesn't get merged then no need to create this intermediate pull request to be honest. I will make one in the hope I get some support. I have no idea what else could be needed for the Android builds, I tried to upgrade the ndk version but this leads to errors when building OpenSSL, directly linking the vulkan header dirs and libs in the configure step of ffmpeg did not work either.

[streamingdv]

WIP: pull request is here

https://github.com/bytedeco/javacpp-presets/pull/1460/files

[streamingdv]

@saudet just a question, is there any specifc reason why Windows x86 is disabled for the ffmpeg builds? I just tried recently and it compiled successfully? I know 32 bit is maybe outdated but having the option would not hurt?

https://github.com/bytedeco/javacpp-presets/blob/bfdd4943b248dc8df1e56c240bacc4640da06b0d/.github/workflows/ffmpeg.yml#L98

[saudet]

You can compile and use those by yourself if you need, that's not a problem

[streamingdv]

Yes I know :) It was just a question if it should be re-enabled or if it is like that on purpose and x86 is no longer officially supported for ffmpeg?

[saudet]

I'm not aware of any users, so...

[saudet]

Released with version 1.5.10! Thanks for the contribution