Closed philipl closed 4 months ago
AMD released new preview driver for Windows with VK_KHR_videodecode* support among other things: AMD Software: Adrenalin Edition 23.10.01.41 for Additional Vulkan Extensions Release Notes
I tried on 5700 XT (RDNA) and it basically causes device reset with both h264 and hevc. YMMV with different hardware. AMD Driver: 23.10.01.41 mpv: https://github.com/mpv-player/mpv/commit/e7902eb83dc68dcb0adb99568d72bd3f9a4e4550 FFmpeg: https://github.com/FFmpeg/FFmpeg/commit/6ff95758e6bf47004f69c56d7547a971c4782b21 libplacebo: https://github.com/haasn/libplacebo/commit/8ceb7a544b70aff7e432cbb9708666115a6cb985
Some logs:
AMD released new preview driver with VK_KHR_videodecode* among other things: AMD Software: Adrenalin Edition 23.10.01.41 for Additional Vulkan Extensions Release Notes
Just tried it... EDIT: I'm blind, still no support for RX570, it was falling back to software... Still doesn't have the extension.
Could you/anyone else with the driver post the log of --msg-level=ffmpeg=debug?
There is not much there really, it just freeze on queue submit, reset device and throws error.
Thanks, I don't see anything bad. Guess they didn't test their driver well at all on anything but the utterly useless "example" program. Do report it to them if you can.
I also tested on RDNA2 integrated GPU, same result.
23.7.1 has released with Vulkan HW Decode but still no support on Polaris :(
23.7.1 has released with Vulkan HW Decode but still no support on Polaris :(
It doesn't work on supported hardware either =)
I've told AMD it's broken, hopefully they'll fix it.
23.7.1 has released with Vulkan HW Decode but still no support on Polaris :(
It doesn't work on supported hardware either =)
can confirm doesn't work on my supported igpu either
Got this working on Debian on Radeon RX 580 (POLARIS10) with Mesa RADV driver with the commits stated in the OP, except that I added MR 22202 and MR 23304 on top of Debian's Mesa 23.1.3 rather than using upstream mesa master branch directly.
Anyway, it works with 1080p HD video but not 8K video:
$ ffprobe PXL_[..].mp4
[..]
Stream #0:1[0x2](eng): Video: hevc (Main) (hev1 / 0x31766568), yuvj420p(pc, bt470bg/bt470bg/smpte170m), 1920x1080, 12041 kb/s, SAR 1:1 DAR 16:9, 30.18 fps, 60 tbr, 90k tbn (default)
[..]
$ ./build/mpv [..]
[..]
[vd] Trying hardware decoding via hevc-vulkan.
[vd] Selected codec: hevc (HEVC (High Efficiency Video Coding))
[..]
[vd] Pixel formats supported by decoder: vaapi vdpau cuda vulkan yuvj420p
[vd] Codec profile: Main (0x1)
[ffmpeg/video] hevc: Decoder capabilities for hevc profile "Main":
[ffmpeg/video] hevc: Maximum level: 51 (stream 150)
[ffmpeg/video] hevc: Width: from 16 to 4096
[ffmpeg/video] hevc: Height: from 16 to 4096
[ffmpeg/video] hevc: Width alignment: 16
[ffmpeg/video] hevc: Height alignment: 16
[ffmpeg/video] hevc: Bitstream offset alignment: 128
[ffmpeg/video] hevc: Bitstream size alignment: 128
[ffmpeg/video] hevc: Maximum references: 17
[ffmpeg/video] hevc: Maximum active references: 8
[ffmpeg/video] hevc: Codec header version: 1.0.0 (driver), 1.0.0 (compiled)
[ffmpeg/video] hevc: Decode modes: dedicated_dpb
[ffmpeg/video] hevc: Capability flags: none
[ffmpeg/video] hevc: Chosen frame pixfmt: nv12 (Vulkan ID: 1000156003)
[vd] Requesting pixfmt 'vulkan' from decoder.
[ffmpeg/video] hevc: Allocating 131072 bytes in bind index 0 for video session
[ffmpeg/video] hevc: Allocating 21204992 bytes in bind index 1 for video session
[ffmpeg/video] hevc: Vulkan decoder initialization sucessful
[ffmpeg/video] hevc: Decoding frame, 203715 bytes, 3 slices
Using hardware decoding (vulkan).
[vd] Decoder format: 1920x1080 [0:1] vulkan[nv12] bt.601/bt.601-625/bt.1886/full/auto CL=mpeg2/4/h264
[vd] Using container aspect ratio.
[..]
$ ffprobe GX01_[..].mp4
[..]
Stream #0:0[0x1](eng): Video: hevc (Main) (hvc1 / 0x31637668), yuvj420p(pc, bt709), 5312x2988 [SAR 1:1 DAR 16:9], 59867 kb/s, 29.97 fps, 29.97 tbr, 30k tbn (default)
[..]
$ ./build/mpv [..]
[..]
[vd] Trying hardware decoding via hevc-vulkan.
[vd] Selected codec: hevc (HEVC (High Efficiency Video Coding))
[..]
[vd] Pixel formats supported by decoder: vaapi vdpau cuda vulkan yuvj420p
[vd] Codec profile: Main (0x1)
[ffmpeg/video] hevc: Decoder capabilities for hevc profile "Main":
[ffmpeg/video] hevc: Maximum level: 51 (stream 180)
[ffmpeg/video] hevc: Width: from 16 to 4096
[ffmpeg/video] hevc: Height: from 16 to 4096
[ffmpeg/video] hevc: Width alignment: 16
[ffmpeg/video] hevc: Height alignment: 16
[ffmpeg/video] hevc: Bitstream offset alignment: 128
[ffmpeg/video] hevc: Bitstream size alignment: 128
[ffmpeg/video] hevc: Maximum references: 17
[ffmpeg/video] hevc: Maximum active references: 8
[ffmpeg/video] hevc: Codec header version: 1.0.0 (driver), 1.0.0 (compiled)
[ffmpeg/video] hevc: Decode modes: dedicated_dpb
[ffmpeg/video] hevc: Capability flags: none
[vd] Hardware decoding of this stream is unsupported?
[vd] Requesting pixfmt 'yuvj420p' from decoder.
[af] [in] 48000Hz stereo 2ch floatp
[af] [userspeed] 48000Hz stereo 2ch floatp
[af] [userspeed] (disabled)
[af] [convert] 48000Hz stereo 2ch floatp
[vd] Falling back to software decoding.
[..]
[.. CPU fan goes brrrrrrrr... shit slows down ... desync ..]
AV: 00:00:28 / 00:01:57 (24%) A-V: 0.473 Dropped: 134
Audio/Video desynchronisation detected! Possible reasons include too slow
hardware, temporary CPU spikes, broken drivers, and broken files. Audio
position will not match to the video (see A-V status field).
[..]
Actually even with --hwdec=vaapi
I got:
[ffmpeg/video] hevc: Hardware does not support image size 5312x2992 (constraints: width 0-4096 height 0-4096).
Is there any way to make my 8K video play on my GPU, can it be fixed in software/firmware, are we waiting for future code to make it work, or is it just physically not supported by my hardware Radeon RX 580 (POLARIS10)?
Is there any way to make my 8K video play on my GPU, can it be fixed in software/firmware, are we waiting for future code to make it work, or is it just physically not supported by my hardware Radeon RX 580 (POLARIS10)?
No. Your hardware is limited to decoding up to 4K, which is to be expected from that hardware generation.
Damn that's a shame. Now to think about whether I want to fork out another $400 on a graphics card. :(
My video is actually 5.3k x 3k px. Not even a theoretical hacky way to make it "fit" into the 4k x 4k space limited by the existing old card?
Damn that's a shame. Now to think about whether I want to fork out another $400 on a graphics card. :(
My video is actually 5.3k x 3k px. Not even a theoretical hacky way to make it "fit" into the 4k x 4k space limited by the existing old card?
No. You can't do that - the layout is part of the encoded structure. As for hardware, you can get a significant upgrade moving to a 6700xt or 6750xt for significantly less than $400.
@infinity0 Interesting. But that's old article. What's the development status of it now, if you know ?
that's not me, you just pinged a random guy Outta the blue
23.7.1 has released with Vulkan HW Decode but still no support on Polaris :(
I tried to use it, but my computer froze for a few minutes when I started playing a video, then restarted. (7900XTX)
Did it actually decode something?
I seriously doubt that Intel and AMD released the Vulkan Video driver without testing it at all and everything doesn't work.
Prepare to be very surprised then, it also doesn't work on my 6600 XT on Windows fwiw
I seriously doubt that Intel and AMD released the Vulkan Video driver without testing it at all and everything doesn't work.
Apparently they betrayed your expectations?
a new libplacebo is already released and ffmpeg will cut a new release soonish, may as well update all of them at once. Mesa should also contain all the necessary parts as of 23.2
I don't see --hwdec=vulkan
in the latest release, but it can decode with VAAPI.
Nvidia
Both H.264 and H.265 are supported
Vulkan filters are supported, but not for streams that are decoded with Vulkan video decoding (but of course).
Filters willl work with content that is uploaded to Vulkan after being decoded by nvdec or software decoding.
eg:
mpv --vo=gpu-next --gpu-api=vulkan --hwdec=nvdec --vf=format=vulkan
'hwdec=vulkan' worked for me using Nvidia driver 536.67 on Windows running mpv 0.36 built with ffmpeg version git-2023-07-30.
'hwdec=vulkan' worked for me using Nvidia driver 536.67 on Windows running mpv 0.36 built with ffmpeg version git-2023-07-30.
Glad to hear. We'll have to see when we get a Linux driver release with the fix. I haven't been able to try 535.86.05 yet.
The vulkan hwdec on nvidia has always worked, the problem is that the vk hwdec+filter never works. Not sure if the fix is included in R545, my GPU PCI ID is not in there to test it
The 'hwdec=vulkan' command did not work before. Vulkan as detailed in this user guide only worked by using the 'hwdec=nvdec' workaround. As far as I can tell, everything else works. Is there anything in particular to check?
vulkan hwdec is always available since 535.98, but the vk filters don't work.
set vf hflip_vulkan
, if everything works, you should see the video has been flipped
I knew my driver (earlier than .98) was an issue after I upgraded it which is why I specified it in my post. Anyway "hflip" works under hwdec_vulkan. "hflip_vulkan" does not.
Vulkan filters are supported, but not for streams that are decoded with Vulkan video decoding (but of course).
hflip
is irrelevant. Because what you quoted is tallking about vulkan-filter. hflip_vulkan
is a vulkan-filter while hflip
is not.
Vulkan filters are supported, but not for streams that are decoded with Vulkan video decoding (but of course).
hflip
is irrelevant. Because what you quoted is tallking about vulkan-filter.hflip_vulkan
is a vulkan-filter whilehflip
is not.
Error message: [ffmpeg] Impossible to convert between the formats supported by the filter 'mpv_scr_default_in' and the filter 'auto_scale_0'
I seriously doubt that Intel and AMD released the Vulkan Video driver without testing it at all and everything doesn't work.
Apparently they betrayed your expectations?
Sure, at first, I thought Vulkan Video was the future, but look at it now, Google explicitly banned Vulkan Video in the Android CDD, Chrome Media said they weren't interested in it, there are countless problems with all the Vulkan Video drivers on Windows, Khronos is still delaying AV1 and VP9, and mesa is the only one working properly.
Please can you reference the explicit "ban"?
Also for Chrome Media? I understand there could be nice wins for VK Video on mobile when implementations start to appear there. Chromium at least seem they would be open to welcoming patches: https://bugs.chromium.org/p/chromium/issues/detail?id=1200559
Khronos isn't delaying AV1 or VP9, AV1 is in progress (there exists a Mesa extension already), and VP9 is on the future roadmap.
Please can you reference the explicit "ban"?
https://source.android.com/docs/compatibility/13/android-13-cdd#7142_vulkan
[C-1-11] MUST NOT enumerate support for the VK_KHR_video_queue, VK_KHR_video_decode_queue, or VK_KHR_video_encode_queue extensions.
Vulkan filters are supported, but not for streams that are decoded with Vulkan video decoding (but of course).
hflip
is irrelevant. Because what you quoted is tallking about vulkan-filter.hflip_vulkan
is a vulkan-filter whilehflip
is not.
hflip_vulkan
doesn't seem to exist for me. :thinking:
I do meet requirements set by mpv for Vulkan Video decoding for Nvidia on Linux.
Or is the hwdec=nvdec
still in the realm of yank, while hwdec=vulkan
is supported in 0.36.0?
https://github.com/FFmpeg/FFmpeg/commit/c0709706ddda0d1f079d4020047bcdeb0bc4a7f1 may have fixed some H264 streams on Nvidia
@arm64-v9a I have the same issue and that's reason number 2 why I don't use hwdec=vulkan
(I didn't get to test the new patch, will test it tomorrow if there's a build available to download)
Is there an open issue regarding this specific case? I would like to follow up on this.
The other issue I don't use vulkan is the stuttering issue I've mentioned to you before. (This reminds me - try to reproduce that stuttering issue with OSD turned on, I did not see you mention anything about OSD)
FFmpeg/FFmpeg@c070970 may have fixed some H264 streams on Nvidia
This sample still has problems
!!!! This sample is 2s cut from the beginning of an adult movie and is not NSFW, I couldn't find any other samples with this problem, but it's still recommended to check it without other people around !!!!
What exactly is the problem you are seeing? The sample decodes fine on my GTX 3050, both with FFmpeg and the NVIDIA Vulkan sample decoder.
FFmpeg here is at 2cd09fec9f8d3b16d03e762dec6001cc0b792659
.
Command line to verify,
ffmpeg_g -hide_banner -loglevel verbose -y -init_hw_device vulkan=vk:0,debug=0 -hwaccel_output_format vulkan -hwaccel vulkan -i /home/cturner/fail.webm -fps_mode passthrough -vf hwdownload,format=nv12,format=pix_fmts=yuv420p /tmp/ffmpeg.yuv
FFmpeg/FFmpeg@c070970 may have fixed some H264 streams on Nvidia
This sample still has problems !!!! This sample is 2s cut from the beginning of an adult movie and is not NSFW, I couldn't find any other samples with this problem, but it's still recommended to check it without other people around !!!!
What exactly is the problem you are seeing? The sample decodes fine on my GTX 3050, both with FFmpeg and the NVIDIA Vulkan sample decoder. FFmpeg here is at
2cd09fec9f8d3b16d03e762dec6001cc0b792659
. Command line to verify,ffmpeg_g -hide_banner -loglevel verbose -y -init_hw_device vulkan=vk:0,debug=0 -hwaccel_output_format vulkan -hwaccel vulkan -i /home/cturner/fail.webm -fps_mode passthrough -vf hwdownload,format=nv12,format=pix_fmts=yuv420p /tmp/ffmpeg.yuv
You can of course decode this video without errors, but you will notice that the output is flawed, there is an image below my sample, if you play it with
mpv --hwdec=vulkan
, it should have the same jagged grid as my image.
Ah, apologies I only checked the output with the NVIDIA sample decoder, which doesn't have the jagged grid here. I also notice your problem with FFmpeg. I'll look into what's going wrong there.
Edit: Tested today's build with the new patch and the "jagged grid" issue is still an issue.
Edit: Tested today's build with the new patch and the "jagged grid" issue is still an issue.
I checked and this is an issue with FFmpeg's AVC parser, specifically it appears to be parsing scaling lists incorrectly still, they are the biggest difference compared to reference.
There was another patch yesterday to fix h.264 scaling list order. Do you think there is still a bug after that?
FFmpeg/FFmpeg@c070970 may have fixed some H264 streams on Nvidia
This sample still has problems
Just checked again. Looks fine on Intel. Looks wrong on nvidia (535.98). That means it's presumably a driver issue.
AMD on Windows appears to be 5000/6000 series according to https://www.amd.com/en/support/kb/release-notes/rn-rad-win-22-11-2-vlk-video-code-decode (possibly 7000 works?). It doesn't work on Polaris:
0.339][e][ffmpeg/video] h264: Device does not support the VK_KHR_video_decode_queue extension!
5300u also writes that there is no support)
5300u also writes that there is no support)
I was referring to 5000/6000 series GPUs not CPUs. Your CPU has Vega integrated which was before the 5000 series of GPUs.
NVIDIA released a new driver last night (237.13), but it seems there are still a few kinks to work out:
Is that H264?
Does anyone have an h.264 sample that exhibits the blockiness/jaggedness on nvidia - I can't find one locally. There's another scaling list order fix for ffmpeg I need to test out. Note that it's weird that it would work fine on Intel and not nvidia but given that the scaling order thing slipped through in ffmpeg, it might also be wrong inside the Intel driver and that's what it works out.
Does anyone have an h.264 sample that exhibits the blockiness/jaggedness on nvidia - I can't find one locally. There's another scaling list order fix for ffmpeg I need to test out. Note that it's weird that it would work fine on Intel and not nvidia but given that the scaling order thing slipped through in ffmpeg, it might also be wrong inside the Intel driver and that's what it works out.
I can confirm the sample image I posted before was from an H.264 source. Odly, the issue only seems to affect the remux I have from my JP disk, as my ITA one playes perfectly (and both work perfectly while using nvdec). As for sample files, I hope these help (they are < 30 second adverts included with the bonus disk, so I am sure no one will mind).
I can confirm the sample image I posted before was from an H.264 source. Odly, the issue only seems to affect the remux I have from my JP disk, as my ITA one playes perfectly (and both work perfectly while using nvdec). As for sample files, I hope these help (they are < 30 second adverts included with the bonus disk, so I am sure no one will mind).
Bingo. Ben Cheng's zig-zag fix makes this play correctly on nvidia. Although I cannot test immediately, that almost certainly means his fix will break Intel because the driver itself uses the wrong order wrt to the spec.
@cyanreg
amd 23.8.1 driver(on windows) dropped, can't check it right now to see if vulkan hwdec is working, can anyone else here check please?
Just pushed the scaling list fixes to FFmpeg, which should fix H264 on nvidia, and maybe on AMD.
amd 23.8.1 driver(on windows) dropped, can't check it right now to see if vulkan hwdec is working, can anyone else here check please?
still not working, confirmed
Vulkan Video Decoding: Usage Guide and FAQ
On the 28th of May, we reached the significant milestone of finally merging all the required functionality into ffmpeg, libplacebo, and mpv to do end-to-end Vulkan video decoding and presentation. Although the functionality is now all there, we still have a complex landscape in terms of what is supported on what hardware and with what drivers. This document attempts to lay out all the requirements and limitations, so you have a chance of successfully using the feature.
Why should I even care about Vulkan video decoding?
It's a fair question. Right now, the actual functionality you gain access to is not terribly different from what you would get with existing video decoding APIs that ffmpeg and mpv already support. However, Vulkan video decoding has the potential to be a credible cross-vendor, cross-platform API that is well supported, and can work efficiently with Vulkan based filtering and post-processing. In the short term, there will be rough edges, and only a narrow set of supported codecs (only H.264 and H.265 are standardised today, with AV1 still a work in progress), but in the future, it will hopefully be the most performant and capable way to do hardware decoding and processing.
Software Requirements
We have finally reached the point where all the required components have made official releases. Ensure you are running with the following releases or newer:
These requirements mean you will likely have to compile everything for yourself (especially on Linux), but Shinchiro's windows builds appear to be functional.
Hardware and Driver Requirements
Usage
The basic command line arguments:
Depending on your system configuration, you may additionally need to force the right
gpu-context
:--gpu-context=winvk
--gpu-context=x11vk
--gpu-context=waylandvk
Capabilities
ANV_VIDEO_DECODE=1
in your environment to expose video decodingRADV_PERFTEST=video_decode
in your environment to expose video decoding