search

nvpro-samples / vk_video_samples

Vulkan video samples
Apache License 2.0
237 stars 40 forks source link

Encoder: RADV not supported #71

Open dabrain34 opened 3 months ago

dabrain34 commented 3 months ago

When trying to encode using RADV encoder, vk-video-enc-test fails with:

Selected codec: avc
Input file size is: 1140480
No output file name provided. Using out.264.
No QP was provided. Using default value: 20.
The clock resolution of high_resolution_clock is: 1e-09
The clock resolution of steady_clock is: 1e-09
The clock resolution of system_clock is: 1e-09
HasAllDeviceExtensions : WARNING: requested optional device extension VK_EXT_ycbcr_2plane_444_formats is missing for device with name: AMD Radeon RX 7600 (RADV NAVI33)
     Found transfer only queue family 0 with 1 max num of queues.
ERROR: Found physical device with name: AMD Radeon RX 7600 (RADV NAVI33), vendor ID: 1002, and device ID: 7480 NOT having the required queue families!
vk-video-enc-test: /DEV/vk_video_samples/vk_video_encoder/demos/vk-video-enc/Main.cpp:253: int main(int, char**): Assertion `!"Can't initialize the Vulkan physical device!"' failed.
Aborted (core dumped)
charlie-ht commented 3 months ago

Hi @dabrain34 - https://github.com/nvpro-samples/vk_video_samples/pull/57 originally fixed this, and also more recently https://github.com/nvpro-samples/vk_video_samples/pull/69 - hopefully #69 can be merged soon to fix this issue for encode (although it exists in the decoder too)

zlatinski commented 3 months ago

Yes, we need a separate transfer queue for both AMD and Intel implementations. Can you please try the ToT. It should be fixed now.

dabrain34 commented 3 months ago

Since the last update (20240329) I get this kind of error with AMD(RADV) and INTEL(ANV)

176x144_30_i420.yuv --inputWidth 176 --inputHeight 144 --startFrame 0 --numFrames 5
Selected codec: avc
Input file size is: 1140480
No output file name provided. Using out.264.
No QP was provided. Using default value: 20.
The clock resolution of high_resolution_clock is: 1e-09
The clock resolution of steady_clock is: 1e-09
The clock resolution of system_clock is: 1e-09
HasAllDeviceExtensions : WARNING: requested optional device extension VK_EXT_ycbcr_2plane_444_formats is missing for device with name: Intel(R) Graphics (RPL-S)
HasAllDeviceExtensions : WARNING: requested optional device extension VK_EXT_descriptor_buffer is missing for device with name: Intel(R) Graphics (RPL-S)
     Found transfer only queue family 0 with 1 max num of queues.
     Found video encode only queue family 1 with 2 max num of queues.
*** Selected Vulkan physical device with name: Intel(R) Graphics (RPL-S), vendor ID: 8086, and device ID: a780, Num Decode Queues: 0, Num Encode Queues: 2 ***
            encode h.264encode capabilities: 
            minBitstreamBufferOffsetAlignment: 32
            minBitstreamBufferSizeAlignment: 4096
            pictureAccessGranularity: 16 x 16
            minExtent: 16 x 16
            maxExtent: 4096 x 4096
            maxDpbSlots: 16
            maxActiveReferencePictures: 16

GOP frame count: 16, IDR period: 60, Consecutive B frames: 3

Display order:   0   1   2   3   4   5   6   7   8   9  10  11  12  13  14  15  16  17  18  19  20 
Frame Type:    IDR   B   B   B   P   B   B   B   P   B   B   B   P   B   B   B   I   B   B   B   P
Decode  order:   0   2   3   4   1   6   7   8   5  10  11  12   9  14  15  16   0   2   3   4   1 
vk-video-enc-test: ../src/vulkan/runtime/vk_image.c:452: vk_image_view_init: Assertion `image_view->format == image->format' failed.
Aborted (core dumped)
zlatinski commented 3 months ago

Thank you, Stephane! Can you please tell us if this is happening during the pre-processing stage or the encode stage? An applicaiton call- stack would be very helpful.

dabrain34 commented 3 months ago

The assert is located in the radv driver, here is the backtrace:

#6  0x00007ffff5239206 in __assert_fail
    (assertion=0x7fffebb0d010 "image_view->format == image->format", file=0x7fffebb0cbc8 "../src/vulkan/runtime/vk_image.c", line=452, function=0x7fffebb0d5c0 <__PRETTY_FUNCTION__.8> "vk_image_view_init") at ./assert/assert.c:101
#7  0x00007fffeb4c9a6b in vk_image_view_init (device=0x5555557a95f0, image_view=0x5555557610a0, driver_internal=false, pCreateInfo=0x7fffffffc570)
    at ../src/vulkan/runtime/vk_image.c:452
#8  0x00007fffeb3b5e54 in radv_image_view_init (iview=0x5555557610a0, device=0x5555557a95f0, pCreateInfo=0x7fffffffc570, img_create_flags=0, extra_create_info=0x7fffffffc4fc)
    at ../src/amd/vulkan/radv_image_view.c:754
#9  0x00007fffeb3b6893 in radv_CreateImageView (_device=0x5555557a95f0, pCreateInfo=0x7fffffffc570, pAllocator=0x0, pView=0x7fffffffc5c8) at ../src/amd/vulkan/radv_image_view.c:925
#10 0x00005555555d72ab in VkImageResourceView::Create (vkDevCtx=0x7fffffffcb50, imageResource=..., imageSubresourceRange=..., imageResourceView=...)
    at /DEV/vk_video_samples/common/libs/VkCodecUtils/VkImageResource.cpp:194
#11 0x00005555555d7c28 in VulkanVideoImagePoolNode::CreateImage
    (this=0x555556192e30, vkDevCtx=0x7fffffffcb50, pImageCreateInfo=0x555555ecda40, requiredMemProps=14, imageIndex=0, imageArrayParent=..., imageViewArrayParent=..., useLinear=true)
    at /DEV/vk_video_samples/common/libs/VkCodecUtils/VulkanVideoImagePool.cpp:84
#12 0x00005555555d88ea in VulkanVideoImagePool::Configure
    (this=0x555555ecd9a0, vkDevCtx=0x7fffffffcb50, numImages=16, imageFormat=VK_FORMAT_G8_B8R8_2PLANE_420_UNORM, maxImageExtent=..., imageUsage=13, queueFamilyIndex=4, requiredMemProps=14, pVideoProfile=0x0, useImageArray=false, useImageViewArray=false, useLinearImage=true)
    at /DEV/vk_video_samples/common/libs/VkCodecUtils/VulkanVideoImagePool.cpp:325
#13 0x00005555555a3d2e in VkVideoEncoder::InitEncoder (this=0x555556191d10, encoderConfig=...)
    at/DEV/vk_video_samples/vk_video_encoder/libs/VkVideoEncoder/VkVideoEncoder.cpp:376
#14 0x000055555558a481 in VkVideoEncoderH264::InitEncoderCodec (this=0x555556191d10, encoderConfig=...)
    at /DEV/vk_video_samples/vk_video_encoder/libs/VkVideoEncoder/VkVideoEncoderH264.cpp:48
#15 0x000055555558a2cf in CreateVideoEncoderH264 (vkDevCtx=0x7fffffffcb50, encoderConfig=..., encoder=...)
    at /DEV/vk_video_samples/vk_video_encoder/libs/VkVideoEncoder/VkVideoEncoderH264.cpp:27
#16 0x00005555555a2152 in VkVideoEncoder::CreateVideoEncoder (vkDevCtx=0x7fffffffcb50, encoderConfig=..., encoder=...)
    at /DEV/vk_video_samples/vk_video_encoder/libs/VkVideoEncoder/VkVideoEncoder.cpp:32
#17 0x000055555557b300 in main
dabrain34 commented 3 months ago

Here is the version for ANV:

#5  0x00007ffff52267db in __assert_fail_base
    (fmt=0x7ffff53c5168 "%s%s%s:%u: %s%sAssertion `%s' failed.\n%n", assertion=assertion@entry=0x7fffeb943988 "image_view->format == image->format", file=file@entry=0x7fffeb943600 "../src/vulkan/runtime/vk_image.c", line=line@entry=452, function=function@entry=0x7fffebb190e0 <__PRETTY_FUNCTION__.8> "vk_image_view_init") at ./assert/assert.c:92
#6  0x00007ffff5239206 in __assert_fail
    (assertion=assertion@entry=0x7fffeb943988 "image_view->format == image->format", file=file@entry=0x7fffeb943600 "../src/vulkan/runtime/vk_image.c", line=line@entry=452, function=function@entry=0x7fffebb190e0 <__PRETTY_FUNCTION__.8> "vk_image_view_init") at ./assert/assert.c:101
#7  0x00007fffeb36b6fc in vk_image_view_init
    (device=device@entry=0x5555557545d0, image_view=image_view@entry=0x5555557355c0, driver_internal=driver_internal@entry=false, pCreateInfo=pCreateInfo@entry=0x7fffffffc680)
    at ../src/vulkan/runtime/vk_image.c:452
#8  0x00007fffea6da4f0 in anv_image_view_init
    (device=device@entry=0x5555557545d0, iview=iview@entry=0x5555557355c0, pCreateInfo=pCreateInfo@entry=0x7fffffffc680, surface_state_stream=surface_state_stream@entry=0x0)
    at ../src/intel/vulkan/anv_image.c:3501
#9  0x00007fffea6dadfb in anv_CreateImageView (_device=0x5555557545d0, pCreateInfo=0x7fffffffc680, pAllocator=<optimized out>, pView=0x7fffffffc6d8)
    at ../src/intel/vulkan/anv_image.c:3656
#10 0x00005555555d72ab in VkImageResourceView::Create (vkDevCtx=0x7fffffffcc60, imageResource=..., imageSubresourceRange=..., imageResourceView=...)
    at /DEV/vk_video_samples/common/libs/VkCodecUtils/VkImageResource.cpp:194
#11 0x00005555555d7c28 in VulkanVideoImagePoolNode::CreateImage
    (this=0x555555733db0, vkDevCtx=0x7fffffffcc60, pImageCreateInfo=0x55555570b540, requiredMemProps=14, imageIndex=0, imageArrayParent=..., imageViewArrayParent=..., useLinear=true)
    at /DEV/vk_video_samples/common/libs/VkCodecUtils/VulkanVideoImagePool.cpp:84
#12 0x00005555555d88ea in VulkanVideoImagePool::Configure
    (this=0x55555570b4a0, vkDevCtx=0x7fffffffcc60, numImages=16, imageFormat=VK_FORMAT_G8_B8R8_2PLANE_420_UNORM, maxImageExtent=..., imageUsage=13, queueFamilyIndex=1, requiredMemProps=14, pVideoProfile=0x0, useImageArray=false, useImageViewArray=false, useLinearImage=true)
    at /DEV/vk_video_samples/common/libs/VkCodecUtils/VulkanVideoImagePool.cpp:325
#13 0x00005555555a3d2e in VkVideoEncoder::InitEncoder (this=0x555555778ee0, encoderConfig=...)
    at /DEV/vk_video_samples/vk_video_encoder/libs/VkVideoEncoder/VkVideoEncoder.cpp:376
#14 0x000055555558a481 in VkVideoEncoderH264::InitEncoderCodec (this=0x555555778ee0, encoderConfig=...)
    at /DEV/vk_video_samples/vk_video_encoder/libs/VkVideoEncoder/VkVideoEncoderH264.cpp:48
#15 0x000055555558a2cf in CreateVideoEncoderH264 (vkDevCtx=0x7fffffffcc60, encoderConfig=..., encoder=...)
    at /DEV/vk_video_samples/vk_video_encoder/libs/VkVideoEncoder/VkVideoEncoderH264.cpp:27
#16 0x00005555555a2152 in VkVideoEncoder::CreateVideoEncoder (vkDevCtx=0x7fffffffcc60, encoderConfig=..., encoder=...)
    at /DEV/vk_video_samples/vk_video_encoder/libs/VkVideoEncoder/VkVideoEncoder.cpp:32
#17 0x000055555557b300 in main (argc=13, argv=0x7fffffffd8f8) at /home/scerveau/DEV/IGALIA/PROJECTS/VALVE/DEV/vk_video_samples/vk_video_encoder/demos/vk-video-enc/Main.cpp:273
zlatinski commented 3 months ago

Thank you, for the backtrace, Stephane! It looks like the open-source driver does not support creating individual views against YCbCr image. We'll have to deal with this by using properties, but in the mean time, you can patch the sample app by removing the code that creates individual planes for the resource. It is currently only used for the YCbCr filter. In VkImageResourceView::Create() .... const VkMpFormatInfo* mpInfo = YcbcrVkFormatInfo(viewInfo.format);

I'll add this as an explicit option, but I would highly recommend that drivers support the creation of views against individual planes. I'm not even sure if this is not required by the spec.

However, the issue here could also be that the encoder is requesting the VK_IMAGE_USAGE_SAMPLED_BIT bit. Alternately, in VkVideoEncoder::InitEncoder() : line 376

result = m_linearInputImagePool->Configure(... Can you please try replacing: ( VK_IMAGE_USAGE_SAMPLED_BIT | VK_IMAGE_USAGE_STORAGE_BIT | VK_IMAGE_USAGE_TRANSFER_SRC_BIT),

with VK_IMAGE_USAGE_TRANSFER_SRC_BIT?

Essentially, removing the VK_IMAGE_USAGE_SAMPLED_BIT and VK_IMAGE_USAGE_STORAGE_BIT.

Sorry, but I do not have a setup with the open source driver and will not be able to try it.

lolzballs commented 3 months ago

@dabrain34 @zlatinski, I took at look at the location of that assert. It will trigger if the VkImage was not created with VK_IMAGE_CREATE_MUTABLE_FORMAT_BIT.

I was able to work around this by setting VkImageCreateInfo::flags = VK_IMAGE_CREATE_MUTABLE_FORMAT_BIT here. Obviously to be entirely correct the app needs to check if the implementation supports this flag for the format, but this was enough as a work-around for now.

After that, RADV (and ANV as well probably) segfaults in vkQueueSubmit. pWaitDstStageMask has to be a pointer to an array of waitSemaphoreCount masks, but it is always nullptr here.

zlatinski commented 3 months ago

Thank you, lolzballs!

I was able to work around this by setting VkImageCreateInfo::flags = VK_IMAGE_CREATE_MUTABLE_FORMAT_BIT

Yes, indeed, this is the correct use when the individual planes of YCbCr MP formats are used. There is no need to check for support for that, because this feature is required for MP formats.

After that, RADV (and ANV as well probably) segfaults in vkQueueSubmit. pWaitDstStageMask has to be a pointer to an array of waitSemaphoreCount masks

Yes, I've missed that for the encoder - sorry!

Fixed both issues at the ToT of main.