pl_test_pixfmt failed for hardware decoder

[Hlongyu]

compile log https://github.com/haasn/libplacebo/issues/268

gdb ./plplay.exe
r -w glfw-gl -H 8k_test.mkv

Old value = <unreadable>
New value = AV_PIX_FMT_DXVA2_VLD
0x00007ff6c48a698f in init_codec (p=0x7ff6c490e040 <state>) at ../demos/plplay.c:176
176             for (int i = 0; (hwcfg = avcodec_get_hw_config(codec, i)); i++) {
(gdb)
177                 if (!pl_test_pixfmt(p->win->gpu, hwcfg->pix_fmt))
(gdb)
178                     continue;
(gdb)
176             for (int i = 0; (hwcfg = avcodec_get_hw_config(codec, i)); i++) {
(gdb)

Thread 1 hit Hardware watchpoint 4: hwcfg->pix_fmt

Old value = AV_PIX_FMT_DXVA2_VLD
New value = AV_PIX_FMT_D3D11VA_VLD
0x00007ff6c48a698f in init_codec (p=0x7ff6c490e040 <state>) at ../demos/plplay.c:176
176             for (int i = 0; (hwcfg = avcodec_get_hw_config(codec, i)); i++) {
(gdb)
177                 if (!pl_test_pixfmt(p->win->gpu, hwcfg->pix_fmt))
(gdb)
178                     continue;
(gdb)
176             for (int i = 0; (hwcfg = avcodec_get_hw_config(codec, i)); i++) {
(gdb)

Thread 1 hit Hardware watchpoint 4: hwcfg->pix_fmt

Old value = AV_PIX_FMT_D3D11VA_VLD
New value = AV_PIX_FMT_D3D11
0x00007ff6c48a698f in init_codec (p=0x7ff6c490e040 <state>) at ../demos/plplay.c:176
176             for (int i = 0; (hwcfg = avcodec_get_hw_config(codec, i)); i++) {
(gdb)
177                 if (!pl_test_pixfmt(p->win->gpu, hwcfg->pix_fmt))
(gdb)
178                     continue;
(gdb)
176             for (int i = 0; (hwcfg = avcodec_get_hw_config(codec, i)); i++) {
(gdb)

Thread 1 hit Hardware watchpoint 4: hwcfg->pix_fmt

Old value = AV_PIX_FMT_D3D11
New value = AV_PIX_FMT_CUDA
0x00007ff6c48a698f in init_codec (p=0x7ff6c490e040 <state>) at ../demos/plplay.c:176
176             for (int i = 0; (hwcfg = avcodec_get_hw_config(codec, i)); i++) {
(gdb)
177                 if (!pl_test_pixfmt(p->win->gpu, hwcfg->pix_fmt))
(gdb)
178                     continue;
(gdb)
176             for (int i = 0; (hwcfg = avcodec_get_hw_config(codec, i)); i++) {
(gdb)

Thread 1 hit Hardware watchpoint 4: hwcfg->pix_fmt

Old value = AV_PIX_FMT_CUDA
New value = AV_PIX_FMT_VAAPI
0x00007ff6c48a698f in init_codec (p=0x7ff6c490e040 <state>) at ../demos/plplay.c:176
176             for (int i = 0; (hwcfg = avcodec_get_hw_config(codec, i)); i++) {
(gdb)
177                 if (!pl_test_pixfmt(p->win->gpu, hwcfg->pix_fmt))
(gdb)
178                     continue;
(gdb)
176             for (int i = 0; (hwcfg = avcodec_get_hw_config(codec, i)); i++) {
(gdb)

Thread 1 hit Hardware watchpoint 4: hwcfg->pix_fmt

Old value = AV_PIX_FMT_VAAPI
New value = <unreadable>
0x00007ff6c48a698f in init_codec (p=0x7ff6c490e040 <state>) at ../demos/plplay.c:176
176             for (int i = 0; (hwcfg = avcodec_get_hw_config(codec, i)); i++) {
(gdb)
197         if (!hwcfg)
(gdb)
198             printf("Using software decoding\n");
(gdb)
[Thread 27056.0x5820 exited with code 0]
Using software decoding

pl_plane_data_from_pixfmt (out_data=0x55b730, out_bits=0x55b934, pix_fmt=AV_PIX_FMT_DXVA2_VLD)
    at ../src/include/libplacebo/utils/libav_internal.h:455
455         const AVPixFmtDescriptor *desc = av_pix_fmt_desc_get(pix_fmt);
(gdb) n
456         int planes = av_pix_fmt_count_planes(pix_fmt);
(gdb)
460         if (!desc || planes < 0) // e.g. AV_PIX_FMT_NONE
(gdb)
463         if (desc->flags & AV_PIX_FMT_FLAG_BITSTREAM) {
(gdb)
468         if (desc->flags & AV_PIX_FMT_FLAG_PAL) {
(gdb)
473         if (desc->flags & AV_PIX_FMT_FLAG_BAYER) {
(gdb)
479         if (desc->nb_components == 0 || desc->nb_components > 4) {
(gdb)
481             return 0;

It's seem all hw_pix_fmt failed with desc->nb_componetns == 0.

Did I misunderstand something? I thought this library could render any format of FFmpeg frames, including hardware frames.

[Hlongyu]

Did I misunderstand something? I thought this library could render any format of FFmpeg frames, including hardware frames.

Of course not, libplacebo does not have header file support for hardware-related APIs.

I checked the MPV source code and found that need to call the hardware map.

Edited: Add link https://github.com/mpv-player/mpv/blob/master/video/out/hwdec/hwdec_cuda.c

[haasn]

VAAPI needs DMABUF support:

• https://github.com/haasn/libplacebo/blob/master/src/include/libplacebo/utils/libav_internal.h#L573