Pixel format '(null)' not supported

[dwffls]

When using the h264_vaapi encoder AVHWFrameContext gives the following error:

[AVHWFramesContext @ 0x7eff081a2100] The hardware pixel format '(null)' is not supported by the device type 'VAAPI'
[ERROR] [1700131267.578674463] [FFMPEGEncoder]: failed to initialize VAAPI frame context: Function not implemented
[ERROR] [1700131267.579830043] [FFMPEGPublisher]: cannot initialize encoder!

When using the default libx264 encoder everything works as expected.

VAAPI is correlty installed on my machine with the following vainfo output:

libva info: VA-API version 1.7.0
libva info: User environment variable requested driver 'iHD'
libva info: Trying to open /usr/lib/x86_64-linux-gnu/dri/iHD_drv_video.so
libva info: Found init function __vaDriverInit_1_7
libva info: va_openDriver() returns 0
vainfo: VA-API version: 1.7 (libva 2.6.0)
vainfo: Driver version: Intel iHD driver for Intel(R) Gen Graphics - 20.1.1 ()
vainfo: Supported profile and entrypoints
      VAProfileNone                   : VAEntrypointVideoProc
      VAProfileNone                   : VAEntrypointStats
      VAProfileMPEG2Simple            : VAEntrypointVLD
      VAProfileMPEG2Main              : VAEntrypointVLD
      VAProfileH264Main               : VAEntrypointVLD
      VAProfileH264Main               : VAEntrypointEncSliceLP
      VAProfileH264High               : VAEntrypointVLD
      VAProfileH264High               : VAEntrypointEncSliceLP
      VAProfileJPEGBaseline           : VAEntrypointVLD
      VAProfileJPEGBaseline           : VAEntrypointEncPicture
      VAProfileH264ConstrainedBaseline: VAEntrypointVLD
      VAProfileH264ConstrainedBaseline: VAEntrypointEncSliceLP
      VAProfileHEVCMain               : VAEntrypointVLD
      VAProfileHEVCMain               : VAEntrypointEncSliceLP
      VAProfileHEVCMain10             : VAEntrypointVLD
      VAProfileHEVCMain10             : VAEntrypointEncSliceLP
      VAProfileVP9Profile0            : VAEntrypointVLD
      VAProfileVP9Profile1            : VAEntrypointVLD
      VAProfileVP9Profile2            : VAEntrypointVLD
      VAProfileVP9Profile3            : VAEntrypointVLD
      VAProfileHEVCMain12             : VAEntrypointVLD
      VAProfileHEVCMain422_10         : VAEntrypointVLD
      VAProfileHEVCMain422_12         : VAEntrypointVLD
      VAProfileHEVCMain444            : VAEntrypointVLD
      VAProfileHEVCMain444            : VAEntrypointEncSliceLP
      VAProfileHEVCMain444_10         : VAEntrypointVLD
      VAProfileHEVCMain444_10         : VAEntrypointEncSliceLP
      VAProfileHEVCMain444_12         : VAEntrypointVLD
      VAProfileHEVCSccMain            : VAEntrypointVLD
      VAProfileHEVCSccMain10          : VAEntrypointVLD
      VAProfileHEVCSccMain444         : VAEntrypointVLD

[berndpfrommer]

It's obviously hard for me to debug without being on your system so I have to enlist you for debugging help. The error messages you post could come from libffmpeg not finding a good pixel format, so it would help to sprinkle some debug statements into the code section below.

Also, does vaapi encoding with straight ffmpeg work, i.e. can you do something like this (taken from here) on the command line?

ffmpeg -y -vaapi_device /dev/dri/renderD128 -i file_in.avi -vf 'format=nv12,hwupload' -c:v h264_vaapi file_out.mp4

Here the snippet of code (ffmpeg_encoder.cpp) that you need to modify for debugging:

if (usesHardwareFrames_) {
    const auto fmts = utils::get_hwframe_transfer_formats(hw_frames_ref);
    // new:
    std::cout << "formats found: " << fmts.size() << std::endl;
    frames_ctx->sw_format = utils::get_preferred_pixel_format("h264_vaapi", fmts);
    std::cout  << "format used: " << static_cast<int>(frames_ctx->sw_format) << std::endl;
    if (pixFormat_ != AV_PIX_FMT_NONE) {
      RCLCPP_INFO_STREAM(
        logger_, "user overriding software pix fmt " << utils::pix(frames_ctx->sw_format));
      RCLCPP_INFO_STREAM(logger_, "with " << utils::pix(pixFormat_));
      frames_ctx->sw_format = pixFormat_;  // override default at your own risk!
    } else {
      RCLCPP_INFO_STREAM(
        logger_, "using software pixel format: " << utils::pix(frames_ctx->sw_format));
    }
    if (frames_ctx->sw_format == AV_PIX_FMT_NONE) {
      av_buffer_unref(&hw_frames_ref);
      throw(std::runtime_error("cannot find valid sw pixel format!"));
    }
  }

[dwffls]

The ffmpeg example runs without any problem. Next to this, ive already tried to looking at/ changing the code snippet you provided. However without any luck. It appears as though net usesHardwareFrames_ bool is never set correctly.

I'll let you know if i find out anything more

[berndpfrommer]

@dwffls have you made any progress on this issue? If usesHardwareFrames_ is false, then that means something is going wrong in find_hw_config(). Can you put some debug printing in there to find out why usesHWFrames is not set correctly? It could be that either the device type does not match or config->methods does not have the right flags set.

[berndpfrommer]

closing for lack of activity. Not clear what the problem is.