Closed akhilxavi closed 3 weeks ago
Which version of onevpl/vpl-gpu-rt did you use? Could you try the master code?
BTW, if possible, please use kernel 6.9.0+. See more details in https://github.com/intel/cartwheel-ffmpeg/issues/322
@feiwan1 I was trying the filter chain "vpp_qsv=framerate=60,scale_qsv=w=1920:h=1080:format=vuyx,hwdownload,format=vuyx" but it failed with vpp_qsv issues. Now I modified the filter chain to:
filter chain: decoder(qsv)->scale_qsv->hwdownload->output(vuyx)
"scale_qsv=w=1920:h=1080:format=vuyx,hwdownload,format=vuyx" and the filter source is feed with AV_PIX_FMT_QSV from the decoder itself. It works with the same code which I set hw and frame context for each members in the filter chain.
if(avfilter_graph_parse_ptr(filter_graph, args, &inputs, &outputs, NULL) <0)
av_log(NULL, AV_LOG_ERROR, "failed to parse filter graph description\n");
for(unsigned int i = 0; i < filter_graph->nb_filters; i++){
if(!(filter_graph->filters[i]->hw_device_ctx = avctx->hw_device_ctx))
av_log(NULL, AV_LOG_ERROR, "hw device ctx to filters failed\n");
}
seems like the issue is only with "vpp_qsv" component. Does "vpp_qsv" need any extra initialization than the above one?
My kernel version is: 6.8.0-31-generic Vaapi version: 1.22.0 (libva 2.22.0.pre1) libmfx: 1.2.10 intel iHD drivers - 24.2.1 firmwares: DMC firmware i915/adlp_dmc.bin (v2.20) GuC firmware i915/tgl_guc_70.bin version 70.20.0 HuC firmware i915/tgl_huc.bin version 7.9.3
seems like the issue is only with "vpp_qsv" component. Does "vpp_qsv" need any extra initialization than the above one?
No, I guess. vpp_qsv and scale_qsv shares same configuration and initialization.
But when I code after linking the filters source -> vpp_qsv -> scale_qsv -> hwdownload -> format -> sink, I am seeing below error,
Does the source support hw frames (AV_PIX_FMT_QSV) ? If not, you should specify avctx->hw_device_ctx for vpp_qsv filter in your code.
Now I modified the filter chain to:
filter chain: decoder(qsv)->scale_qsv->hwdownload->output(vuyx)
"scale_qsv=w=1920:h=1080:format=vuyx,hwdownload,format=vuyx" and the filter source is feed with AV_PIX_FMT_QSV from the decoder itself. It works with the same code which I set hw and frame context for each members in the filter chain.
The filter chain: decoder(qsv)->vpp_qsv->hwdownload->output(vuyx) should work too.
But when I code after linking the filters source -> vpp_qsv -> scale_qsv -> hwdownload -> format -> sink, I am seeing below error,
Does the source support hw frames (AV_PIX_FMT_QSV) ? If not, you should specify avctx->hw_device_ctx for vpp_qsv filter in your code.
Yes my Alder lake decoder is supporting hw frames. By outputting AV_PIX_FMT_QSV from decoder, below configuration works with hw context and and frame context in my code.
filter chain: decoder(qsv)->scale_qsv->hwdownload->output(vuyx)
If I introduce vpp_qsv into my code filter chain and specifying avctx->hw_device_ctx through the above for loop, I am not seeing any frames output from sink.
vpp_qsv -> scale_qsv -> hwdownload -> format -> sink
Below script works from cmd line though. ffmpeg -hwaccel qsv -c:v hevc_qsv -i /media/pcie/latency-test/complex-video/nv12-5000.h265 -vf 'vpp_qsv=framerate=60,scale_qsv=w=1920:h=1080:format=vuyx,hwdownload,format=vuyx' -f null -
Not sure what is missing in code which is specific to vpp_qsv.
If I introduce vpp_qsv into my code filter chain and specifying avctx->hw_device_ctx through the above for loop, I am not seeing any frames output from sink.
vpp_qsv -> scale_qsv -> hwdownload -> format -> sink
Does vpp_qsv->hwdownload->format->sink work ?
If I introduce vpp_qsv into my code filter chain and specifying avctx->hw_device_ctx through the above for loop, I am not seeing any frames output from sink. vpp_qsv -> scale_qsv -> hwdownload -> format -> sink
Does vpp_qsv->hwdownload->format->sink work ?
with above filter, when configuring input to filter as QSV and output as vuyx, seeing below error
libva info: va_openDriver() returns 0 [hwdownload @ 0x59108a444fc0] Invalid output format vuyx for hwframe download. [Parsed_hwdownload_1 @ 0x59108a44b800] Failed to configure output pad on Parsed_hwdownload_1 failed to configure filter graph
when configuring input to filter as codeccontext pix format and output as vuyx, seeing below error (same with vpp_qsv -> scale_qsv -> hwdownload -> format -> sink) [AVHWFramesContext @ 0x61446b6369c0] Unsupported pixel format [Parsed_vpp_qsv_0 @ 0x61446b634e80] Error creating frames_ctx for output pad. [Parsed_vpp_qsv_0 @ 0x61446b634e80] Failed to configure output pad on Parsed_vpp_qsv_0 failed to configure filter graph [hevc_qsv @ 0x61446ad36140] Invalid pkt_timebase, passing timestamps as-is. [in @ 0x61446b629a00] Changing video frame properties on the fly is not supported by all filters. [in @ 0x61446b629a00] filter context - w: 3840 h: 2160 fmt: 0 csp: unknown range: unknown, incoming frame - w: 3840 h: 2160 fmt: 114 csp: unknown range: tv pts_time: 0 [hwdownload @ 0x61446b635080] Input frames must have hardware context. [hwdownload @ 0x61446b635080] Input frames must have hardware context.
scale_qsv and vpp_qsv have the same code for initialization, it is weird scale_qsv can work but vpp_qsv doesn't work. What's your FFmpeg version ?
scale_qsv and vpp_qsv have the same code for initialization, it is weird scale_qsv can work but vpp_qsv doesn't work. What's your FFmpeg version ?
I build it from source:
ffmpeg version N-114973-gce33a5f16d Copyright (c) 2000-2024 the FFmpeg developers built with gcc 13 (Ubuntu 13.2.0-23ubuntu4)
If I introduce vpp_qsv into my code filter chain and specifying avctx->hw_device_ctx through the above for loop, I am not seeing any frames output from sink. vpp_qsv -> scale_qsv -> hwdownload -> format -> sink
Does vpp_qsv->hwdownload->format->sink work ?
with above filter, when configuring input to filter as QSV and output as vuyx, seeing below error
libva info: va_openDriver() returns 0 [hwdownload @ 0x59108a444fc0] Invalid output format vuyx for hwframe download. [Parsed_hwdownload_1 @ 0x59108a44b800] Failed to configure output pad on Parsed_hwdownload_1 failed to configure filter graph
Now I modified the filter chain to:
filter chain: decoder(qsv)->scale_qsv->hwdownload->output(vuyx)
"scale_qsv=w=1920:h=1080:format=vuyx,hwdownload,format=vuyx" and the filter source is feed with AV_PIX_FMT_QSV from the decoder itself. It works with the same code which I set hw and frame context for each members in the filter chain.
Seems your test clip is not a YUV444 8bit stream, you set the output sw format of scale_qsv to vuyx when testing filter chain: decoder(qsv)->scale_qsv->hwdownload->output(vuyx), you should change the output sw format of vpp_qsv to vuyx too when trying filter chain: decoder (qsv)->vpp_qsv->hwdownload->output(vuyx) "vpp_qsv=format=vuyx,hwdownload,format=vuyx"
If I introduce vpp_qsv into my code filter chain and specifying avctx->hw_device_ctx through the above for loop, I am not seeing any frames output from sink. vpp_qsv -> scale_qsv -> hwdownload -> format -> sink
Does vpp_qsv->hwdownload->format->sink work ?
with above filter, when configuring input to filter as QSV and output as vuyx, seeing below error libva info: va_openDriver() returns 0 [hwdownload @ 0x59108a444fc0] Invalid output format vuyx for hwframe download. [Parsed_hwdownload_1 @ 0x59108a44b800] Failed to configure output pad on Parsed_hwdownload_1 failed to configure filter graph
Now I modified the filter chain to: filter chain: decoder(qsv)->scale_qsv->hwdownload->output(vuyx) "scale_qsv=w=1920:h=1080:format=vuyx,hwdownload,format=vuyx" and the filter source is feed with AV_PIX_FMT_QSV from the decoder itself. It works with the same code which I set hw and frame context for each members in the filter chain.
Seems your test clip is not a YUV444 8bit stream, you set the output sw format of scale_qsv to vuyx when testing filter chain: decoder(qsv)->scale_qsv->hwdownload->output(vuyx), you should change the output sw format of vpp_qsv to vuyx too when trying filter chain: decoder (qsv)->vpp_qsv->hwdownload->output(vuyx) "vpp_qsv=format=vuyx,hwdownload,format=vuyx"
If I introduce vpp_qsv into my code filter chain and specifying avctx->hw_device_ctx through the above for loop, I am not seeing any frames output from sink. vpp_qsv -> scale_qsv -> hwdownload -> format -> sink
Does vpp_qsv->hwdownload->format->sink work ?
with above filter, when configuring input to filter as QSV and output as vuyx, seeing below error libva info: va_openDriver() returns 0 [hwdownload @ 0x59108a444fc0] Invalid output format vuyx for hwframe download. [Parsed_hwdownload_1 @ 0x59108a44b800] Failed to configure output pad on Parsed_hwdownload_1 failed to configure filter graph
Now I modified the filter chain to: filter chain: decoder(qsv)->scale_qsv->hwdownload->output(vuyx) "scale_qsv=w=1920:h=1080:format=vuyx,hwdownload,format=vuyx" and the filter source is feed with AV_PIX_FMT_QSV from the decoder itself. It works with the same code which I set hw and frame context for each members in the filter chain.
Seems your test clip is not a YUV444 8bit stream, you set the output sw format of scale_qsv to vuyx when testing filter chain: decoder(qsv)->scale_qsv->hwdownload->output(vuyx), you should change the output sw format of vpp_qsv to vuyx too when trying filter chain: decoder (qsv)->vpp_qsv->hwdownload->output(vuyx) "vpp_qsv=format=vuyx,hwdownload,format=vuyx"
with format set for vpp_qsv the code starts working. The decoder output is in hw_frame format.
Thank you for the help.
does the decoder should always output hw_frames to feed to filter source? My decoder is always outputting frames in hw_frames format, is that because I set AVHWFramesContext -> format to AV_PIX_FMT_QSV?
does the decoder should always output hw_frames to feed to filter source? My decoder is always outputting frames in hw_frames format, is that because I set AVHWFramesContext -> format to AV_PIX_FMT_QSV?
hevc_qsv should be able to output frames in system memory if you neither set avctx->hw_device_ctx nor set avctx->hw_frames_ctx in your code. You may run the command below to verify it:
ffmpeg -c:v hevc_qsv -i input.mp4 -vf "scale=w=1920:h=1080" -f null -
On my ADL device running 6.8.0.11 kernel, I am trying to decode, scale and format conversion using hevc_qsv, vpp_qsv, vpp_scale respectively. below command work as expected on my device
ffmpeg -hwaccel qsv -c:v hevc_qsv -i /media/pcie/latency-test/complex-video/nv12-5000.h265 -vf 'vpp_qsv=framerate=60,scale_qsv=w=1920:h=1080:format=vuyx,hwdownload,format=vuyx' -f null -
But when I code after linking the filters source -> vpp_qsv -> scale_qsv -> hwdownload -> format -> sink, I am seeing below error,
[vpp_qsv @ 0x5c5eb9dc18c0] No hw context provided. [vpp_qsv @ 0x5c5eb9dc18c0] Failed to configure output pad on vpp_qsv
How to specify hw context for vpp in code? How the vpp sink pad is expected to be configured after linking? I configures the in and out pads as below.
Thank you in advance.