calling xilinx ffmpeg from code

[Ben8823]

I'm using an AWS vt1 instance (Ubuntu 18.04), and I want to use the xilinx ffmpeg from my code. If I try to open the mpsoc_vcu_h264 codec from my code, I get the following error:

[XMA] xmaencoder XMA session creation must be after initialization
Error: encoder error: ERROR: Unable to allocate MPSoC encoder session : ffmpeg pid 3704 on device index =  0 cu index = 38 [lxlnx_hwdev]
Could not open video codec: Generic error in an external library

If I use the CLI for ffmpeg it works fine without any errors. Are there any additional steps I have to carry out when I call it from my code?

[NastoohX]

Hi, Thank you for bring this issue to our attention. Once you have confirmed that your VT1 is configured as per https://xilinx.github.io/video-sdk/v2.0/getting_started_on_vt1.html, can you provide the output of the following commands: 1- xbutil examine 2- ffmpeg -codecs | grep mpsoc Cheers,

[Ben8823]

Yes, I've configured the instance according to the doc. As I said, ffmpeg is working fine from the CLI, just when I want to call it from code with avcodec_open2 I get the stated errors. Below you can find the output of the commands.

xbutil examine:

System Configuration
  OS Name              : Linux
  Release              : 5.4.0-1096-aws
  Version              : #104~18.04.1-Ubuntu SMP Wed Jan 25 19:58:53 UTC 2023
  Machine              : x86_64
  CPU Cores            : 12
  Memory               : 22535 MB
  Distribution         : Ubuntu 18.04.6 LTS
  GLIBC                : 2.27
  Model                : vt1.3xlarge

XRT
  Version              : 2.11.691
  Branch               : 2021.1
  Hash                 : 3e695ed86d15164e36267fb83def6ff2aaecd758
  Hash Date            : 2021-11-18 18:16:39
  XOCL                 : 2.11.691, 3e695ed86d15164e36267fb83def6ff2aaecd758
  XCLMGMT              : unknown, unknown

Devices present
  [0000:00:1f.0] : xilinx_u30_gen3x4_base_2
  [0000:00:1e.0] : xilinx_u30_gen3x4_base_2

ffmpeg -codecs | grep mpsoc

ffmpeg version n4.4.xlnx.1 Copyright (c) 2000-2021 the FFmpeg developers
  built with gcc 7 (Ubuntu 7.5.0-3ubuntu1~18.04)
  configuration: --prefix=/opt/xilinx/ffmpeg --datadir=/opt/xilinx/ffmpeg/etc --enable-x86asm --enable-libxma2api --disable-doc --enable-libxvbm --enable-libxrm --enable-libfreetype --enable-libfontconfig --extra-cflags=-I/opt/xilinx/xrt/include/xma2 --extra-ldflags=-L/opt/xilinx/xrt/lib --extra-libs=-lxma2api --extra-libs=-lxrt_core --extra-libs=-lxrt_coreutil --extra-libs=-lpthread --extra-libs=-ldl --disable-static --enable-shared
  libavutil      56. 70.100 / 56. 70.100
  libavcodec     58.134.100 / 58.134.100
  libavformat    58. 76.100 / 58. 76.100
  libavdevice    58. 13.100 / 58. 13.100
  libavfilter     7.110.100 /  7.110.100
  libswscale      5.  9.100 /  5.  9.100
  libswresample   3.  9.100 /  3.  9.100

<<<<<<<==  FFmpeg xrm ===>>>>>>>>
No device set hence falling to default device 0
------------------i=0------------------------------------------

   xclbin_name :  /opt/xilinx/xcdr/xclbins/transcode.xclbin
   device_id   :  0
------------------------------------------------------------

 DEV.LS h264                 H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10 (decoders: h264 h264_v4l2m2m mpsoc_vcu_h264 ) (encoders: h264_v4l2m2m h264_vaapi mpsoc_vcu_h264 )
 DEV.L. hevc                 H.265 / HEVC (High Efficiency Video Coding) (decoders: hevc hevc_v4l2m2m mpsoc_vcu_hevc ) (encoders: hevc_v4l2m2m hevc_vaapi mpsoc_vcu_hevc )

[NastoohX]

Hi, Thank you for the info. By the looks of it VT1 instance is set properly. To help with the debugging process, kindly, provide the output logs of the attached script. Logs are ffmpeg.log, dmesg.log and journal.log. It would also be useful if you could provide sample code lines on how you are invoking FFmpeg, from you program, Cheers, 60.sh.zip

[Ben8823]

The code is part of a bigger project, but below you can find the calls which happening basically. Attached, you can find the log files from the script.

*pCodec = avcodec_find_encoder_by_name("mpsoc_vcu_h264");

avformat_new_stream(m_pFormatContext, *pCodec);

AVDictionary *pOptions = nullptr;
ret = avcodec_open2(c, pCodec, &pOptions); // this one throws the error

dmseg.log journal.log ffmpeg.log

[jmouroux]

@Ben8823 Hello. In our ffmpeg.c implementation, in the main() routine, the XMA API xma_initialize() must be called before creating any accelerator sessions. If you are not using ffmpeg.c but your own application based on libav*, you must refer to the XMA API programming guide and/or review the main() routine in our SDK for a reference. Please see: https://github.com/Xilinx/app-ffmpeg4-xma/blob/35d02c9dc819a1db8533484a69ad5641c44fbe01/fftools/ffmpeg.c#L5104

[NastoohX]

Hi, Thank you for the attached log files. As noted by @jmouroux, the issue could be due to improper set of initialisation calls, Given that your FFmpeg cli is working properly, one way to confirm the proper initialisation sequence is to compare log files from your application vs those generated by FFmpeg cli, in particular the journal logs. Specifically, note that the sequence should follow: xrmCreateContext -> xmainitialize -> xma[enc|dec|...|_session_create. Alternatively, if issue is not due to initialisation sequence, then confirm that the shell session, which is running your application, has been properly initialised and has access to environment variables set by source /opt/xilinx/xcdr/setup.sh Let us know if the above help with your issue. Cheers,

[Ben8823]

Thank you both, that was the hint I needed. It's working now.