This library provides an API for using hardware video codecs on i.MX platforms. The API abstracts away platform specific details and allows for using the same code with different hardware video codecs on different i.MX platforms.
The hardware video codec is referred to as the VPU.
Currently, the following platforms are supported (listed with their VPUs):
Not yet supported:
This is the second version of this library. Differences to the older version are described below.
This library is licensed under the LGPL v2.1.
libimxvpuapi depends on libimxdmabuffer.
Additional dependencies are specific to the target platform:
imx-vpu package version 3.10.17 or newer.imx-vpu-hantro 1.8.0 or newer.
Please note that the build scripts assume that this is a version of the
imx-vpu-hantro package with fixed header installation destination. Earlier
versions installed all Hantro headers in the main include directory. Newer
ones create hantro_enc and hantro_decsubdirectories. libimxvpuapi
expects these directories to exist.imx-vpu-hantro 1.8.0 or newer, just like above
(since the plus has the same G1 / G2 decoder as the i.MX8m mini),
and also imx-vpu-hantro-vc 1.1.0 or newer (for the VC8000E encoder).This project uses the waf meta build system. To configure , first set the following environment variables to whatever is necessary for cross compilation for your platform:
CCCFLAGSLDFLAGSPKG_CONFIG_PATHPKG_CONFIG_SYSROOT_DIRThen, run:
./waf configure --prefix=PREFIX --imx-platform=IMX_PLATFORM --sysroot-path=SYSROOT(The aforementioned environment variables are only necessary for this configure call.)
The arguments are as follows:
PREFIX defines the installation prefix, that is, where the built binaries
will be installed.IMX_PLATFORM specifies what i.MX platform to build for. See the list below
for the valid values.SYSROOT is the absolute path to the sysroot for the platform. This is the
path where usr/include/imx/mxcfb.h can be found. In cross compilation
environments like Yocto or buildroot, this is where the sysroot files for the
target i.MX platforms are.Valid IMX_PLATFORM values are:
imx6 : i.MX6 (all variants)imx8m : i.MX8m quadimx8mm : i.MX8m miniimx8mp : i.MX8m plusOnce configuration is complete, run:
./wafThis builds the library. Finally, to install, run:
./waf installThis will install the headers in $PREFIX/include/imxvpuapi2/, the libraries
in $PREFIX/lib/, and generate a pkg-config .pc file, which is placed in
$PREFIX/lib/pkgconfig/ .
The API is documented in the imxvpuapi/imxvpuapi.h header.
libimxvpuapi comes with these examples in the example/ directory:
decode-example.c : demonstrates how to use the decoder APIencode-example.c : demonstrates how to use the encoder API(Other source files in the example/ directory are common utility code used
by all examples above.)
Raw frames are read/written as YUV4MPEG2 (y4m) data, and encoded to / decoded from h.264.
i.MX6 VPU timeout
If errors like imx_vpu_api_dec_decode() failed: timeout or VPU blocking: timeout
are observed, check if the following workarounds for known problems help:
Overclocked VPU: The i.MX6 VPU is clocked at 266 MHz by default (according to the VPU documentation). Some configurations clock the VPU at 352 MHz, and can exhibit VPU timeout problems, particularly during h.264 encoding. Try running the VPU at 266 MHz.
Known issue with IPU configuration: As shown by this Github entry,
the CONFIG_IMX_IPUV3_CORE kernel config flag can cause problems with the
VPU. Disable it, then try again.
Low-level i.MX6 VPU library bug: imx-vpu versions prior to 5.4.31 also have been observed to cause VPU timeouts. These seem to be related to the 5.4.31 fix described as: "Fix VPU blocked in BWB module".
VP6 frames decoded upside down by Hantro G1 decoder
This seems to be a bug in imx-vpu-hantro. A workaround is currently not known.
imx_vpu_api_dec_add_framebuffers_to_pool() fails on Hantro VPUs, logs show "CODEC_ERROR_UNSPECIFIED" This can happen when too many framebuffers are added. One common situation is when libimxvpuapi is used through gstreamer-imx, and downstream has a queue that soaks up too many buffers. Example:
gst-launch-1.0 filesrc location=test-vp8.mkv ! matroskademux ! imxvpudec_vp8 ! queue ! autovideosinkIf the video sink takes a while to start, it is possible that the queue in this example keeps storing more and more buffers without releasing them. This causes the VPU to repeatedly request more framebuffers to decode more frames into. Newer versions of imx-vpu-hantro define upper limits to how many framebuffers can be added, and if these limits are exceeded, the error occurs.
It is recommended to not hold on to too many framebuffers at once. Keep it below 16
framebuffers. In the GStreamer case above, it is sufficient to set the max-size-buffers
property of the queue there to 1, since a queue that is right before a video sink is
meant for decoupling the upstream portion of that pipeline from the video sink. This
helps with reducing the likelihood of lost frames. More than 1 buffer in the queue is
not needed for this.
On machines with Hantro based VPUs, "DWL" and "EWL" errors may appear. DWL and EWL are low level layers used by the Hantro drivers. Errors typically happen when certain device nodes are not available. If such errors occur, check for the presence of:
/dev/ion : Used with older imx kernels (<5.15)/dev/dma_heap/linux,cma-uncached : Used with newer imx kernels (>=5.15)/dev/mxc_hantro : Hantro G1 and G2 decoder ; found on the i.MX8m mini,
i.MX8m quad, i.MX8m plus/dev/mxc_hantro_h1 : Hantro H1 encoder ; found on the i.MX8m mini/dev/mxc_hantro_vc8000e : Hantro VC8000E encoder ; found on the i.MX8m plusIf at least one of these are missing, check the kernel configuration. The ION and/or the dma-heap CMA allocator needs to be enabled. Also check that the Hantro VPU is enabled.
This is version 2 of libimxvpuapi. Major changes are:
imx_vpu_ prefix has been changed to imx_vpu_api_.imxvpuapi* to imxvpuapi2 to reflect the
new and incompatible API and to allow for coexistence with the old version
of the library.imx_vpu_dec_check_if_can_decode() to see
if decoding is possible now to avoid a deadlock. This is no longer needed.
Decoders either perform internal DMA-based copies of frames (i.MX6, in the
same step as detiling from the VPU layout), or accept additional framebuffers
in their framebuffer pools on the fly. This simplifies the use of the
decoding API considerably.imx_vpu_load() and imx_vpu_unload() functions are gone. (Un)loading
the i.MX6 VPU firmware is now done internally.codec_format was renamed to compression_format to avoid confusion with
the notion of hardware video codecs.