search

balena-os / balena-variscite

Balena support for Variscite boards
Apache License 2.0
3 stars 0 forks source link

[imx8m-var-dart] VPU doesn’t seem to be detected #82

Closed shaunmulligan closed 5 years ago

shaunmulligan commented 5 years ago

In an ubuntu 18.10 base image with gstreamer-1.0 installed, then follow the instructions http://variwiki.com/index.php?title=DART-MX8M_GSTREAMER 

No such element or plugin 'vpudec'

The vpudec element should be detected.

acostach commented 5 years ago

The element vpudec appears to be a gstreamer plugin - a shared library coming from a set of gstreamer plugins provided by Freescale. It will be detected only if is present in the hostOs and queried from there, or in a container, in a path like /usr/lib/gstreamer-1.0/libgstvpu.so. Mainline gstreamer-plugins from ubuntu don't include with these sort of libraries, instead they seem to be provided by gstreamer-imx which needs to be built separately.

In a container, to have it detected, I could only build the version of imxvpudec library for imx6q according to the docs at: https://github.com/Freescale/gstreamer-imx/blob/master/docs/debian-ubuntu.md . Similar steps could be adapted for the imx8 but the docs don't specify how.

Also, depending on installation steps it may be necessary to tell gstreamer where to look for these plugins with GST_PLUGIN_PATH. The sources for this library were obtained by running in an ubuntu container the steps in the above docs at section 8) Build and install gstreamer-imx

root@43a2569:/# export GST_PLUGIN_PATH=$GST_PLUGIN_PATH:/root/gstreamer-imx/build/src/vpu/
root@43a2569:/# gst-inspect-1.0 | grep imx
imxvpu:  imxvpuenc_mjpeg: Freescale VPU motion JPEG video encoder
imxvpu:  imxvpuenc_mpeg4: Freescale VPU MPEG-4 video encoder
imxvpu:  imxvpuenc_h264: Freescale VPU h.264 video encoder
imxvpu:  imxvpuenc_h263: Freescale VPU h.263 video encoder
**imxvpu**:  **imxvpudec**: Freescale VPU video decoder

root@43a2569:/# gst-inspect-1.0 **imxvpudec**
Factory Details:
  Rank                     primary + 1 (257)
  Long-name                Freescale VPU video decoder
  Klass                    Codec/Decoder/Video
  Description              hardware-accelerated video decoding using the Freescale VPU engine

Plugin Details:
  Name                     imxvpu
  Description              video en- and decoder elements using the Freescale i.MX VPU
  Filename                 //root/gstreamer-imx/build/src/vpu/libgstimxvpu.so
  Version                  0.13.0
  License                  LGPL
  Source module            gstreamer-imx
  Binary package           Unknown package release
  Origin URL               Unknown package origin

GObject
 +----GInitiallyUnowned
         +----GstObject
               +----GstElement
                     +----GstVideoDecoder
                           +----GstImxVpuDecoder

...

shaunmulligan commented 5 years ago

Thanks for the deep investigation @acostach , I think the part I was missing was GST_PLUGIN_PATH env var. I will reach out to the Variscite team to ask about specific builds for the MX8 variant 👍

shaunmulligan commented 5 years ago

It seems that currently gstreamer-imx does not support at all iMX8 family.

https://github.com/Freescale/gstreamer-imx/issues/205

eranmati commented 5 years ago

Indeed, gstreamer-imx (provided by the fsl community) currently doesn't support iMX8.

However, NXP provides imx-gst1.0-plugin and custom versions of gstreamer1.0-plugins-bad, gstreamer1.0-plugins-base and gstreamer1.0-plugins-good, with iMX8 support.

The vpudec plugin described in the following link is from imx-gst1.0-plugin (as can be seen in the gst-inspect-1.0 output): http://variwiki.com/index.php?title=DART-MX8M_GSTREAMER

acostach commented 5 years ago

@eranmati thank you! Do you happen to have some knowledge of building these in Ubuntu? It would be great if you knew some branches for each of these or some SHAs and configure options similar to what https://github.com/Freescale/gstreamer-imx/blob/master/docs/debian-ubuntu.md does. Thank you!

eranmati commented 5 years ago

@acostach, no problem. Yes, we have built it into our Debian release for the DART-MX8M. Please see: http://variwiki.com/index.php?title=DART-MX8M http://variwiki.com/index.php?title=Debian_Build_Release&release=RELEASE_STRETCH_V1.0_DART-MX8M

If you have questions about it, please open a ticket in the Variscite portal.

Regards, Eran

acostach commented 5 years ago

Thank you very much Eran for pointing this out!

@shaunmulligan Looks like the Variscite Debian release contains these prebuilt nxp gstreamer packages that can be installed in a container. The steps I did are as follows:

git clone https://github.com/varigit/debian-var.git -b debian_stretch_mx8m_var01 var_mx8m_dart_debian

Copy all contents from the repository path /variscite/deb/* inside a debian container, like balenalib/imx8m-var-dart-node:10-stretch-run.

Then install dependencies that will be further needed:

install_packages libglib2.0-0 gir1.2-glib-2.0 libcap2-bin libogg0 libopus0 liborc-0.4-0 libtheora0 libvisual-0.4-0 libvorbis0a libvorbisenc2 libcdparanoia0 iso-codes

After this install these copied debs from Variscte inside the container with dpkg -i - please note that they are in separate folders:

After this vpudec is present along the plugins:

root@6725c7b:/usr/src/app/gstpluginsbad# gst-inspect-1.0 vpudec
Factory Details:
  Rank                     primary + 1 (257)
  Long-name                IMX VPU-based video decoder
  Klass                    Codec/Decoder/Video
  Description              Decode compressed video to raw data
  Author                   Multimedia Team <shmmmw@freescale.com>

Plugin Details:
  Name                     vpu
  Description              VPU video codec
  Filename                 /usr/lib/aarch64-linux-gnu/gstreamer-1.0/libgstvpu.so
  Version                  4.4.4
  License                  LGPL
  Source module            imx-gst1.0-plugin
  Binary package           Freescle Gstreamer Multimedia Plugins
  Origin URL               http://www.freescale.com

GObject
 +----GInitiallyUnowned
       +----GstObject
             +----GstElement
                   +----GstVideoDecoder
                         +----GstVpuDec
Pad Templates:
  SRC template: 'src'
    Availability: Always

For opencv plugin I did:

install_packages libopencv-core2.4v5 libopencv-calib3d2.4v5 libopencv-highgui2.4-deb0 libopencv-imgproc2.4v5 libopencv-objdetect2.4v5 libopencv-video2.4v5

Then I installed these debs from the gstpluginsbad folder:

dpkg -i libgstreamer-opencv1.0-0_1.14.0-2-var_arm64.deb
dpkg -i gstreamer1.0-opencv_1.14.0-2-var_arm64.deb

Then I could see the opencv plugin present as well:

root@6725c7b:/usr/src/app/gstpluginsbad# gst-inspect-1.0 opencv
Plugin Details:
  Name                     opencv
  Description              GStreamer OpenCV Plugins
  Filename                 /usr/lib/aarch64-linux-gnu/gstreamer-1.0/libgstopencv.so
  Version                  1.14.0
  License                  LGPL
  Source module            gst-plugins-bad
  Source release date      2018-03-19
  Binary package           GStreamer Bad Plugins (Debian)
  Origin URL               http://packages.qa.debian.org/gst-plugins-bad1.0

  cvdilate: cvdilate
  cvequalizehist: cvequalizehist
  cverode: cverode
  cvlaplace: cvlaplace
  cvsmooth: cvsmooth
  cvsobel: cvsobel
  edgedetect: edgedetect
  faceblur: faceblur
  facedetect: facedetect
  motioncells: motioncells
  templatematch: templatematch
  opencvtextoverlay: opencvtextoverlay
  handdetect: handdetect
  skindetect: skindetect
  retinex: Retinex image colour enhacement
  segmentation: Foreground/background video sequence segmentation
  grabcut: Grabcut-based image FG/BG segmentation
  disparity: Stereo image disparity (depth) map calculation
  dewarp: Dewarp fisheye images

  19 features:
  +-- 19 elements
acostach commented 5 years ago

BalenaOS 2.31.1 is now available with kernel 4.14.

shaunmulligan commented 5 years ago

@acostach This isn't in production or staging yet though?

floion commented 5 years ago

It is in staging

shaunmulligan commented 5 years ago

2.31.5+rev1 is now in production with these changes :)

acostach commented 5 years ago

Thank you Shaun, guess we can close this ticket now.