luxonis / depthai

DepthAI Python API utilities, examples, and tutorials.
https://docs.luxonis.com
MIT License
919 stars 231 forks source link

[BUG] -vid failed MFX: Can't initialize session #461

Open franva opened 3 years ago

franva commented 3 years ago

Describe the bug After today's git pull and/or maybe installation of openvino, whenever I ran the depthai_demp.py with -vid param, I receive this error:

python depthai_demo.py -cnn road-segmentation-adas-0001 --sync -vid videos/CamVid.mp4

Using depthai module from:  /media/winstonfan/Workspace/Learning/Github/depthai/myvenv/lib/python3.8/site-packages/depthai.cpython-38-x86_64-linux-gnu.so
Depthai version installed:  2.9.0.0
Available devices:
[0] 14442C10013762D700 [X_LINK_UNBOOTED]
Enabling low-bandwidth mode due to low USB speed... (speed: UsbSpeed.HIGH)

(python:27014): GStreamer-WARNING **: 11:30:01.695: Failed to load plugin '/opt/intel/openvino_2021/data_processing/gstreamer/lib/gstreamer-1.0/libgstplayback.so': /opt/intel/openvino_2021/data_processing/gstreamer/lib/gstreamer-1.0/libgstplayback.so: undefined symbol: gst_type_mark_as_plugin_api
[ WARN:0] global ../opencv/modules/videoio/src/cap_gstreamer.cpp (2120) cv_capture_open_with_params GStreamer: Exception is raised: OpenCV(4.5.3-openvino) ../opencv/modules/videoio/src/cap_gstreamer.cpp:884: error: (-215:Assertion failed) uridecodebin in function 'open'

[ WARN:0] global ../opencv/modules/videoio/src/cap_gstreamer.cpp (597) isPipelinePlaying OpenCV | GStreamer warning: GStreamer: pipeline have not been created
libva info: VA-API version 1.11.0
libva info: User environment variable requested driver 'iHD'
libva info: Trying to open /opt/intel/mediasdk/lib64/iHD_drv_video.so
libva info: Found init function __vaDriverInit_1_11
DRM_IOCTL_I915_GEM_APERTURE failed: Invalid argument
Assuming 131072kB available aperture size.
May lead to reduced performance or incorrect rendering.
get chip id failed: -1 [22]
param: 4, val: 0
libva error: /opt/intel/mediasdk/lib64/iHD_drv_video.so init failed
libva info: va_openDriver() returns 18
MFX: Can't initialize session
Creating MJPEG link for ColorCamera node and color xlink stream...
[14442C10013762D700] [98.544] [NeuralNetwork(3)] [warning] Network compiled for 8 shaves, maximum available 13, compiling for 6 shaves likely will yield in better performance
=== TOTAL FPS ===

To Reproduce Steps to reproduce the behavior:

  1. get the latest code of depthai from this repo
  2. set up Depthai by following this tutorial
  3. install Intel's OpenVINO by the GUI approach by following this tutorial
  4. run the demo py with any cnn plus the -vid param

Expected behavior A video window will show up and start seeing draws on top of the video.

Additional context This issue could be treated like "kick away ball" between DepthAI and OpenVINO since it might be caused by DepthAI and/or OpenVINO, so users ask question on either side could be kicked away and asked to post issue on the other side.

But to me if both sides don't want to take this issue, then it falls on the user's shoulder and as an individual user, I have to admit that I don't have that much knowledge on either side, so impossible for me to figure it out.

Plus, the DepthAI users has to setup OpenVINO env anyway as the true purpose of using OAK cameras + DepthAI is to deploy custom trained models onto the camera and utilize DepthAI to implement our ideas.

What have I tried so far I tried to install the gstreamer by running

  1. sudo apt-get install --reinstall gstreamer1.0-alsa gstreamer1.0-libav gstreamer1.0-plugins-bad gstreamer1.0-plugins-base gstreamer1.0-plugins-good gstreamer1.0-plugins-ugly gstreamer1.0-pulseaudio libgstreamer-plugins-bad1.0-0 libgstreamer-plugins-base1.0-0 libgstreamer-plugins-good1.0-0 libgstreamer1.0-0 . this doesn't work
  2. also followed the official gstreamer website to install it apt-get install libgstreamer1.0-dev libgstreamer-plugins-base1.0-dev libgstreamer-plugins-bad1.0-dev gstreamer1.0-plugins-base gstreamer1.0-plugins-good gstreamer1.0-plugins-bad gstreamer1.0-plugins-ugly gstreamer1.0-libav gstreamer1.0-doc gstreamer1.0-tools gstreamer1.0-x gstreamer1.0-alsa gstreamer1.0-gl gstreamer1.0-gtk3 gstreamer1.0-qt5 gstreamer1.0-pulseaudio, also doesn't work

What I will do for helping solving this issue I will also post an issue on Intel's OpenVINO github and hook these 2 issues with each other by providing links inside these 2 issues.

Here is the link.

franva commented 3 years ago

Update

The issue seems fixed once I removed the below line from ~/.bashrc

source /opt/intel/openvino_2021/bin/setupvars.sh

So now, it is clear that the setupvars.sh breaks gstreamer somehow.

So the workaround is:

  1. For running the cnn model with -vid, make sure to remove that line in .bashrc
  2. For converting other models to OpenVINO model, adding that line to .bashrc

Not ideal, hopefully some "hero" will show up and have a decent solution for this issue.

VanDavv commented 3 years ago

The missing symbol - gst_type_mark_as_plugin_api - comes from GstUtils and was introduced in GStreamer 1.18, so 1.0 version won't have it. I haven't dig into it yet, but I think either LD_LIBRARY_PATH or PATH env variables are overwriting a default GStreamer location in the system (as they are modified inside the setupvars.sh script)

From our side, we would add a warning if we detect an active OpenVINO version (e.g. by checking InferenceEngine_DIR or INTEL_OPENVINO_DIR or INTEL_CVSDK_DIR env variables) that it may cause issues when inferencing on video