Output of Gstreamer Pipeline on Google Coral Dev Board

[andrubrown]

I have a google coral dev board with usb camera and HDMI monitor attached. I am connected the board via MDT.

When I run python3 bodypix_gl_imx.py I get the following output in the terminal:

Model: models/bodypix_mobilenet_v1_075_1024_768_16_quant_decoder_edgetpu.tflite
Heatmap size:  (65, 49)
Stride:  16 (65, 49)
Inference size: (1024, 768)
Source size: (1920, 1080)
Toggle mode keys:
 Toggle skeletons:  s
 Toggle bounding boxes:  b
 Toggle anonymizer mode:  a
 Toggle heatmaps:  h
 Toggle bodyparts:  p

Gstreamer pipeline:

aggregator name=agg ! glsvgoverlaysink name=overlaysink 
v4l2src device=/dev/video0 ! video/x-raw,width=1920,height=1080,framerate=30/1
 ! queue max-size-buffers=1 leaky=downstream ! identity ! glupload ! glvideoflip video-direction=identity ! tee name=t
           t. ! queue max-size-buffers=1 ! glfilterbin filter=glbox name=glbox ! video/x-raw,format=RGB,width=1024,height=768 !
                queue max-size-buffers=1 ! inference name=inf ! agg.
           t. ! queue max-size-buffers=1 ! agg.

Which I think is the expected behavior. But there there is no actual output video displayed. In the other USB camera examples a dialog box opens to display the output of the face detection model, for example. But I see no such dialog box. I have tried running the script remotely via MDT and on the coral itself.

[hjonnala]

Please check the input resolutions supported for usb camera with the following command and pass them as arguments. Thanks!

v4l2-ctl --list-formats-ext --device /dev/video1

example: python3 bodypix_gl_imx.py --videosrc /dev/video1 --width 640 --height 480

[andrubrown]

Thanks for your help.

The output of v4l2-ctl --list-formats-ext --device /dev/video1 is:

ioctl: VIDIOC_ENUM_FMT
    Type: Video Capture

    [0]: 'MJPG' (Motion-JPEG, compressed)
        Size: Discrete 1920x1080
            Interval: Discrete 0.033s (30.000 fps)
        Size: Discrete 640x480
            Interval: Discrete 0.033s (30.000 fps)
        Size: Discrete 1024x768
            Interval: Discrete 0.033s (30.000 fps)
        Size: Discrete 1280x720
            Interval: Discrete 0.033s (30.000 fps)
        Size: Discrete 1280x1024
            Interval: Discrete 0.033s (30.000 fps)
        Size: Discrete 2048x1536
            Interval: Discrete 0.033s (30.000 fps)
        Size: Discrete 320x240
            Interval: Discrete 0.033s (30.000 fps)
    [1]: 'YUYV' (YUYV 4:2:2)
        Size: Discrete 640x480
            Interval: Discrete 0.040s (25.000 fps)
        Size: Discrete 1024x768
            Interval: Discrete 0.100s (10.000 fps)
        Size: Discrete 1280x720
            Interval: Discrete 0.100s (10.000 fps)
        Size: Discrete 1280x1024
            Interval: Discrete 0.200s (5.000 fps)
        Size: Discrete 1920x1080
            Interval: Discrete 0.200s (5.000 fps)
        Size: Discrete 320x240
            Interval: Discrete 0.033s (30.000 fps)
        Size: Discrete 2048x1536
            Interval: Discrete 0.200s (5.000 fps)

When I run python3 bodypix_gl_imx.py --videosrc /dev/video1 --width 640 --height 480 I get the following error:

Model: models/bodypix_mobilenet_v1_075_1024_768_16_quant_decoder_edgetpu.tflite
Heatmap size:  (65, 49)
Stride:  16 (65, 49)
Inference size: (1024, 768)
Source size: (640, 480)
Toggle mode keys:
 Toggle skeletons:  s
 Toggle bounding boxes:  b
 Toggle anonymizer mode:  a
 Toggle heatmaps:  h
 Toggle bodyparts:  p

Gstreamer pipeline:

aggregator name=agg ! glsvgoverlaysink name=overlaysink 
v4l2src device=/dev/video1 ! video/x-raw,width=640,height=480,framerate=30/1
 ! queue max-size-buffers=1 leaky=downstream ! identity ! glupload ! glvideoflip video-direction=identity ! tee name=t
           t. ! queue max-size-buffers=1 ! glfilterbin filter=glbox name=glbox ! video/x-raw,format=RGB,width=1024,height=768 !
                queue max-size-buffers=1 ! inference name=inf ! agg.
           t. ! queue max-size-buffers=1 ! agg.

Traceback (most recent call last):
  File "bodypix_gl_imx.py", line 890, in do_fixate_src_caps
    self.ensure_pads_found()
  File "bodypix_gl_imx.py", line 843, in ensure_pads_found
    feature = caps.get_features(0).get_nth(0)
AttributeError: 'NoneType' object has no attribute 'get_features'
Error: gst-stream-error-quark: Internal data stream error. (1): gstbasesrc.c(3055): gst_base_src_loop (): /GstPipeline:pipeline0/GstV4l2Src:v4l2src0:
streaming stopped, reason not-negotiated (-4)

[hjonnala]

YUYV' (YUYV 4:2:2) Size: Discrete 320x240 Interval: Discrete 0.033s (30.000 fps)

Please try the YUUV resolutions with 30 fps.

python3 bodypix_gl_imx.py --videosrc /dev/video1 --width 320 --height 240

[andrubrown]

Thanks this works but I have a couple follow up questions.

1. YUYV 640x480 is available with this camera why does the resolution have to be changed to 320x240 to get this to work?
2. The demo works but the FPS is 1.95. Why is the frame rate so low?
3. I stopped the script with Ctrl+z. But when I try to run the model again I get the following error:

   
   Model: models/bodypix_mobilenet_v1_075_1024_768_16_quant_decoder_edgetpu.tflite
   Traceback (most recent call last):
   File "/usr/lib/python3/dist-packages/tflite_runtime/interpreter.py", line 160, in load_delegate
   delegate = Delegate(library, options)
   File "/usr/lib/python3/dist-packages/tflite_runtime/interpreter.py", line 119, in __init__
   raise ValueError(capture.message)
   ValueError

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "bodypix_gl_imx.py", line 1021, in main() File "bodypix_gl_imx.py", line 1002, in main engine = PoseEngine(model) File "/home/mendel/coral/project-bodypix/pose_engine.py", line 142, in init edgetpu_delegate = load_delegate(EDGETPU_SHARED_LIB) File "/usr/lib/python3/dist-packages/tflite_runtime/interpreter.py", line 163, in load_delegate library, str(e))) ValueError: Failed to load delegate from libedgetpu.so.1

Is this the expected behavior?

Thanks

[hjonnala]

• YUYV 640x480 is available with this camera why does the resolution have to be changed to 320x240 to get this to work?

This is because your only 320x240 supports 30 fps for your YUUV format..

• The demo works but the FPS is 1.95. Why is the frame rate so low?

Not sure why you are getting such low FPS..

• I stopped the script with Ctrl+z. But when I try to run the model again I get the following error:

This is expected. Please stop the script with CTRL+C.

Thanks!!

[hwjalapeno]

Hi @andrubrown

You have to use Ctrl+C to terminate any script or process, Ctrl+Z does not kill all processes and hence you cant run anything on EdgeTPU, in case you hit Ctrl+Z, sudo reboot now and thereafter use Ctrl+C only

Cheers!