Missing a required plugin (tcp)

[joachimnielandt]

Hi all, I'm trying to get a basic video stream up and running using JavaCV but I guess I'm stumbling due to environment issues. The stream is started on a raspberry pi (raspi cam) using this command: raspivid -t 0 -b 2000000 -fps 30 -w 800 -h 600 -o - | nc -l -k -v 192.168.178.111 2222 Then, on my development machine, I can open this using the python example:

import cv2
# Create a VideoCapture object and read from input file
# If the input is the camera, pass 0 instead of the video file name
cap = cv2.VideoCapture('tcp://raspberrypi.fritz.box:2222')

However, using the same approach in JavaCV (using Kotlin)...

private val capture = VideoCapture()
capture.open("tcp://raspberrypi.fritz.box:2222")

... I'm getting this error:

[ WARN:0] global /build/opencv/src/opencv-4.3.0/modules/videoio/src/cap_gstreamer.cpp (1743) handleMessage OpenCV | GStreamer warning: your GStreamer installation is missing a required plugin
[ WARN:0] global /build/opencv/src/opencv-4.3.0/modules/videoio/src/cap_gstreamer.cpp (1758) handleMessage OpenCV | GStreamer warning: Embedded video playback halted; module uridecodebin0 reported: No URI handler implemented for "tcp".
[ WARN:0] global /build/opencv/src/opencv-4.3.0/modules/videoio/src/cap_gstreamer.cpp (888) open OpenCV | GStreamer warning: unable to start pipeline

I tried using OpenCV through the maven dependency, and through the locally installed /usr/share/java/opencv4/opencv-430.jar, same result. Before loading the java GUI, I call this System.loadLibrary(Core.NATIVE_LIBRARY_NAME) method. Any ideas? Weirdly enough, when troubleshooting and testing different configurations, I had exactly 1 attempt that actually got a frame through, but subsequent attempt again failed. Perhaps there's a race condition with library loading?

[saudet]

Try to use FFmpegFrameGrabber instead.

[joachimnielandt]

Hi Saudet, as far as I understood raspivid outputs raw h264. I tried ffmpeggrabber (and additionally added .setFormat("h264") as it complained about setFormat not being called, but that resulted in the following error:

[h264 @ 0x7f75e0b26640] no frame!
[h264 @ 0x7f75e0b26640] non-existing PPS 0 referenced
[h264 @ 0x7f75e0b1f8c0] decoding for stream 0 failed
[h264 @ 0x7f75e0b1f8c0] Could not find codec parameters for stream 0 (Video: h264, none): unspecified size
Consider increasing the value for the 'analyzeduration' and 'probesize' options
Input #0, h264, from 'tcp://raspberrypi.fritz.box:2222':
  Duration: N/A, bitrate: N/A
    Stream #0:0: Video: h264, none, 25 fps, 25 tbr, 1200k tbn, 50 tbc

Would it be feasible to have a FrameGrabber take in the raw h264 stream in JavaCV? Or is it a must to get an FFMPEG stream going?

[joachimnielandt]

Okay, think I got something working (going off different examples on the interwebs: Replacing my server by this:

raspivid -t 0 -l -o tcp://0.0.0.0:3333

I get a frame going on my client side (using the ffmpeggrabber, setFormat H264). Time to start tinkering with options, but my problem seems not to have been with JavaCV... Thanks!

[saudet]

Ok, good! BTW, I'm pretty sure Raspberry Pi supports v4l2 to capture from its camera.

/cc @vb216

[vb216]

I'm guessing you're intentionally wanting to stream the video, rather than process directly on the Pi? You can run JavaCV natively on the Pi, so long as you're not doing too much image work the performance isn't too bad. On the Pi: FFmpegFrameGrabber grabber = new FFmpegFrameGrabber("/dev/video0");

'Should' just work.. But yeah v4l2 rings a bell too, if you're wanting to open the stream and push it over the network for processing on another machine

[joachimnielandt]

Yes, I got another server always running that is way more capable, and I'd like the ease of use of processing on my dev machine, server, ... and not be dependent on the raspberry pi for things like visualisation.

For reference, I got a working environment now, using V4L2:

v4l2-ctl --set-fmt-video=width=800,height=600,pixelformat=4
v4l2-ctl -c sharpness=30,compression_quality=100,video_bitrate_mode=1,video_bitrate=200000
cvlc --no-audio v4l2:///dev/video0 --v4l2-width 800 --v4l2-height 600 --v4l -chroma h264 --v4l2-fps 30 --sout '#standard{access=http,mux=ts,dst=:2222/}' -I "dummy"

This can be read by VLC on the client side: vlc -vvv http://192.168.178.111:2222 :demux=ffmpeg or in JavaCV:

grabber = FFmpegFrameGrabber("http://...:2222")
converter = JavaFXFrameConverter()
image = converter.convert(grabber.grabImage())
Platform.runLater {
    currentFrame.imageProperty().set(image) //currentFrame is an ImageFrame
}