module v4l2src0 reported: Internal data stream error.

Hey, I had this running on raspberryos for more than a year. Used it almost daily. I wanted to bump it to the latest version and I got this error. I started completely from scratch with a clean ubuntu server install on the rpi4. Same hardware that worked before. After adding my user to the video group I was finally getting video, but not in the script. I see the Input on videotest.py (installed gnome and used xforwarding...) but its showing the warning. Harmonize script itself wont work.

lsusb
Bus 003 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 001 Device 003: ID eba4:7588 USB3.0 HD Audio Capture USB3.0 HD Video Capture
Bus 001 Device 002: ID 2109:3431 VIA Labs, Inc. Hub
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub

v4l2-ctl -D
Driver Info:
        Driver name      : uvcvideo
        Card type        : USB3.0 HD Video Capture: USB3.0
        Bus info         : usb-0000:01:00.0-1.1
        Driver version   : 5.15.64
        Capabilities     : 0x84a00001
                Video Capture
                Metadata Capture
                Streaming
                Extended Pix Format
                Device Capabilities
        Device Caps      : 0x04200001
                Video Capture
                Streaming
                Extended Pix Format
Media Driver Info:
        Driver name      : uvcvideo
        Model            : USB3.0 HD Video Capture: USB3.0
        Serial           : HU123450
        Bus info         : usb-0000:01:00.0-1.1
        Media version    : 5.15.64
        Hardware revision: 0x00000328 (808)
        Driver version   : 5.15.64
Interface Info:
        ID               : 0x03000002
        Type             : V4L Video
Entity Info:
        ID               : 0x00000001 (1)
        Name             : USB3.0 HD Video Capture: USB3.0
        Function         : V4L2 I/O
        Flags            : default
        Pad 0x01000007   : 0: Sink
          Link 0x02000010: from remote pad 0x100000a of entity 'Processing 3' (Video Pixel Formatter): Data, Enabled, Immutable

v4l2-ctl --info -d /dev/video0 --list-formats-ext
Driver Info:
        Driver name      : uvcvideo
        Card type        : USB3.0 HD Video Capture: USB3.0
        Bus info         : usb-0000:01:00.0-1.1
        Driver version   : 5.15.64
        Capabilities     : 0x84a00001
                Video Capture
                Metadata Capture
                Streaming
                Extended Pix Format
                Device Capabilities
        Device Caps      : 0x04200001
                Video Capture
                Streaming
                Extended Pix Format
Media Driver Info:
        Driver name      : uvcvideo
        Model            : USB3.0 HD Video Capture: USB3.0
        Serial           : HU123450
        Bus info         : usb-0000:01:00.0-1.1
        Media version    : 5.15.64
        Hardware revision: 0x00000328 (808)
        Driver version   : 5.15.64
Interface Info:
        ID               : 0x03000002
        Type             : V4L Video
Entity Info:
        ID               : 0x00000001 (1)
        Name             : USB3.0 HD Video Capture: USB3.0
        Function         : V4L2 I/O
        Flags            : default
        Pad 0x01000007   : 0: Sink
          Link 0x02000010: from remote pad 0x100000a of entity 'Processing 3' (Video Pixel Formatter): Data, Enabled, Immutable
ioctl: VIDIOC_ENUM_FMT
        Type: Video Capture

        [0]: 'MJPG' (Motion-JPEG, compressed)
                Size: Discrete 1920x1080
                        Interval: Discrete 0.017s (60.000 fps)
                        Interval: Discrete 0.033s (30.000 fps)
                Size: Discrete 1280x720
                        Interval: Discrete 0.017s (60.000 fps)
                        Interval: Discrete 0.033s (30.000 fps)
                Size: Discrete 960x540
                        Interval: Discrete 0.017s (60.000 fps)
                        Interval: Discrete 0.033s (30.000 fps)
                Size: Discrete 800x450
                        Interval: Discrete 0.017s (60.000 fps)
                        Interval: Discrete 0.033s (30.000 fps)
                Size: Discrete 640x360
                        Interval: Discrete 0.017s (60.000 fps)
                        Interval: Discrete 0.033s (30.000 fps)

python3 videotest.py
Press ESC to exit.
[ WARN:0] global ./modules/videoio/src/cap_gstreamer.cpp (2075) handleMessage OpenCV | GStreamer warning: Embedded video playback halted; module v4l2src0 reported: Internal data stream error.
[ WARN:0] global ./modules/videoio/src/cap_gstreamer.cpp (1053) open OpenCV | GStreamer warning: unable to start pipeline
[ WARN:0] global ./modules/videoio/src/cap_gstreamer.cpp (616) isPipelinePlaying OpenCV | GStreamer warning: GStreamer: pipeline have not been created

./harmonize.py -v
--- Starting Harmonize Project ---
INFO: Detected Philips Hue - 6D35D3._hue._tcp.local. via mDNS at IP address: 172.17.172.8
INFO: Single Hue bridge detected on network via mDNS.
INFO: Hue bridge located at: 172.17.172.8
Checking whether Harmonizer application is already registered (Looking for client.json file).
INFO: Client data found from client.json file.
INFO: Client data valid: {'username': 'XXXXXXXXXXXXXXXX', 'clientkey': 'XXXXXXXXXXXXXXXXXXXXXXXX'}
Requesting bridge information...
INFO: The bridge is capable of streaming via APIv2. Firmware version 1955082050 detected...
Querying hue bridge for entertainment areas on local network.
groupid =  7
Using Entertainment area with group_id: 7
Selected Entertainment UDID: XXXXXXXXXXXXXXXXXXXXXXXX
INFO: 5 light(s) found in selected Entertainment area. Locations [x,y,z] are as follows:
 {'0': [-0.93, -0.28, 0.0], '1': [1.0, 0.43, 1.0], '2': [-1.0, 0.92, 0.0], '3': [0.95999, 0.68997, 0.0], '4': [0.91, 0.97, 0.0]}
Hue application id: 3c90b91a-6f58-4272-ae4b-987a6257ba27
Enabling streaming to your Entertainment area
{'data': [{'rid': 'XXXXXXXXXXXXXXXXXXXXXXX', 'rtype': 'entertainment_configuration'}], 'errors': []}
Starting computer vision engine...
OpenCV version: 4.5.4
--- INFO: Detected video capture card on /dev/video0 ---
[ WARN:0] global ./modules/videoio/src/cap_gstreamer.cpp (2075) handleMessage OpenCV | GStreamer warning: Embedded video playback halted; module v4l2src0 reported: Internal data stream error.
[ WARN:0] global ./modules/videoio/src/cap_gstreamer.cpp (1053) open OpenCV | GStreamer warning: unable to start pipeline
[ WARN:0] global ./modules/videoio/src/cap_gstreamer.cpp (616) isPipelinePlaying OpenCV | GStreamer warning: GStreamer: pipeline have not been created
Disabling streaming on Entertainment area...
{'data': [{'rid': 'XXXXXXXXXXXXXXXXXXXXXXXX', 'rtype': 'entertainment_configuration'}], 'errors': []}
ERROR: Unable to open capture device.

Okay - I figured out the Test script is calling the VideoCapture() method with just the index of the video device while the script itself gives the parameter cv2.CAP_GSTREAMER.

vc = cv2.VideoCapture(0)
vc = cv2.VideoCapture(0,cv2.CAP_GSTREAMER)

by removing the parameter the script seems to run - I can see that sync is active in the hue app. Unfortunately im not at home atm so I cant see if the lights are actually changing 🤣

I checked with videotest.py that the resolution is 640x360:

print("width: ")
print(vc.get(cv2.CAP_PROP_FRAME_WIDTH))
print("height: ")
print(vc.get(cv2.CAP_PROP_FRAME_HEIGHT))

I dont know if calling with cv2.CAP_GSTREAMER changes the resolution... I am to dumb to find a proper documentation I understand..

If found this:

def __gstreamer_pipeline(
        camera_id,
        capture_width=1920,
        capture_height=1080,
        display_width=1920,
        display_height=1080,
        framerate=30,
        flip_method=0,
    ):
    return (
            "nvarguscamerasrc sensor-id=%d ! "
            "video/x-raw(memory:NVMM), "
            "width=(int)%d, height=(int)%d, "
            "format=(string)NV12, framerate=(fraction)%d/1 ! "
            "nvvidconv flip-method=%d ! "
            "video/x-raw, width=(int)%d, height=(int)%d, format=(string)BGRx ! "
            "videoconvert ! "
            "video/x-raw, format=(string)BGR ! appsink max-buffers=1 drop=True"
            % (
                    camera_id,
                    capture_width,
                    capture_height,
                    framerate,
                    flip_method,
                    display_width,
                    display_height,
            )
    )

stream = cv2.VideoCapture(__gstreamer_pipeline(camera_id=0, flip_method=2), cv2.CAP_GSTREAMER)

Adding the define and changing the call of VideoCapture seems not to provoke the warnings/errors.. however, I dont get any video shown. I am a bit lost now.. I will try again when I have local access and dont have to fiddle with VPN.

At home.. changing vc = cv2.VideoCapture(0,cv2.CAP_GSTREAMER) to vc = cv2.VideoCapture(0) seems to have it solved for me. Lights are running back again - Yay! I cant see a difference to before.

Adding the last codesnippet above seems not to open the videodevice - just nothing happens. Seems like gstreamer is waiting for something.

Maybe somebody is able to explain ^^

Great news!

What’s interesting is that VideoCapture should default to GStreamer anyway, so don’t quite understand either what the fuss is except if the VideoCapture is actually capturing via FFMPEG instead.

If you would be willing to let me know if the following line works I would greatly appreciate it:

cap = cv2.VideoCapture(0,cv2.CAP_FFMPEG)

The code was revamped to mainly support gradient light strips, so the script should perform the same generally.

Thanks, Brad

On Sat, Jan 7, 2023 at 9:12 AM vbartels @.***> wrote:

At home.. changing vc = cv2.VideoCapture(0,cv2.CAP_GSTREAMER) to vc = cv2.VideoCapture(0) seems to have it solved for me. Lights are running back again - Yay! I cant see a difference to before.

Adding the last codesnippet above seems not to open the videodevice - just nothing happens. Seems like gstreamer is waiting for something.

Maybe somebody is able to explain ^^

— Reply to this email directly, view it on GitHub https://github.com/MCPCapital/HarmonizeProject/issues/34#issuecomment-1374494508, or unsubscribe https://github.com/notifications/unsubscribe-auth/AANVBG47VCZIQSG74I654UTWRF2U5ANCNFSM6AAAAAATQVSL4Y . You are receiving this because you are subscribed to this thread.Message ID: @.***>

I tried that at first, because the line is still in the code. Unfortunately it results in:

./harmonize.py
--- Starting Harmonize Project ---
INFO: Single Hue bridge detected on network via mDNS.
Querying hue bridge for entertainment areas on local network.
groupid =  7
Using Entertainment area with group_id: 7
Starting computer vision engine...
--- INFO: Detected video capture card on /dev/video0 ---
Disabling streaming on Entertainment area...
ERROR: Unable to open capture device.

Thanks for trying it out. FFMPEG is arguably faster than GStreamer but I’ve never got it working.

I’ll have to decide how to update the code if as you say it works with no parameters. I was using OpenCV 4.6. Perhaps 4.5 has some differences?

Brad

On Sat, Jan 7, 2023 at 9:55 AM vbartels @.***> wrote:

I tried that at first, because the line is still in the code. Unfortunately it results in:

./harmonize.py --- Starting Harmonize Project --- INFO: Single Hue bridge detected on network via mDNS. Querying hue bridge for entertainment areas on local network. groupid = 7 Using Entertainment area with group_id: 7 Starting computer vision engine... --- INFO: Detected video capture card on /dev/video0 --- Disabling streaming on Entertainment area... ERROR: Unable to open capture device.

— Reply to this email directly, view it on GitHub https://github.com/MCPCapital/HarmonizeProject/issues/34#issuecomment-1374506118, or unsubscribe https://github.com/notifications/unsubscribe-auth/AANVBG67GPXW34EZ6WW2CTTWRF7UXANCNFSM6AAAAAATQVSL4Y . You are receiving this because you commented.Message ID: @.***>

Not knowing much about python I fiddled around half the day. my OpenCV has definetely ffmpeg support builtin:

>>> import cv2
>>> cv2.__version__
>>> info = cv2.getBuildInformation()
>>> video, parallel = info.index('Video'), info.index('Parallel')
>>> print(info[video:parallel])

results in:

'4.5.4'

Video I/O:
    DC1394:                      YES (2.2.6)
    FFMPEG:                      YES
      avcodec:                   YES (58.134.100)
      avformat:                  YES (58.76.100)
      avutil:                    YES (56.70.100)
      swscale:                   YES (5.9.100)
      avresample:                NO
    GStreamer:                   YES (1.19.90)
    PvAPI:                       NO
    v4l/v4l2:                    YES (linux/videodev2.h)
    gPhoto2:                     YES

But I didnt get any further. I also took a look into: ffmpegcv but was not able to get it reading from /dev/video0 . Sadly all the documentations are terrible....

MCPCapital / HarmonizeProject

module v4l2src0 reported: Internal data stream error. #34