search

mpromonet / v4l2rtspserver

RTSP Server for V4L2 device capture supporting HEVC/H264/JPEG/VP8/VP9
The Unlicense
1.82k stars 423 forks source link

v4l2rtspserver breaks on Raspberry Pi OS bullseye #257

Open danieltwagner opened 2 years ago

danieltwagner commented 2 years ago

Describe the bug With the switch to Debian bullseye, Raspberry Pi OS has adopted libcamera which seems to have changed how data is made available via v4l2, with h264, jpeg etc no longer being available.

To Reproduce I installed the current version of Raspberry Pi OS, enabled the camera interface, and captured the following data:

v4l2 output:

$ v4l2-ctl --list-devices
unicam (platform:20801000.csi):
    /dev/video0
    /dev/video1
    /dev/media2
[...]

$ v4l2-ctl -d /dev/video0 --list-formats-ext
ioctl: VIDIOC_ENUM_FMT
    Type: Video Capture

    [0]: 'pBAA' (10-bit Bayer BGBG/GRGR Packed)
        Size: Discrete 3280x2464
        Size: Discrete 1920x1080
        Size: Discrete 1640x1232
        Size: Discrete 640x480
    [1]: 'BG10' (10-bit Bayer BGBG/GRGR)
        Size: Discrete 3280x2464
        Size: Discrete 1920x1080
        Size: Discrete 1640x1232
        Size: Discrete 640x480
    [2]: 'BA81' (8-bit Bayer BGBG/GRGR)
        Size: Discrete 3280x2464
        Size: Discrete 1920x1080
        Size: Discrete 1640x1232
        Size: Discrete 640x480

Example raspistill invocation:

$ libcamera-still -v -o test.jpg
Options:
    verbose: 1
    info_text:#%frame (%fps fps) exp %exp ag %ag dg %dg
    timeout: 5000
    width: 0
    height: 0
    output: test.jpg
    post_process_file:
    rawfull: 0
    preview: default
    qt-preview: 0
    transform: identity
    roi: all
    metering: centre
    exposure: normal
    ev: 0
    awb: auto
    flush: false
    wrap: 0
    brightness: 0
    contrast: 1
    saturation: 1
    sharpness: 1
    framerate: 30
    denoise: auto
    viewfinder-width: 0
    viewfinder-height: 0
    tuning-file: (libcamera)
    lores-width: 0
    lores-height: 0
    encoding: jpg
    quality: 93
    raw: 0
    restart: 0
    timelapse: 0
    framestart: 0
    datetime: 0
    timestamp: 0
    keypress: 0
    signal: 0
    thumbnail width: 320
    thumbnail height: 240
    thumbnail quality: 70
    latest:
    immediate 0
No connector ID specified.  Choosing default from list:
Connector 32 (crtc 0): type 11, 0x0
Connector 49 (crtc 83): type 5, 720x480 (chosen)
Opening camera...
[1:08:43.836864267] [3649]  INFO Camera camera_manager.cpp:293 libcamera v0.0.0+3156-f4070274
Acquired camera /base/soc/i2c0mux/i2c@1/imx219@10
Configuring viewfinder...
Viewfinder size chosen is 1640x1232
Final viewfinder size is 1640x1232
[1:08:44.136273476] [3649]  INFO Camera camera.cpp:937 configuring streams: (0) 1640x1232-YUV420
[1:08:44.139077534] [3651]  INFO RPI raspberrypi.cpp:122 Mode: 3280x2464 fmt RG10 Score: 2218 (best 2218)
[1:08:44.140220558] [3651]  INFO RPI raspberrypi.cpp:122 Mode: 1920x1080 fmt RG10 Score: 2041.48 (best 2041.48)
[1:08:44.141220579] [3651]  INFO RPI raspberrypi.cpp:122 Mode: 1640x1232 fmt RG10 Score: 1500 (best 1500)
[1:08:44.142178599] [3651]  INFO RPI raspberrypi.cpp:122 Mode: 640x480 fmt RG10 Score: 5004.81 (best 1500)
[1:08:44.143152619] [3651]  INFO RPI raspberrypi.cpp:122 Mode: 3280x2464 fmt pRAA Score: 1718 (best 1500)
[1:08:44.144123639] [3651]  INFO RPI raspberrypi.cpp:122 Mode: 1920x1080 fmt pRAA Score: 1541.48 (best 1500)
[1:08:44.145088659] [3651]  INFO RPI raspberrypi.cpp:122 Mode: 1640x1232 fmt pRAA Score: 1000 (best 1000)
[1:08:44.146052679] [3651]  INFO RPI raspberrypi.cpp:122 Mode: 640x480 fmt pRAA Score: 4504.81 (best 1000)
[1:08:44.147047700] [3651]  INFO RPI raspberrypi.cpp:122 Mode: 3280x2464 fmt RGGB Score: 3218 (best 1000)
[1:08:44.148025720] [3651]  INFO RPI raspberrypi.cpp:122 Mode: 1920x1080 fmt RGGB Score: 3041.48 (best 1000)
[1:08:44.148993740] [3651]  INFO RPI raspberrypi.cpp:122 Mode: 1640x1232 fmt RGGB Score: 2500 (best 1000)
[1:08:44.149954760] [3651]  INFO RPI raspberrypi.cpp:122 Mode: 640x480 fmt RGGB Score: 6004.81 (best 1000)
[1:08:44.151476791] [3651]  INFO RPI raspberrypi.cpp:620 Sensor: /base/soc/i2c0mux/i2c@1/imx219@10 - Selected mode: 1640x1232-pRAA
Camera streams configured
Buffers allocated and mapped
Viewfinder setup complete
Requests created
[1:08:44.294774761] [3651]  INFO RPISTREAM rpi_stream.cpp:122 No buffers available for ISP Output0
[1:08:44.299829866] [3651]  INFO RPISTREAM rpi_stream.cpp:122 No buffers available for ISP Output0
[1:08:44.301538901] [3651]  INFO RPISTREAM rpi_stream.cpp:122 No buffers available for ISP Output0
Camera started!
[...]

Similar output and format selection happens when running libcamera-vid -v -o out.h264.

Example v4l2rtspserver invocation:

$ v4l2rtspserver -W 1640 -H 1232 -F 25 -P 8555 -S /dev/video0
log level:500

[NOTICE] /home/pi/v4l2rtspserver/main.cpp:294
    Version: 0.2.4-6-g90dfcb1 live555 version:2021.11.10
[NOTICE] /home/pi/v4l2rtspserver/src/V4l2RTSPServer.cpp:37
    Create V4L2 Source.../dev/video0
[NOTICE] /home/pi/v4l2rtspserver/v4l2wrapper/src/V4l2Device.cpp:133
    driver:unicam capabilities:85a00001 mandatory:4000001
[NOTICE] /home/pi/v4l2rtspserver/v4l2wrapper/src/V4l2Device.cpp:136
    /dev/video0 support capture
[NOTICE] /home/pi/v4l2rtspserver/v4l2wrapper/src/V4l2Device.cpp:138
    /dev/video0 support read/write
[NOTICE] /home/pi/v4l2rtspserver/v4l2wrapper/src/V4l2Device.cpp:139
    /dev/video0 support streaming
[ERROR] /home/pi/v4l2rtspserver/v4l2wrapper/src/V4l2Device.cpp:212
    /dev/video0: Cannot set pixelformat to:H264 format is:pBAA
[ERROR] /home/pi/v4l2rtspserver/v4l2wrapper/src/V4l2Device.cpp:212
    /dev/video0: Cannot set pixelformat to:MJPG format is:pBAA
[ERROR] /home/pi/v4l2rtspserver/v4l2wrapper/src/V4l2Device.cpp:212
    /dev/video0: Cannot set pixelformat to:JPEG format is:pBAA
[ERROR] /home/pi/v4l2rtspserver/v4l2wrapper/src/V4l2Device.cpp:83
    Cannot init device:/dev/video0
[NOTICE] /home/pi/v4l2rtspserver/v4l2wrapper/src/V4l2MmapDevice.cpp:141
VIDIOC_STREAMOFF: Bad file descriptor
VIDIOC_REQBUFS: Bad file descriptor
    Device /dev/video0
[NOTICE] /home/pi/v4l2rtspserver/inc/V4l2RTSPServer.h:163
    HLS       http://192.168.101.153:8555/ts.m3u8
[NOTICE] /home/pi/v4l2rtspserver/inc/V4l2RTSPServer.h:164
    MPEG-DASH http://192.168.101.153:8555/ts.mpd

I'm afraid I don't really know much about v4l2 but would be happy to provide further information if that's at all useful.

mpromonet commented 2 years ago

Hi @danieltwagner

I don't try libcamera-apps, it seems this is capturing a v4l2 device and compress the stream ?

v4l2rtspserver read a v4l2 device, and support some format (h264, h265, vp8, jpeg, and maybe yuyv). From what we see from v4l2-ctl -d /dev/video0 --list-formats-ext , your device support pBAA, BG10 & BA81. None of these format are supported by v4l2rtspserver.

You may look to v4l2loopback that may be an intermediate device between the compression done by libcamera-apps and v4l2rtspserver.

Best Regards, Michel.

jasp106 commented 2 years ago

Hello,

right now i have a setup where i use rpos from Breeze. Unfortunatly i run into the same problem as @danieltwagner. My v4l2rtspServer is working well on Buster, the second i switch to Bullseye with the new way libcamera is implemented my setup is in trouble. My attempts to fix this issue ended up in failure. If someone is capable of fixing this i would appreciate it. The moment i get a working workaround i will post it here.

Greetings, Jasp

jasp106 commented 2 years ago

Hello,

My workaround now is mentioned here: https://forums.raspberrypi.com/viewtopic.php?t=323390&sid=9327da4837fe5a986060d9e3c64c46c1

It is just some simple config setup, which will end up in rebooting the old v4l2driver.

The Solution was to edit /boot/config.txt, remove the line "camera_auto_detect=1", and add "start_x=1" and "gpu_mem=128".

I hope that this might help someones setup.

Greetings, Jasp

cyrildtm commented 2 years ago

Hello,

My workaround now is mentioned here: https://forums.raspberrypi.com/viewtopic.php?t=323390&sid=9327da4837fe5a986060d9e3c64c46c1

It is just some simple config setup, which will end up in rebooting the old v4l2driver.

The Solution was to edit /boot/config.txt, remove the line "camera_auto_detect=1", and add "start_x=1" and "gpu_mem=128".

I hope that this might help someones setup.

Greetings, Jasp

Good job finding the temporary fix, but as you pointed out in your original post, it rolls back to the old camera driver, so it's really a "workaround". I have the same concern and I know any new drivers will break my camera streaming setup, so I'm holding out until everything is ready.

Rakhmanov commented 2 years ago

You may look to v4l2loopback that may be an intermediate device between the compression done by libcamera-apps and v4l2rtspserver.

I have attempted this configuration on Raspberry PI B3+ and it worked somewhat but did not produce the video upon the connection to RTSP server.

Issue is resolved when editing: sudo nano /boot/config.txt

#camera_auto_detect=1

start_x=1
gpu_mem=128

Reboot and the driver is now not unicam but familiar:

mmal service 16.1 (platform:bcm2835-v4l2-0):
        /dev/video0

When camera_auto_detect=1 is enabled the output looks like this:

v412-ctl --list-devices looks like this:

 v4l2-ctl --list-devices
unicam (platform:3f801000.csi):
        /dev/video0
        /dev/video1
        /dev/media2

bcm2835-codec-decode (platform:bcm2835-codec):
        /dev/video10
        /dev/video11
        /dev/video12
        /dev/video18
        /dev/media1

bcm2835-isp (platform:bcm2835-isp):
        /dev/video13
        /dev/video14
        /dev/video15
        /dev/video16
        /dev/media0

Dummy video device (0x0000) (platform:v4l2loopback-000):
        /dev/video2

If anyone wants to reproduce:

sudo apt-get update && sudo apt-get upgrade
sudo apt-get install cmake liblog4cpp5-dev libv4l-dev libssl-dev git raspberrypi-kernel-headers autoconf libtool -y
git clone https://github.com/umlaeute/v4l2loopback.git; (cd v4l2loopback/ ; make clean; sudo make install; sudo depmod -a ; sudo modprobe v4l2loopback)
git clone https://github.com/mpromonet/v4l2rtspserver.git; (cd v4l2rtspserver/ ; cmake . ; make ; sudo make install)
git clone https://github.com/mpromonet/v4l2tools.git ; (cd v4l2tools/ ; make ; make install)

sudo modprobe v4l2loopback

# Find the device that v4l2loopback is running on with helper command below or manually `v4l2-ctl --list-devices` looking for v4l2loopback. 

LOOPBACK_DEVICE=$(v4l2-ctl --list-devices | grep v4l2loopback -A1 | awk 'FNR == 2 {print $1}')

./v4l2tools/v4l2compress -fH264  /dev/video0 $LOOPBACK_DEVICE &
v4l2rtspserver -W 640 -H 480 -F 15 -P 8554  $LOOPBACK_DEVICE
rogithub commented 2 years ago

I tried loading driver... With no success which in docs camera seems to be the one for this $ sudo modprobe bcm2835-unicam $ sudo modprobe v4l2loopback $ v4l2compress -fH264 /dev/video0 /dev/video17 & v4l2rtspserver /dev/video17 &

With no success I connected that one on https://github.com/umlaeute/v4l2loopback but even so it is not working...

Raspberry pi 4 8gb bullseye lite (libcamera)

Devices 

~ » v4l2-ctl --list-devices                                                                                                      
bcm2835-codec-decode (platform:bcm2835-codec):
    /dev/video10
    /dev/video11
    /dev/video12
    /dev/video18
    /dev/media2

bcm2835-isp (platform:bcm2835-isp):
    /dev/video13
    /dev/video14
    /dev/video15
    /dev/video16
    /dev/media1

unicam (platform:fe801000.csi):
    /dev/video0
    /dev/video1
    /dev/media0

Dummy video device (0x0000) (platform:v4l2loopback-000):
    /dev/video2

looking forward to making it work on bullseye. My output


~ » v4l2compress -fH264 /dev/video0 /dev/video2 & v4l2rtspserver /dev/video2 &                                                   
[1] 29119
[2] 29120
2021-11-21 16:53:46,462 [NOTICE] - src/V4l2Device.cpp:133                                                                                   
    driver:unicam capabilities:85a00001 mandatory:4000001
2021-11-21 16:53:46,462 [NOTICE] - src/V4l2Device.cpp:136
    /dev/video0 support capture
2021-11-21 16:53:46,463 [NOTICE] - src/V4l2Device.cpp:138
    /dev/video0 support read/write
2021-11-21 16:53:46,463 [NOTICE] - src/V4l2Device.cpp:139
    /dev/video0 support streaming
2021-11-21 16:53:46,463 [NOTICE] - src/V4l2Device.cpp:225
    /dev/video0:pBAA size:1640x1232 bufferSize:2562560
2021-11-21 16:53:46,463 [NOTICE] - src/V4l2MmapDevice.cpp:49
    Device /dev/video0
--------------------------------------------------------------------------------------------------------------------------------------------
~ » 2021-11-21 16:53:46,490 [NOTICE] - src/V4l2MmapDevice.cpp:73                                                                 
    Device /dev/video0 nb buffer:10
log level:500

[NOTICE] /home/ro/Documents/code/v4l2rtspserver/main.cpp:294
    Version: 0.2.4-7-g9638a15-dirty live555 version:2020.01.19
[NOTICE] /home/ro/Documents/code/v4l2rtspserver/src/V4l2RTSPServer.cpp:37
    Create V4L2 Source.../dev/video2
[NOTICE] /home/ro/Documents/code/v4l2rtspserver/v4l2wrapper/src/V4l2Device.cpp:133
    driver:v4l2 loopback capabilities:85208003 mandatory:4000001
[NOTICE] /home/ro/Documents/code/v4l2rtspserver/v4l2wrapper/src/V4l2Device.cpp:135
    /dev/video2 support output
[NOTICE] /home/ro/Documents/code/v4l2rtspserver/v4l2wrapper/src/V4l2Device.cpp:136
    /dev/video2 support capture
[NOTICE] /home/ro/Documents/code/v4l2rtspserver/v4l2wrapper/src/V4l2Device.cpp:138
    /dev/video2 support read/write
[NOTICE] /home/ro/Documents/code/v4l2rtspserver/v4l2wrapper/src/V4l2Device.cpp:139
    /dev/video2 support streaming
[ERROR] /home/ro/Documents/code/v4l2rtspserver/v4l2wrapper/src/V4l2Device.cpp:192
    /dev/video2: Cannot get format Invalid argument
[ERROR] /home/ro/Documents/code/v4l2rtspserver/v4l2wrapper/src/V4l2Device.cpp:192
    /dev/video2: Cannot get format Invalid argument
[ERROR] /home/ro/Documents/code/v4l2rtspserver/v4l2wrapper/src/V4l2Device.cpp:192
    /dev/video2: Cannot get format Invalid argument
[ERROR] /home/ro/Documents/code/v4l2rtspserver/v4l2wrapper/src/V4l2Device.cpp:83
    Cannot init device:/dev/video2
[NOTICE] /home/ro/Documents/code/v4l2rtspserver/v4l2wrapper/src/V4l2MmapDevice.cpp:141
VIDIOC_STREAMOFF: Bad file descriptor
VIDIOC_REQBUFS: Bad file descriptor
    Device /dev/video2
[2]  + 29120 done       v4l2rtspserver /dev/video2
--------------------------------------------------------------------------------------------------------------------------------------------
~ » 2021-11-21 16:53:46,544 [NOTICE] - src/V4l2Device.cpp:133                                                                    
    driver:v4l2 loopback capabilities:85208003 mandatory:4000002
2021-11-21 16:53:46,545 [NOTICE] - src/V4l2Device.cpp:135
    /dev/video2 support output
2021-11-21 16:53:46,545 [NOTICE] - src/V4l2Device.cpp:136
    /dev/video2 support capture
2021-11-21 16:53:46,545 [NOTICE] - src/V4l2Device.cpp:138
    /dev/video2 support read/write
2021-11-21 16:53:46,545 [NOTICE] - src/V4l2Device.cpp:139
    /dev/video2 support streaming
2021-11-21 16:53:46,551 [NOTICE] - src/V4l2Device.cpp:225
    /dev/video2:H264 size:1640x1232 bufferSize:8085504
2021-11-21 16:53:46,551 [NOTICE] - src/V4l2MmapDevice.cpp:49
    Device /dev/video2
2021-11-21 16:53:46,551 [NOTICE] - src/V4l2MmapDevice.cpp:73
    Device /dev/video2 nb buffer:2
2021-11-21 16:53:46,560 [NOTICE] - src/v4l2compress.cpp:74
    Start Compressing to /dev/video2

[1]  + 29119 segmentation fault  v4l2compress -fH264 /dev/video0 /dev/video2