search

guysoft / OctoPi

Scripts to build OctoPi, a Raspberry PI distro for controlling 3D printers over the web
GNU General Public License v3.0
2.48k stars 367 forks source link

[Enhancement] Simplify multiple webcam handling and number of mjpg-streamer instances #750

Open SilverWingedHawk opened 3 years ago

SilverWingedHawk commented 3 years ago

mjpg-streamer is able to handle multiple input and/or output streams (by specifying multiple -i or -o flags). In order to reduce the overhead created by spawning multiple instances of mjpg-streamer, as well as simplifying the reverse proxy configuration (either HAProxy or other reverse proxy software), I've modified the webcamd script to be able to allow for multiple input streams.

If multiple input streams are available, a setting in octopi.txt or any supplementary *.txt config file is introduced, which will cap the maximum input streams to 2 so that the pi doesn't suffer too much strain. The default value is now 2, but that can be reduced or increased, I just thought it would be a sensible default.

Also, this modification removes the limitation of offering either USB or Raspicam support, both should be available with this modification. The script is now able to add multiple stream input automatically, not limited to the first /dev/video device encountered. It also relies on the v4l2-ctl command (available through the v4l-utils package) to determine whether a /dev/video* device offers Video Capture capabilities and so can be used as an input stream for mjpg-streamer.

Unfortunately, I don't own an Rpi device nor a Raspicam, so I'm not able to test those functionalities on real hardware, though the script runs the appropriate commands in a similar Linux environment.

This would actually solve #677.

I welcome any feedback you might have regarding this PR.

Best regards

guysoft commented 3 years ago

Nice! have you tested it? If not what got you to write this?

SilverWingedHawk commented 3 years ago

Unfortunately I haven't tested it on real hardware as stated in my message, I don't own a Raspberry Pi, nor a RaspiCam. I've just tested that the script calls the correct commands with different situations I'm able to reproduce on a Linux (Fedora) machine.

What prompted this PR was that I'd seen my father struggle with his own Rpi setting up multiple webcams months ago. He switched to a full blown computer with Fedora, and I installed OctoPrint on it, and fiddled with mjpg-streamer a bit. Going to the forums I saw that multicam support on OctoPi was a bit hard for newcomers to set up, and I thought that I could write something up to ease the configuration of multiple webcams, hence this PR

cp2004 commented 3 years ago

This would make my guide https://community.octoprint.org/t/setting-up-multiple-webcams-in-octopi-the-right-way/32669 easier, as we could skip the haproxy step, one less thing to break for sure!

I will try and test/review this at some point, but I'm not a script-person so it would mostly be from a potential user's point of view.

guysoft commented 3 years ago

@cp2004 Any chance you could test it before I merge? Or alternatively , i can merge and you can test the nightly?

I just dont want to ship it unless someone tests it

cp2004 commented 3 years ago

I can try and give it a go over this weekend.

cp2004 commented 3 years ago

Sorry, haven't been able to test this yet. Slightly forgot about it and made a new OctoPrint plugin instead. Will get on to it!

cp2004 commented 3 years ago

Hmm, this doesn't entirely work on my Pi. Currently, it has no webcams plugged in, and with and without nb_max_camera=2 set in the config. I have two config files setup, as I would normally for multiple webcams.

webcamd output ``` Starting up webcamDaemon... --- Configuration: ---------------------------- cfg_file: /boot/octopi.txt camera: usb usb options: -r 640x480 -f 10 -d /dev/video0 raspi options: -fps 10 http options: -w ./www-octopi -n --listen 127.0.0.1 Explicitly set USB device: /dev/video0 ----------------------------------------------- cfg_file: /boot/octopi.conf.d/camera1.txt camera: auto usb options: -r 640x480 -f 10 raspi options: -fps 10 http options: -w ./www-octopi -n -p 8081 Explicitly set USB device: ----------------------------------------------- Found video devices: /dev/video10 /dev/video11 /dev/video12 /dev/video13 /dev/video14 /dev/video15 /dev/video16 config file='/boot/octopi.conf.d/camera1.txt':USB device was not set in options, adding input stream to MJPG-streamer with found video device: /dev/video10 <13>Aug 10 21:55:53 pi: Starting USB webcam config file='/boot/octopi.conf.d/camera1.txt':USB device was not set in options, adding input stream to MJPG-streamer with found video device: /dev/video11 <13>Aug 10 21:55:53 pi: Starting USB webcam Cannot add /dev/video12 to input streams as it does not offer video capture capability Cannot add /dev/video13 to input streams as it does not offer video capture capability Cannot add /dev/video14 to input streams as it does not offer video capture capability Cannot add /dev/video15 to input streams as it does not offer video capture capability Cannot add /dev/video16 to input streams as it does not offer video capture capability Checking for VL805 (Raspberry Pi 4)... - It seems that you don't have VL805 (Raspberry Pi 4). There should be no problems with USB (a.k.a. select() timeout) Running ./mjpg_streamer -o output_http.so -w ./www-octopi -n -p 8081 -i 'input_uvc.so -r 640x480 -f 10 -d /dev/video10' -i 'input_uvc.so -r 640x480 -f 10 -d /dev/video11' MJPG Streamer Version: git rev: 5554f42c352ecfa7edaec6fc51e507afce605a34 ERROR: could not find input plugin Perhaps you want to adjust the search path with: # export LD_LIBRARY_PATH=/path/to/plugin/folder dlopen: 'input_uvc.so -r 640x480 -f 10 -d /dev/video10' -i 'input_uvc.so -r 640x480 -f 10 -d /dev/video11' : cannot open shared object file: No such file or directory ```

As you can see in the final line, it seems to pick up both /dev/video10 and /dev/video11 - those are always there on most Pis, so this issue impacts everyone. I currently don't recommend merging this PR as it is, as I have a few issues with the way it works. Basically, I don't entirely understand the logic so I think it will be hard to explain to the majority of OctoPi users as well.

SilverWingedHawk commented 3 years ago

Thanks a lot for the test feedback.

cp2004 commented 3 years ago

Adding multiple webcams to a single mjpg-streamer process does and doesn't make sense, here's what I thought of.

For:

Against:

Might also need clarification where camera_http_options comes in. It's confusing how adding multiple USB options works in multiple files, but if it should all be merged into a single process then we only have one camera_http_options.

cp2004 commented 3 years ago

v4l2-ctl --list-formats -d /dev/video10

ioctl: VIDIOC_ENUM_FMT
        Type: Video Capture Multiplanar

        [0]: 'YU12' (Planar YUV 4:2:0)
        [1]: 'YV12' (Planar YVU 4:2:0)
        [2]: 'NV12' (Y/CbCr 4:2:0)
        [3]: 'NV21' (Y/CrCb 4:2:0)
        [4]: 'RGBP' (16-bit RGB 5-6-5)

v4l2-ctl --list-formats -d /dev/video11 

ioctl: VIDIOC_ENUM_FMT
        Type: Video Capture Multiplanar

        [0]: 'H264' (H.264, compressed)
        [1]: 'MJPG' (Motion-JPEG, compressed)
SilverWingedHawk commented 3 years ago

Thanks for the command's output. As for the current logic of the script, it's expected behavior those devices get added to the list of input streams... I'm relying on this command to output something which contains more than 3 lines (v4l2-ctl --list-formats still outputs the ioctl lines whether the device has video capture or not). But I can't even filter out based on MJPG format, as first, this might falsely ignore a valid camera not offering this kind of stream, second, it wouldn't filter out /dev/video11... So I'd have to resort to hard blacklisting both of those devices, hoping they never change of /dev/video node... Are there udev rules that are in place to ensure this ?

Concerning the multi-core support, a cursory look at the mjpg-streamer source code shows the use of pthreads so it would stand to reason that we're not sacrificing multi-core support, but I'd like a second look to be sure.

As for the event of one camera stopping working, would it stop the entire streaming process, my guess would be no, as both streams are presented by differents URLs... but I'd rather have a real test confirming that.

As for the camera_http_options, yes, it stands to reason that it should only appear once.

cp2004 commented 3 years ago

I have no idea of the first part - it's out of my knowledge of how this stuff works.

Concerning the multi-core support, a cursory look at the mjpg-streamer source code shows the use of pthreads so it would stand to reason that we're not sacrificing multi-core support, but I'd like a second look to be sure.

That's definitely good to hear, 👍

As for the event of one camera stopping working, would it stop the entire streaming process, my guess would be no, as both streams are presented by differents URLs... but I'd rather have a real test confirming that.

I don't currently have the cameras wired up to test that... but I know the process ends when one camera stops streaming, if you are just streaming one camera.

SilverWingedHawk commented 3 years ago

Could you be so kind as to provide the output of udevadm info -a /dev/video10 and udevadm info -a /dev/video11 ? Be aware, the output of those commands can be rather lengthy...

Looking deeper in the source code of mjpg-streamer, especially in input_uvc.c file, line 494, the input_run function which is responsible for starting the related input stream makes use of pthread_create and pthread_detach so the multi-core support would still be there.

It also seems that each camera stream can be stopped at will, at least from the source code point of view... So perhaps the removal of one webcam wouldn't impact the streaming process... But as said earlier, a test in real conditions is the only proof I'd find acceptable

cp2004 commented 3 years ago

1st one

(oprint) pi@octopi:~/mjpg-streamer $ udevadm info -a /dev/video10

Udevadm info starts with the device specified by the devpath and then
walks up the chain of parent devices. It prints for every device
found, all possible attributes in the udev rules key format.
A rule to match, can be composed by the attributes of the device
and the attributes from one single parent device.

  looking at device '/devices/platform/soc/fe00b840.mailbox/bcm2835-codec/video4linux/video10':
    KERNEL=="video10"
    SUBSYSTEM=="video4linux"
    DRIVER==""
    ATTR{dev_debug}=="0"
    ATTR{index}=="0"
    ATTR{name}=="bcm2835-codec-decode"

  looking at parent device '/devices/platform/soc/fe00b840.mailbox/bcm2835-codec':
    KERNELS=="bcm2835-codec"
    SUBSYSTEMS=="platform"
    DRIVERS=="bcm2835-codec"
    ATTRS{driver_override}=="(null)"

  looking at parent device '/devices/platform/soc/fe00b840.mailbox':
    KERNELS=="fe00b840.mailbox"
    SUBSYSTEMS=="platform"
    DRIVERS=="bcm2835_vchiq"
    ATTRS{driver_override}=="(null)"

  looking at parent device '/devices/platform/soc':
    KERNELS=="soc"
    SUBSYSTEMS=="platform"
    DRIVERS==""
    ATTRS{driver_override}=="(null)"

  looking at parent device '/devices/platform':
    KERNELS=="platform"
    SUBSYSTEMS==""
    DRIVERS==""

2nd one:

(oprint) pi@octopi:~/mjpg-streamer $ udevadm info -a /dev/video11

Udevadm info starts with the device specified by the devpath and then
walks up the chain of parent devices. It prints for every device
found, all possible attributes in the udev rules key format.
A rule to match, can be composed by the attributes of the device
and the attributes from one single parent device.

  looking at device '/devices/platform/soc/fe00b840.mailbox/bcm2835-codec/video4linux/video11':
    KERNEL=="video11"
    SUBSYSTEM=="video4linux"
    DRIVER==""
    ATTR{name}=="bcm2835-codec-encode"
    ATTR{index}=="0"
    ATTR{dev_debug}=="0"

  looking at parent device '/devices/platform/soc/fe00b840.mailbox/bcm2835-codec':
    KERNELS=="bcm2835-codec"
    SUBSYSTEMS=="platform"
    DRIVERS=="bcm2835-codec"
    ATTRS{driver_override}=="(null)"

  looking at parent device '/devices/platform/soc/fe00b840.mailbox':
    KERNELS=="fe00b840.mailbox"
    SUBSYSTEMS=="platform"
    DRIVERS=="bcm2835_vchiq"
    ATTRS{driver_override}=="(null)"

  looking at parent device '/devices/platform/soc':
    KERNELS=="soc"
    SUBSYSTEMS=="platform"
    DRIVERS==""
    ATTRS{driver_override}=="(null)"

  looking at parent device '/devices/platform':
    KERNELS=="platform"
    SUBSYSTEMS==""
    DRIVERS==""
SilverWingedHawk commented 3 years ago

This is getting somewhere, I hadn't thought that the hardware encoder/decoder would be presented as a video device... Well... sorry to ask again, but what is the output of v4l2-ctl -D -d /dev/video10 and v4l2-ctl -D -d /dev/video11. I do hope they offer some capabilty that a camera doesn't or vice-versa...

cp2004 commented 3 years ago 
(oprint) pi@octopi:~/mjpg-streamer $ v4l2-ctl -D -d /dev/video10
Driver Info:
        Driver name      : bcm2835-codec
        Card type        : bcm2835-codec-decode
        Bus info         : platform:bcm2835-codec
        Driver version   : 5.10.17
        Capabilities     : 0x84204000
                Video Memory-to-Memory Multiplanar
                Streaming
                Extended Pix Format
                Device Capabilities
        Device Caps      : 0x04204000
                Video Memory-to-Memory Multiplanar
                Streaming
                Extended Pix Format
Media Driver Info:
        Driver name      : bcm2835-codec
        Model            : bcm2835-codec
        Serial           : 0000
        Bus info         : platform:bcm2835-codec
        Media version    : 5.10.17
        Hardware revision: 0x00000001 (1)
        Driver version   : 5.10.17
Interface Info:
        ID               : 0x0300000c
        Type             : V4L Video
Entity Info:
        ID               : 0x00000001 (1)
        Name             : bcm2835-codec-decode-source
        Function         : V4L2 I/O
        Pad 0x01000002   : 0: Source
          Link 0x02000008: to remote pad 0x1000004 of entity 'bcm2835-codec-decode-proc': Data, Enabled, Immutable
(oprint) pi@octopi:~/mjpg-streamer $ v4l2-ctl -D -d /dev/video11
Driver Info:
        Driver name      : bcm2835-codec
        Card type        : bcm2835-codec-encode
        Bus info         : platform:bcm2835-codec
        Driver version   : 5.10.17
        Capabilities     : 0x84204000
                Video Memory-to-Memory Multiplanar
                Streaming
                Extended Pix Format
                Device Capabilities
        Device Caps      : 0x04204000
                Video Memory-to-Memory Multiplanar
                Streaming
                Extended Pix Format
Media Driver Info:
        Driver name      : bcm2835-codec
        Model            : bcm2835-codec
        Serial           : 0000
        Bus info         : platform:bcm2835-codec
        Media version    : 5.10.17
        Hardware revision: 0x00000001 (1)
        Driver version   : 5.10.17
Interface Info:
        ID               : 0x0300001a
        Type             : V4L Video
Entity Info:
        ID               : 0x0000000f (15)
        Name             : bcm2835-codec-encode-source
        Function         : V4L2 I/O
        Pad 0x01000010   : 0: Source
          Link 0x02000016: to remote pad 0x1000012 of entity 'bcm2835-codec-encode-proc': Data, Enabled, Immutable
SilverWingedHawk commented 3 years ago

OK, this is something that I can work with to filter both of this devices... The Video Memory-to-Memory Multiplanar capability seems to be specific to both these devices, and is not something that appears on a regular webcam. I just have to find a way to query for that capability specifically, or craft a regexp as a last resort.

The nb_max_webcam setting conflicts with the idea of creating multiple files in /boot/octopi.conf.d for each webcam. I would like to define multiple webcams manually (by device ID) but this doesn't seem to be possible with this method - if I explicitly set usb options as -r 640x480 -f 10 -d /dev/video0, it is ignored.

Looking back at the script, that's a behavior I didn't change... The /dev/video0 node doesn't exist as evidenced by the list of found video devices. Rather early in the script, checks are in place to find the real device node behind a device path. As yours is inexistent, the script simply doesn't process it. This has nothing to do with code introduced by this PR... At least in these conditions.

I'll try to commit changes to assess what has been discussed here, but I might not be able to do it before next week.

Thanks again @cp2004 for the help provided, and for taking the time to test this.

cp2004 commented 3 years ago

Looking back at the script, that's a behavior I didn't change... The /dev/video0 node doesn't exist as evidenced by the list of found video devices. Rather early in the script, checks are in place to find the real device node behind a device path. As yours is inexistent, the script simply doesn't process it. This has nothing to do with code introduced by this PR... At least in these conditions.

OK, that makes sense - I should really have tested it with something that actually existed! Seems like you've got a good idea of how we can get this into shape & ready to merge. Let me know if you need any more info.

SilverWingedHawk commented 3 years ago

OK, that makes sense - I should really have tested it with something that actually existed! Seems like you've got a good idea of how we can get this into shape & ready to merge. Let me know if you need any more info.

As an improvement, a message stating that the device doesn't exist can be added to the script. I can do it in this PR, but it does seem that it would need its own PR as it is not related to the problem this PR is trying to solve...

SilverWingedHawk commented 3 years ago

@cp2004, sorry to bother you again, but could you please provide the output of the following commands:

They might be easier to parse than v4l2-ctl -D -d /dev/video(10|11)

SilverWingedHawk commented 3 years ago

Hi everyone,

I have changed the logic behind device detection. It now relies on a regex matching for the output of v4l2-ctl -D -d <device>, matching the capabilities hexadecimal representation which should end with a 1 if the device offer video capture, which the hardware encoders do not, as per this previous comment, while webcams should offer this capability.

I also took the liberty to introduce a message telling if a device isn't found, instead of silently skipping the non-existent device, so that configuration errors may be easier to find.

Please feel free to review these new changes and test them.

SilverWingedHawk commented 3 years ago

Hi,

Has anyone been able to review of test the change I made ?

guysoft commented 3 years ago

Hey, sorry for this hanging, didn't get to it, and no one else seems to be testing it. I will try find time but I am really swamped with other stuff. It would be really helpful if anyone can test this

guysoft commented 2 years ago

Hey, So I built flashed and tested it here on my printer.

For an initial test I started it with my pi camera that works out of the box when I flash mainstream. The camera does not show up and I get this: Screenshot_20211007_183536

output of ls /dev/video* is:

/dev/video0  /dev/video10  /dev/video11  /dev/video12  /dev/video13  /dev/video14  /dev/video15  /dev/video16

raspistill -o img.jpg works so the camera is indeed working on this build.

dmesg output:

[   13.209037] bcm2835-codec bcm2835-codec: Device registered as /dev/video10
[   13.209111] bcm2835-codec bcm2835-codec: Loaded V4L2 decode
[   13.209200] bcm2835-isp bcm2835-isp: Device node output[0] registered as /dev/video13
[   13.210094] bcm2835-isp bcm2835-isp: Device node capture[0] registered as /dev/video14
[   13.210918] bcm2835-isp bcm2835-isp: Device node capture[1] registered as /dev/video15
[   13.211533] bcm2835-isp bcm2835-isp: Device node stats[2] registered as /dev/video16
[   13.211570] bcm2835-isp bcm2835-isp: Register output node 0 with media controller
[   13.211603] bcm2835-isp bcm2835-isp: Register capture node 1 with media controller
[   13.211730] bcm2835-isp bcm2835-isp: Register capture node 2 with media controller
[   13.211831] bcm2835-isp bcm2835-isp: Register capture node 3 with media controller
[   13.212195] bcm2835-isp bcm2835-isp: Loaded V4L2 bcm2835-isp
[   13.219793] bcm2835-codec bcm2835-codec: Device registered as /dev/video11
[   13.219866] bcm2835-codec bcm2835-codec: Loaded V4L2 encode
[   13.226297] bcm2835-codec bcm2835-codec: Device registered as /dev/video12
[   13.226396] bcm2835-codec bcm2835-codec: Loaded V4L2 isp
[   13.241534] bcm2835-v4l2-0: scene mode selected 0, was 0
[   13.246327] bcm2835-v4l2-0: V4L2 device registered as video0 - stills mode > 1280x720
[   13.251686] bcm2835-v4l2-0: Broadcom 2835 MMAL video capture ver 0.0.2 loaded.
[   13.360764] brcmfmac: brcmf_fw_alloc_request: using brcm/brcmfmac43455-sdio for chip BCM4345/6
[   13.373812] brcmfmac: brcmf_c_preinit_dcmds: Firmware: BCM4345/6 wl0: Jan  4 2021 19:56:29 version 7.45.229 (617f1f5 CY) FWID 01-2dbd9d2e
SilverWingedHawk commented 2 years ago

Hi, thanks for the test.

In order to see what went wrong, I would also need the contents of the /var/log/webcamd.log file, as the contents you've provided, although helpful, do not point me to what went wrong with the mjpg_streamer invocation.

guysoft commented 2 years ago

Hey, sorry I didn't get back to you with a log, life got in the way. I think that month in particular I was a week away. Has there been any other progress? Since I expect we will have an RC soon.