Is it possible to launch a streaming process on the fly when the webcam is accessed?

[bluetiger9]

I'm using v4l2loopback with gstreamer as a workaround for a Google Chrome bug. I'am streaming the webcam's image to the virtual device:

gst-launch -v v4l2src ! v4l2sink device=/dev/video1

and than I'm using the virtual device from the browser.

Now the problem with that is webcam is running continuously, even when not actually is used. I just wondering if is it possible to launch the gstreamer process on-demand, when the virtual device is accessed?

[umlaeute]

i don't think this is possible: the v4l2loopback kernel module runs in kernel-space, and it is not possible to launch a user-space application from there.

[bluetiger9]

Actually is it possible to launch user-space applications from a kernel module, but maybe it's not the best idea.

But, I think it would be a nice feature to provide some mechanism that would allow user space applications to provide the input for the virtual device on-the-fly, when the device is actually opened. That would be very useful because would allow virtual devices to stay stand-by when they are not used, just like a the real video devices.

The Implementation could be something like:

• an user space application/service registers itself to v4l2loopback as a producer for one of the virtual devices (with an ioctl call for example)
• when a consumer tries to access the device, v4l2loopback notifies the producer (using some kernel to user-space communication technique) and blocks the consumer until the input is ready
• the producer starts streaming input into the virtual device (ex. starts a gstream process)
• v4l2loopback detects when the input is ready and releases the consumer
• when the consumer closes the device, v4l2loopback notifies the producer to stop streaming input

What do you think?

[Alexplose]

Any news on this ? Or a workaround ?

[umlaeute]

no. no.

there are no immediate plans to implement anything like this, but i would probably accept a PR that implements the signalling mechanism as outlined by @bluetiger9

[wmealing]

I can think of a workaround.

Using your favorite 'inotify' library in python script to watch for OPEN events on the device, and exec your gst-launch command.

When it detects a CLOSE event, terminate your command. While I can't really help you write this, I believe it should solve your problem.

[brianjmurrell]

Isn't this sort of thing what udev is for?

[umlaeute]

@brianjmurrell would udev be able to figure out if, when and how a device is being accessed? (the device is there from the beginning; the OP wants to react on some interaction with the device)

[wmealing]

Afaik, udev doesn't know when a device is being opened. Its device creation and management, not usage. Happy to be wrong though.

[brianjmurrell]

Yes, you are right. I was thinking more along the lines of Linux's _usermodehelper API. An example of it's use.

[iddo]

Here's a working solution:

#!/usr/bin/env bash
set -e

DEVICE_NUMBER="9"
DEVICE_FILE="/dev/video${DEVICE_NUMBER}"
RTSP_URL="rtsp://username:password@WIFIFCAM:554"

function finish {
    # Your cleanup code here
    if [ -n "${PID}" ]; then # streamer is already running
        echo "--- Stopping RTSP streamer"
        kill "${PID}" && sleep 2 || true
    fi
    echo "--- Removing loaded v4l2loopback dkms"
    sudo modprobe -r v4l2loopback
}
trap finish EXIT

if [ ! -e "${DEVICE_FILE}" ]; then

    echo "--- Loading v4l2loopback with device ${DEVICE_FILE}"
    sudo modprobe v4l2loopback exclusive_caps=1 video_nr=${DEVICE_NUMBER} card_label="Wifi Security Camera"

    echo "--- Locking output format"
    sudo v4l2-ctl --device "${DEVICE_FILE}" --set-ctrl keep_format=1
    ffmpeg -rtsp_transport tcp -i "${RTSP_URL}" -f v4l2 "${DEVICE_FILE}" &
    PID="${!}"
    echo "--- PID of ffmpeg is '${PID}'"
    sleep 5

    echo "--- Setting device timeout (blank video if connection is lost)"
    sudo v4l2-ctl --device "${DEVICE_FILE}" --set-ctrl timeout=1500
fi

# Trigger usage check in 1 second so that if the camera is not used the streamer will stop
(sleep 1; touch "${DEVICE_FILE}")&

echo "--- Waiting for video device usage"
sudo inotifywait -e OPEN,CLOSE -m "${DEVICE_FILE}" |
while read dir op file; do
    if [ -n "${PID}" ]; then # streamer is already running
        if [ "$(lsof -t "${DEVICE_FILE}" | grep -v "${PID}" | wc -l)" -eq "0" ]; then # no more clients
            echo "--- No more clients, stopping RTSP streamer"
            kill "${PID}"
            unset PID
        fi
    elif [ "$(lsof -t "${DEVICE_FILE}" | wc -l)" -gt "0" ]; then # new clients
        echo "--- Detected usage of camera, spinning up RTSP streamer"
        ffmpeg -rtsp_transport tcp -i "${RTSP_URL}" -vf format=pix_fmts=yuv420p -f v4l2 "${DEVICE_FILE}" &
        # TODO add disconnection detection and restart streamer
        PID="${!}"
        echo "--- PID of ffmpeg is '${PID}'"
    fi
done

[floe]

@iddo just found this, very helpful! How about turning your script into a gist?

[iddo]

@iddo just found this, very helpful! How about turning your script into a gist?

https://gist.github.com/iddo/a4c03c63c055064a0657d0d36a9f667d

[fredldotme]

I'm in the process of implementing something like this for a component to-be-used in Ubuntu Touch: https://github.com/fredldotme/opticd/commit/ae5fed76357b2020abdcc29b87f92e95d66e9d9d It is a user-session daemon for bridging the Android camera HAL to V4L2 on Android-based devices. The basic protocol between kernel & opticd is defined in AccessMediator.

Among other things this will also integrate into UT's permission dialogs, so reading is blocked until the app is allowed to do so. @bluetiger9 @umlaeute do have anything to base this on, opinions or other input?

[fredldotme]

I have implemented what we require in Ubuntu Touch and it should be enough for more simple use cases too: https://github.com/fredldotme/android_kernel_google_bonito/commit/cf0c08e3e59147c954fb3c83208ac5b609e8d434

The userspace half is implemented here: https://github.com/fredldotme/opticd/commit/7de3e4c8bb0a87a67f765cfd5a2c05a8b80c4cca

When the time is right, let me know if inclusion into the module is appropriate or you have some opinions on this approach @umlaeute.

[vicamo]

v4l2-relayd is the solution we have for Ubuntu oem projects for Intel IPU6 camera.1,2,3,4. All it does are:

1. open hardware camera at startup to retrieve format info,
2. configure v4l2loopback output
3. close hardware camera and wait for notifications from V4L2 Event API. V4l2loopback was patched with a private event type called client_usage.
4. open hardware camera and start streaming when non-zero capturing client(s) open the capturing side; stop streaming and close hardware camera when all client(s) closed v4l2loopback device.

You might want to have a try.