submit v4l2loopback to the upstream kernel

[jiribohac]

It would be so much easier if this module were upstream. I searched lkml archives and found no trace of anyone trying to submit it. Is there a reason for that? Are there outstanding problems that need to be fixed first? Can I help?

[jiribohac]

I don't see why not - we have the network loopback driver, the block loop driver, alsa aloop and others. I think it's a question of whether it's useful and v4l2loopback is.

[umlaeute]

i have talked to the people of the v4l2-subsytem in the past, and their reaction was not all that positive.

iirc the biggest problem is, that v4l2loopback might provide a loophole for camera-vendors (or vendors of other video capturing devices) to not strive for v4l2-compatibility, as they instead might provide their own non-v4l2 driver (but still claim "v4l2 compatibility", as you could just pipe the output of their proprietary capture software interfacing their proprietary driver into a loopback device).

it wasn't all that negative either, e.g. @hverkuil submitted 5568514931b063eda0aa83976fd9a68cacc53e52 to make the code adhere to the kernel-standards (a first prerequisite to upstream the code).

so having said that: i'm totally open to upstreaming v42loopback (it certainly would get better support by people who actually know their business), but i'm probably not the one who is going to do the bureaucratic work :-) so go ahead (PRs for bringing the code up to kernel-standards are highly welcome)

[ctubbsii]

:+1: to getting this upstream. This was the suggestion when I tried to get it packaged into Fedora so that the kernel module would be signed. I think the argument about proprietary capture software is a bit silly... because one could make the same argument for other loopback devices, yet they're still included upstream.

[umlaeute]

regarding silly arguments

are you barking up the wrong tree here?

[ctubbsii]

are you barking up the wrong tree here?

Probably. But, I don't know if I have the time/energy to champion it upstream. I just wanted to express my opinion here to help bolster the argument for anybody who did have that time/energy.

My main points against the argument that it would enable proprietary capture software, if anybody wants to try to use them upstream, are:

1. The argument can't be valid, because if it were valid and applied consistently, then it would have prevented other loopback modules which have already been included upstream,
2. Anything that enables proprietary capture software would also enable open capture software (such as obs-studio and many others),
3. Open source software should enable innovation, not impose unnecessary restrictions,
4. Providing a bridge to proprietary software can actually help some proprietary products from understanding the value of open source, and can encourage them to build future solutions using open standards and software, and
5. The value to open source would outweigh any value to proprietary software, since there are already well-known open source products that require this driver.

Feel free to use these arguments if you think they are any good.

[hverkuil]

On 09/04/2020 23:01, Christopher Tubbs wrote:

are you barking up the wrong tree here?

Probably. But, I don't know if I have the time/energy to champion it upstream. I just wanted to express my opinion here to help bolster the argument for anybody who did have that time/energy.

My main points against the argument that it would enable proprietary capture software, if anybody wants to try to use them upstream, are:

1. The argument can't be valid, because if it were valid and applied consistently, then it would have prevented other loopback modules which have already been included upstream,
2. Anything that enables proprietary capture software would also enable open capture software (such as obs-studio and many others),
3. Open source software should enable innovation, not impose unnecessary restrictions,
4. Providing a bridge to proprietary software can actually help some proprietary products from understanding the value of open source, and can encourage them to build future solutions using open standards and software, and
5. The value to open source would outweigh any value to proprietary software, since there are already well-known open source products that require this driver.

Feel free to use these arguments if you think they are any good.

The person to convince is Mauro and it would be best to post this to the linux-media mailinglist (https://linuxtv.org/lists.php).

This out-of-tree driver is part of the debian distro (and thus also of all distros based on debian), so it's already available pretty much everywhere. The argument that it would enable proprietary capture software is IMHO bogus given the widespread availability of this loopback driver.

It's probably available in other distros (redhat/suse) as well, but I haven't checked that.

I personally would rather have it as a properly vetted and reviewed driver upstream then as an out-of-tree driver.

My personal opinion as co-maintainer of the media subsystem is that it is worth another try to convince Mauro that it is better to have it in the mainline kernel as a properly reviewed driver than as an out-of-tree (but still widely available) unreviewed driver.

Regards,

Hans

[mjg]

I use this module on Fedora 31 with UEFI/secure boot. For this I created a signing key and registered it with my UEFI (MOK). For every kernel update, I compile and sign the module. This is not for everyone ... In any case, it does work here, but basically every secure boot setup would benefit from upstreaming.

I do see several "competitors" such as the droidcam fork and the module from webcamoid. It's not v4l2loopback's duty to integrate those, of course, but it raises the question what the module here is lacking or what the others are adding (besides disregard for kernel module config policies in the case of the latter ...), i.e.: this might be a question when upstreaming.

[aramg]

fwiw.. if this becomes an official/signed module I would certainly consider ditching the (old) droidcam fork and appropriating the app to use the loopback driver directly.

[nettings]

FWIW, this driver is also packaged in openSUSE Tumbleweed. The package is called v4l2loopback-kmp-*, and there is v4l2loopback-utils as well.

Thanks for the great work, this driver is a very useful tool!

[stalkerg]

so having said that: i'm totally open to upstreaming v42loopback (it certainly would get better support by people who actually know their business), but i'm probably not the one who is going to do the bureaucratic work :-) so go ahead (PRs for bringing the code up to kernel-standards are highly welcome)

@umlaeute I suppose you shouldn't do it this bureaucratic work but at the same time will be greater if you just posting this driver in mail lists. We will see real feedbacks, and issues and some people can back to you with PRs. Publish patches as is much better than working in your repo only.

[umlaeute]

you know that:

• everybody can submit PRs right now. it's not "me working alone in my private repo", this is an open source project on a public git server
• everybody can "post this driver in mailing lists". you can do that right now.

[hverkuil]

On 14/10/2020 11:54, umläute wrote:

you know that:

• everybody can submit PRs right now. it's not "me working alone in my private repo", this is an open source project on a public git server
• everybody can "post this driver in mailing lists". /you/ can do that right now.

If someone wants to do this, then:

1) get the latest version from the repo 2) strip out any compat code for older kernels (not needed/wanted for upstream) 3) run 'checkpatch.pl --strict' (found in the scripts directory of the kernel sources) over the driver and fix any issues. When in doubt, you can ask me. 4) (probably the hardest part): convert the streaming I/O to use the videobuf2 framework. This is a requirement for upstreaming. 5) test the driver with v4l2-compliance from the v4l-utils repository and fix any issues. Use either the last released kernel or the master branch of the media_tree git repo (https://git.linuxtv.org/media_tree.git/). Again, ask me when you have questions.

Once done it can be posted for a first review to the linux-media mailinglist.

Item 4 will likely be the most work.

Regards,

Hans

[umlaeute]

thanks @hverkuil for the detailed instructions!

[wt]

@umlaeute I've asked for some help on the linux-media list. A dev there asked that I submit this for upstream inclusion. Would you be okay if I did that? I think it might be there best way going forward. I can get real feedback and land it in mainline.

[umlaeute]

i'm totally for upstream inclusion.

however, the plan was to get the split-devices working first.

as i don't have much time these days, any help there would be highly appreciated. (i'll promise to push my local changes today).

[wt]

For my context, what is split-devices?

[wt]

Okay, so here's my plan. I am going to bundle up the module for inclusion in the kernel and do what I can to sync the changes. I think that getting feedback from kernel devs is important here. I will do my best to sync changes by submitting PRs in this project for the time being. Do you have any major objections to that?

I would like to chat with you about the split-devices. I am not sure I understand what the benefit is. Using the same dev for capture and output seems to make sense to me. And unified device might even mesh really well with the mem2mem framework in the kernel, so I'd really like to understand why this is important.

[umlaeute]

ad split-devices: as the numerous bug-reports have shown, there are clients out there that refuse to accept video-devices that expose both the CAPTURE and OUTPUT capability. arguably this is a problem of those clients, but that does not help the users. we have the exclusive_caps kludge to work around this issue, but

• this is just ugly
• and brings it's own problems (like: not being able to access the device as a capture-device until there's a consumer attached to it)

while i do like the elegance of a single merged-device node (that's why i would like to keep it if possible), but right now it seems that split-device just solves issues that people are having without introducing new kludges.

do you think that split devices would actually be a problem for mem2mem?

[hverkuil]

On 18/03/2021 10:51, umläute wrote:

ad split-devices: as the numerous bug-reports have shown, there are clients out there that refuse to accept video-devices that expose /both/ the |CAPTURE| and |OUTPUT| capability. arguably this is a problem of those clients, but that does not help the users.

It's actually not supported by V4L2. So doing that is a bad idea and it explains why applications get confused.

we have the |exclusive_caps| kludge to work around this issue, but

• this is just ugly
• and brings it's own problems (like: not being able to access the device as a capture-device until there's a consumer attached to it)

while i do like the elegance of a single merged-device node (that's why i would like to keep it if possible), but right now it seems that split-device just solves issues that people are having without introducing new kludges.

do you think that split devices would actually be a problem for mem2mem?

Don't use the v4l2-mem2mem framework. It is specifically designed for memory-to-memory devices like codecs and scalers.

Just create two video devices, one capture, one output, and each with their own vb2 queue.

Regards,

Hans

[umlaeute]

thanks for clarifying this.

[jarkkojs]

I might be looking into deriving in-kernel driver patch set from this, as soon as I have bandwidth (within next few months).

[ozbenh]

Has this been completely abandoned in favor of doing it in pipewire ?

[umlaeute]

most likely.

i personally do not have the bandwidth for trying to push this into the mainline kernel (but if anybody else has: please go ahead). a pure user-space solution (e.g. pipewire) seems like a better approach anyhow.

[mjg]

I'm curious - how would you do in pipewire what this module provides?

[umlaeute]

you have to write specific code that can send video data to a pipewire graph (or receive from it). see https://docs.pipewire.org/ for some (unfortunately rather few) examples.

of course this requires the cooperation of the program(s) that want to capture - they need pipewire video support.

the beauty of v4l2loopback is that it works with any™ legacy program that only supports capturing from v4l2 devices, but hopefully programs will start switching to pipewire.

[zefr0x]

Pipewire Virtual Camera in OBS: https://github.com/obsproject/obs-studio/pull/5377

[Zipdox2]

Has this been completely abandoned in favor of doing it in pipewire ?

How much software actually supports PipeWire for webcams? I know it's used for screen sharing on Wayland, but I'm not so sure about webcam support.

[umlaeute]

i'm not sure what you mean by this. pipewire is an emerging ecosystem. it uses a completely different API for accessing video devices. so obviously it will take some time until "virtually all" software will support it.

just to clarify: I did not say that v4l2loopback itself is abandoned. it (tries to) solve a specific problem and does so in a way that allows programs written 20 years ago (in case they still compile) to use virtual cameras. it is great!

[stephematician]

How much software actually supports PipeWire for webcams? I know it's used for screen sharing on Wayland, but I'm not so sure about webcam support.

In theory its possible using xdg-desktop-portal. I'm not sure how far anyone has got with this. Then, as most applications won't use pipewire for video - you'd need a v4l2 translation for pipewire, e.g. pw-v4l2.

[umlaeute]

as most applications won't use pipewire for video

citation needed

[totaam]

citation needed

For a start, things move slowly. It takes manpower to modify an existing codebase.

xdg-desktop-portal also requires a full desktop environment, v4l2 does not. Adding the portal API to some solutions may be undesirable or even impossible because of the existing architecture. And even when it is possible, the extra API layers between the device and the portal way of doing things, migrating may be totally non-trivial.

---

Was an attempt ever made to merge this module upstream? If so, what was the response? Would it be possible to try again just to get some feedback and see how much work would be needed?

[stephematician]

citation needed

I regret, a little, the way I worded it. For a lot of desktop applications, there certainly is migration towards using pipewire. But even something as 'cutting edge' as OBS is feeling pain migrating to pipewire camera; so I guess I should say "For the time being, many desktop applications will (still) be using V4L2 for video over pipewire".

[umlaeute]

sure. that's why this module is still needed :-)

i'm merely saying that a user-space solution (like pipewire) should eventually replace it (and i do not really see why you shouldn't be able to run pipewire on a non-desktop system).

as for actually attempting to merge this into upstream:

i personally do not have the bandwidth for trying to push this into the mainline kernel (but if anybody else has: please go ahead).

there have been attempts in the past to prepare the codebase for merging (starting with applying the coding starting for the kernel), but no actual attempts (PRs,...)

[jarkkojs]

Pipewire replacing /dev/videoX. That is not really the case as a loopback device can work below Pipewire itself, which is inherently useful qualitative aspect. It's neither I'd see plausible for Pipewire project. This is useful.

I have process ongoing for RFC patch. I rewrite the whole device layer from scratch with the models that I know to work from the past: https://social.kernel.org/notice/AoM1sMba5dfkeR3vxw

It won't be binary compatible with this driver because the model this driver uses for managing devices is racy and it is mostly not right. But I use innards in detail (with appropriate credits for those).

Maybe few weeks and it is out...