feature request: ondemand processing

[mithrandir42]

It would be useful to have the processing only being done when there is a consumer for the video. So we could have the Fake-Background process being run in background and only have system load when it is needed. Maybe it is also possible not to capture from the webcam while there is no consumer. This might increase durability of the cam. Also my camera shows recording by a light. This always is a good indication that video is being captured. So when running the fakecam this light permanently is being lit. Thus I cannot use this as indication anymore.

From what I can imagine, implementation could be like this: On Startup capture a few frames to get the settings and provide an output. Afterwards close the webcam device until there is a consumer for the output stream. When a consumer reads from the vloopback device open the webcam device again and start capturing/processing until the consumer stops capturing from the vloopback device.

This way the fake-webcam process could run in background without wasting resources while not in use. Does this sound possible?

[fangfufu]

What you suggested is probably quite difficult. The best I could do is to disconnect the physical camera, stop the image processing, and output a black screen. I don't know if that helps. Let me know if it sounds like a good idea.

[mithrandir42]

Sounds to be nearly what I have suggested. For the output of the black image we should get nearly no cpu/gpu load, so running the fake background as a service in background gets feasible. Maybe it is better to output the last picture before stopping the processing, instead of the black one, hard to tell. Also the camera would not get used (possible wear, don´t know if this applies) during idle times. Should be a large improvement for people wanting to run this in background. Disconnecting the physical camera should also allow other software accessing the camera directly while the fake stream is not in use. The question is if you can do this based on the presence of consumers (i.e. conferencing software accessing the video stream of the loopback device). Is it possible to detect if a consumer reads from the vloopback device to get the triggers for starting/stopping the processing?

[fangfufu]

Your camera is not going to get worn out by simply using it.

I don't know how to detect if a v4l2loopback device has been opened or not. In principle it should be possible, but this is beyond the effort I am willing to put in.

[mithrandir42]

Did a bit research, detecting usage of the video device should be possible using inotify: https://github.com/umlaeute/v4l2loopback/issues/109#issuecomment-617638198 Maybe, once there is a some (ipc) way to start/stop the processing, this approach could be used to control it.

[fangfufu]

I added the webcam blanking in https://github.com/fangfufu/Linux-Fake-Background-Webcam/commit/7bcd043c6f33b05c988a89696b4bd0ea44110e79.

I don't really fancy dealing with the inotify stuff in Python right now. If someone else's interested in working on this, feel free to send in a pull request.

[mithrandir42]

Great, thanks. With the manual control the fake-background gets usable for me. Maybe I can find some time to look into the inotify approach to get this automated. Python is not my language...

[mithrandir42]

I think I got the inotify aproach working. Will do some real testing tomorrow, having some conferences anyways. Afterwards, probably in the evening I'll create a pull request.

[fangfufu]

Why don't you upload your stuff to your Github repository right now? I could generate the pull request myself and etc. I really want to have a look. I am free right now.

[mithrandir42]

Already lying in bed right now. Just sent the comment with my mobile. It is already after 11 pm here and I need to get some sleep. Sorry...

[fangfufu]

Ah ok, I am looking forward to your code tomorrow! :)

[mithrandir42]

Used the lunch break to create the PR: #125 For me it does what it should. Maybe it needs some refactoring to match your coding style. As told before, python is not my language...

[fangfufu]

Thanks, implemented via https://github.com/fangfufu/Linux-Fake-Background-Webcam/pull/125

[MordesMortes]

is there a way to disable this feature as my implementation pops up shows a couple seconds of video then goes black saying that there's 1 consumer but then immediately says no consumers remaining, paused and it does this over and over in a loop that I can interrupt and get a couple of seconds of video again by hitting cntl-c

I'd much rather have it so I just manually turn it on and it works then manually turn it off when I don't need it I'm using akvcam if that makes a difference

[MordesMortes]

I got it to work by commenting out lines 330 - 333 of fake.py but I think this is probably less than optimal

[fangfufu]

@MordesMortes , does --no-ondemand option not work for you?

[MordesMortes]

I'll check that and see if that is better than my kludge as I wasn't seeing that as an option and now looking at the help it's right there...

ok now to see if I can get this to work with microsoft teams ;)

thanks for the help I feel like a moron for not seeing it in the help

[mithrandir42]

Maybe akvcam makes it behave different here. Can't tell. I have tried the ondemand processing with the v4lloopback. Anyways, MS teams (at least in my environment) cannot work with devices that appear after the programm has been started. It only sees the devices that have been present at startup. This means I have to turn on the bluetooth headset before starting teams in the morning. And it also means I have to start the background (and wait until it is ready) before starting teams.

[fangfufu]

This is why you want the on demand processing - the output device is created and outputting video before Team is opened.

On Wed, 14 Jul 2021, 10:15 mithrandir42, @.***> wrote:

Maybe akvcam makes it behave different here. Can't tell. I have tried the ondemand processing with the v4lloopback. Anyways, MS teams (at least in my environment) cannot work with devices that appear after the programm has been started. It only sees the devices that have been present at startup. This means I have to turn on the bluetooth headset before starting teams in the morning. And it also means I have to start the background before starting teams.

— You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub https://github.com/fangfufu/Linux-Fake-Background-Webcam/issues/119#issuecomment-879733191, or unsubscribe https://github.com/notifications/unsubscribe-auth/AARXHS336ZGCCB6L5TQ6HJDTXVIRNANCNFSM47H3PYWQ .

[mithrandir42]

This is why you want the on demand processing - the output device is created and outputting video before Team is opened. …

Right, that has been the reason why I wanted to have the ondemand processing before. Did not want to have the system load in the time between conferences. Anyways, with this comment I just wanted to help MordesMortes with his teams experiments. Took quite some time to find out the limitations. All applications work with the headset, why teams does not...

[MordesMortes]

Well I have it working with teams now the last hurdle was the on-demand processing just have to write a script to start the camera before teams starts and close it when teams ends.

Will be nice to have for the job interview as I won't have to worry about my background ;)

[fangfufu]

The default on-demand processing works really well for me... I can leave the app running all the time in background. What's the problem you are facing?

[MordesMortes]

well I'll copy what occurs in my terminal for you when I don't have the on-demand bypassed

/fake.py -H 480 -W 640 -v /dev/video7 Real camera original values are set as: 640x480 with 30 FPS and video codec 1448695129 Real camera new values are set as: 640x480 with 30 FPS and video codec 1196444237 Running... Please CTRL-C to pause and reload the background / foreground images Please CTRL-\ to exit INFO: Created TensorFlow Lite XNNPACK delegate for CPU. Consumers: 1 No consumers remaining, paused Consumers: 1 No consumers remaining, paused Consumers: 1 No consumers remaining, paused Consumers: 1 No consumers remaining, paused Consumers: 1 No consumers remaining, paused Consumers: 1 No consumers remaining, paused Consumers: 1 No consumers remaining, paused Consumers: 1 No consumers remaining, paused Consumers: 1 No consumers remaining, paused Consumers: 1 No consumers remaining, paused Consumers: 1 No consumers remaining, paused Consumers: 1 No consumers remaining, paused Consumers: 1 No consumers remaining, paused Consumers: 1 No consumers remaining, paused Consumers: 1 No consumers remaining, paused Consumers: 1 No consumers remaining, paused Consumers: 1 No consumers remaining, paused Consumers: 1 No consumers remaining, paused Consumers: 1 No consumers remaining, paused Consumers: 1 No consumers remaining, paused Consumers: 1 No consumers remaining, paused Consumers: 1 No consumers remaining, paused

[mithrandir42]

Strange. dev/video7 is your fake device, right? It looks like there is some process periodically accessing the device. Mabe this is due to avkcam being a little different to v4lloopback. Can you try with v4lloopback instead, to see if it makes a difference? Maybe we have to filter out some access to the device in the inotify part if there really is some periodic access.

[fangfufu]

@MordesMortes , you can manually unpause by pressing Ctrl+C.

[MordesMortes]

yes and then it immediately re-pauses and that is the issue but for how I'm using it I'm fine with the --no-ondemand flag