OtaK / hide-my-mess-rs

Webcam virtual backgrounds with Rust, Tensorflow & RobustVideoMatting
Apache License 2.0
11 stars 1 forks source link

Cannot use it in Firefox #6

Open raffaem opened 1 year ago

raffaem commented 1 year ago

I can see the webcam with mpv:

mpv av://v4l2:/dev/video4 --profile=low-latency --untimed --title="webcam-mirror" &

and in Chromium, but not in Firefox.

For example, the gUM Test Page returns: "AbortError: Starting videoinput failed".

The same with WebRTC samples.

I'm using the git version of v4l2loopback.

I create the camera with:

❯ cat /etc/modprobe.d/v4l2loopback.conf 
options v4l2loopback exclusive_caps=1
options v4l2loopback max_buffers=2
options v4l2loopback video_nr=4
options v4l2loopback card_label="fake-cam"
❯ cat /etc/modules-load.d/v4l2loopback.conf 
v4l2loopback

I obtain the same outcome with:

sudo v4l2loopback-ctl add 4 -n "fake-cam" -x 1 -b 2 -v

I've tried Firefox 108, and the AppImages of Firefox 109 and 110.

raffaem commented 1 year ago

ok it looks like it doesn't support the image format.

How can I change the image format?

[Parent 830353: VideoCapture]: D/webrtc_trace (device_info_v4l2.cc:300): CreateCapabilityMap called for device usb-0000:00:14.0-6
[Parent 830353: VideoCapture]: D/webrtc_trace (device_info_v4l2.cc:435): Camera capability, width:640 height:480 type:12 fps:30
[Parent 830353: VideoCapture]: D/webrtc_trace (device_info_v4l2.cc:435): Camera capability, width:1280 height:720 type:12 fps:30
[Parent 830353: VideoCapture]: D/webrtc_trace (device_info_v4l2.cc:435): Camera capability, width:640 height:480 type:1 fps:30
[Parent 830353: VideoCapture]: D/webrtc_trace (device_info_v4l2.cc:435): Camera capability, width:1280 height:720 type:1 fps:15
[Parent 830353: VideoCapture]: D/webrtc_trace (device_info_v4l2.cc:435): Camera capability, width:640 height:480 type:9 fps:30
[Parent 830353: VideoCapture]: D/webrtc_trace (device_info_v4l2.cc:435): Camera capability, width:1280 height:720 type:9 fps:15
[Parent 830353: VideoCapture]: D/webrtc_trace (device_info_v4l2.cc:435): Camera capability, width:640 height:480 type:11 fps:30
[Parent 830353: VideoCapture]: D/webrtc_trace (device_info_v4l2.cc:435): Camera capability, width:1280 height:720 type:11 fps:15
[Parent 830353: VideoCapture]: D/webrtc_trace (device_info_v4l2.cc:444): CreateCapabilityMap 8
[Parent 830353: VideoCapture]: D/webrtc_trace (device_info_v4l2.cc:356): CreateCapabilityMap 8
[Parent 830353: VideoCapture]: D/webrtc_trace (device_info_v4l2.cc:300): CreateCapabilityMap called for device platform:v4l2loopback-004
[Parent 830353: VideoCapture]: D/webrtc_trace (device_info_v4l2.cc:444): CreateCapabilityMap 0
[Parent 830353: VideoCapture]: D/webrtc_trace (device_info_v4l2.cc:356): CreateCapabilityMap 0
[Parent 830353: VideoCapture]: D/webrtc_trace (video_capture_v4l2.cc:155): Video Capture enumerats supported image formats:
[Parent 830353: VideoCapture]: D/webrtc_trace (video_capture_v4l2.cc:157):   { pixelformat = RGB3, description = 'cc75c6ac' }
[Parent 830353: VideoCapture]: D/webrtc_trace (video_capture_v4l2.cc:170): no supporting video formats found
[Parent 830353: VideoCapture]: D/webrtc_trace (video_capture_v4l2.cc:155): Video Capture enumerats supported image formats:
[Parent 830353: VideoCapture]: D/webrtc_trace (video_capture_v4l2.cc:157):   { pixelformat = RGB3, description = 'cc75c6ac' }
[Parent 830353: VideoCapture]: D/webrtc_trace (video_capture_v4l2.cc:170): no supporting video formats found
OtaK commented 1 year ago

You cannot; as this program heavily modifies the video stream, I opted to use RGB3 which is (usually) well-supported, easy to work with and doesn't need decompression.

Unsure if you can do this with webrtc as-is - but from your logs it looks like it's actually supported but not selected?

I'm very much out of the loop on this project as I haven't used it in a loooong while, but I'd say it's more of a WebRTC problem than this program's.

RobustVideoMatting has a web version and they seem to do webcam capture through tensorflow.js directly. (https://github.com/PeterL1n/RobustVideoMatting/blob/tfjs/index.html, available at https://peterl1n.github.io/RobustVideoMatting/#/demo)