[Bug]: ALS.webcam makes webcam hang

vega-d commented 2 years ago

Steps for reproducing the issue

install a distro using gnome
run wluma as a user systemd service file
put computer to sleep

What is the buggy behavior?

upon waking up webcam starts behaving weirdly, very.

Before sleep the webcam light was blinking for a couple of milliseconds every second, now it's constantly on.
Wluma thinks it's really bright, and enables outdoors brightness mode, even if I cover the camera with my finger.
Accessing the webcam from any other app (including ffplay /dev/video0 shows nothing but a 720p black screen.
Stopping wluma does not fix the webcam, the light is still on and the video is still a malevich square.
if I reboot, nothing changes, the issue persists. Light doesn't even go off.
If I shutdown the laptop completely, and boot back up the light is off, BUT;
After a shutdown, as soon as I try using the webcam is gets back to showing nothing but a black screen with the light constantly on, even if I'm not using the webcam. ffplay returns NaN instead of a video stream.

What is the expected behavior?

Well perhaps wluma should not murder my webcam.

Logs

vega@dart ~/s/w/t/release [SIGINT]> RUST_LOG=debug ./wluma
[2022-02-01T06:03:12Z DEBUG wluma] Using Config {
        als: Webcam {
            video: 0,
            thresholds: {
                45: "normal",
                15: "dark",
                75: "outdoors",
                60: "bright",
                30: "dim",
                0: "night",
            },
        },
        output: [
            Backlight(
                BacklightOutput {
                    name: "eDP-1",
                    path: "/sys/class/backlight/intel_backlight",
                    capturer: None,
                    min_brightness: 1,
                },
            ),
        ],
    }
[2022-02-01T06:03:12Z INFO  wluma] Continue adjusting brightness and wluma will learn your preference over time.
MESA-INTEL: warning: Performance support disabled, consider sysctl dev.i915.perf_stream_paranoid=0

libv4l2: error setting pixformat: Input/output error
libv4l2: error setting pixformat: Input/output error

Version

version 4.1.0, from the built artifact in releases folder

Environment

default command doesn't work, so here's information from neofetch, I guess.

             .',;::::;,'.                vega@dart 
         .';:cccccccccccc:;,.            --------- 
      .;cccccccccccccccccccccc;.         OS: Fedora Linux 35.20220125.0 (Silverblue) x86_64 
    .:cccccccccccccccccccccccccc:.       Host: Notebook NS50MU 
  .;ccccccccccccc;.:dddl:.;ccccccc;.     Kernel: 5.15.16-200.fc35.x86_64 
 .:ccccccccccccc;OWMKOOXMWd;ccccccc:.    Uptime: 1 day, 7 hours, 3 mins 
.:ccccccccccccc;KMMc;cc;xMMc;ccccccc:.   Packages: 1432 (rpm), 101 (flatpak) 
,cccccccccccccc;MMM.;cc;;WW:;cccccccc,   Shell: fish 3.3.1 
:cccccccccccccc;MMM.;cccccccccccccccc:   Resolution: 1920x1080 
:ccccccc;oxOOOo;MMM0OOk.;cccccccccccc:   DE: GNOME 41.3 (Wayland) 
cccccc;0MMKxdd:;MMMkddc.;cccccccccccc;   Theme: Adwaita-dark [GTK2/3] 
ccccc;XM0';cccc;MMM.;cccccccccccccccc'   Icons: Adwaita [GTK2/3] 
ccccc;MMo;ccccc;MMW.;ccccccccccccccc;    Terminal: gnome-terminal 
ccccc;0MNc.ccc.xMMd;ccccccccccccccc;     CPU: 11th Gen Intel i7-1165G7 (8) @ 4.700GHz 
cccccc;dNMWXXXWM0:;cccccccccccccc:,      GPU: Intel TigerLake-LP GT2 [Iris Xe Graphics] 
cccccccc;.:odl:.;cccccccccccccc:,.       Memory: 6620MiB / 31889MiB 
:cccccccccccccccccccccccccccc:'.
.:cccccccccccccccccccccc:;,..                                    
  '::cccccccccccccc::;,.

cyrinux commented 2 years ago

Hi @vega-d ,

Can you please try to stop all applications trying to access your webcam, including wluma. Then do sudo modprobe -rv uvcvideo.

If this module unloading success, try sudo modprobe -v uvcvideo then ffplay /dev/video0.

Is it working or still black image?

If still black, try to unload and load uvcvideo module a second time, then list all supported format by your webcam with ffplay -f video4linux2 -list_formats all /dev/video0; and try to use the bigger format; eg: ffplay -f video4linux2 -video_size 1280x720 /dev/video0.

Can you also paste me the output of lsusb |grep -i webcam; ffplay -f video4linux2 -list_formats all /dev/video0 ?

vega-d commented 2 years ago

@cyrinux I did a reboot (full power off of a computer), then:

sudo modprobe -rv uvcvideo
sudo modprobe -v uvcvideo
ffplay /dev/video0

It worked once, opened a window and showed me my webcam feed. I gently closed ffplay... and webcam light did not go off. It's back to being a malevich square. Now, if I try to unload the kernel module it does this:

vega@fedora-toolbox-35 ~ [123]> sudo modprobe -rv uvcvideo
rmmod uvcvideo
modprobe: ERROR: could not remove 'uvcvideo': Operation not permitted

Right now it's in it's glitched state. The light is on, the feed is black.

lsusb returns this for a webcamera, it's the only camera on the computer:

Bus 003 Device 003: ID 04f2:b6fb Chicony Electronics Co., Ltd Chicony USB2.0 Camera

ffplay -f video4linux2 -list_formats all /dev/video0 returns this:

ffplay version 4.4.1 Copyright (c) 2003-2021 the FFmpeg developers
  built with gcc 11 (GCC)
  configuration: --prefix=/usr --bindir=/usr/bin --datadir=/usr/share/ffmpeg --docdir=/usr/share/doc/ffmpeg --incdir=/usr/include/ffmpeg --libdir=/usr/lib64 --mandir=/usr/share/man --arch=x86_64 --optflags='-O2 -flto=auto -ffat-lto-objects -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -fstack-protector-strong -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection' --extra-ldflags='-Wl,-z,relro -Wl,--as-needed -Wl,-z,now -specs=/usr/lib/rpm/redhat/redhat-hardened-ld -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 ' --extra-cflags=' -I/usr/include/rav1e' --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libvo-amrwbenc --enable-version3 --enable-bzlib --enable-chromaprint --disable-crystalhd --enable-fontconfig --enable-frei0r --enable-gcrypt --enable-gnutls --enable-ladspa --enable-libaom --enable-libdav1d --enable-libass --enable-libbluray --enable-libbs2b --enable-libcdio --enable-libdrm --enable-libjack --enable-libfreetype --enable-libfribidi --enable-libgsm --enable-libilbc --enable-libmp3lame --enable-libmysofa --enable-nvenc --enable-openal --enable-opencl --enable-opengl --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librsvg --enable-librav1e --enable-librtmp --enable-librubberband --enable-libsmbclient --enable-version3 --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libssh --enable-libsvtav1 --enable-libtesseract --enable-libtheora --enable-libtwolame --enable-libvorbis --enable-libv4l2 --enable-libvidstab --enable-libvmaf --enable-version3 --enable-vapoursynth --enable-libvpx --enable-vulkan --enable-libglslang --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxvid --enable-libxml2 --enable-libzimg --enable-libzmq --enable-libzvbi --enable-lv2 --enable-avfilter --enable-avresample --enable-libmodplug --enable-postproc --enable-pthreads --disable-static --enable-shared --enable-gpl --disable-debug --disable-stripping --shlibdir=/usr/lib64 --enable-lto --enable-libmfx --enable-runtime-cpudetect
  libavutil      56. 70.100 / 56. 70.100
  libavcodec     58.134.100 / 58.134.100
  libavformat    58. 76.100 / 58. 76.100
  libavdevice    58. 13.100 / 58. 13.100
  libavfilter     7.110.100 /  7.110.100
  libavresample   4.  0.  0 /  4.  0.  0
  libswscale      5.  9.100 /  5.  9.100
  libswresample   3.  9.100 /  3.  9.100
  libpostproc    55.  9.100 / 55.  9.100
[video4linux2,v4l2 @ 0x7fc5dc000c80] Compressed:       mjpeg :          Motion-JPEG : 640x480 640x360 352x288 320x240 176x144 160x120 1280x720
[video4linux2,v4l2 @ 0x7fc5dc000c80] Raw       :     yuyv422 :           YUYV 4:2:2 : 640x480 640x360 352x288 320x240 176x144 160x120 1280x720
/dev/video0: Immediate exit requested
    nan    :  0.000 fd=   0 aq=    0KB vq=    0KB sq=    0B f=0/0

vega-d commented 2 years ago

I'm gonna go boot from a usb stick and see if the issue is software or hardware

cyrinux commented 2 years ago

Do you use your webcam a lot before using it with luma ? Its weird, look like kernel module/driver issue maybe ? 🤔 When you try to unload after gently close of ffplay are we agree wluma was not trying to start too ?

I try to reproduce on my side with same steps but I can't reproduce the bug.

Can you try to start ffplay like this ffplay ffplay -f video4linux2 -framerate 30 -video_size 640x480 -input_format mjpeg /dev/video0
Can you run this and send me the output when the webcam light stay one after close ffplay ? sudo lsof /dev/video0 this will show us the program acceding /dev/video0

vega-d commented 2 years ago

Do you use your webcam a lot before using it with luma ?

Yeah, I've been attending some online lectures and it always worked just fine. Specifically when I ran wluma as a systemd service it all went to shit. I also just booted into live cd of ubuntu 21.10 and behavior is exactly the same, indicating a hardware failure.

doing sudo lsof /dev/video0 while it's glitched with light stuck on the output is this:

lsof: WARNING: can't stat() fuse.gvfsd-fuse file system /run/user/1000/gvfs
      Output information may be incomplete.
lsof: WARNING: can't stat() fuse.portal file system /run/user/1000/doc
      Output information may be incomplete.

maximbaz commented 2 years ago

Very sorry this has happened to you!

I'll honestly say that I have a very hard time imaging how a userspace app like wluma, even if executed via systemd, could damage a hardware device. We aren't gonna quickly dismiss this, and I've also been trying out webcam today (so far both of us were unable to reproduce), but we should also consider a possibility that this could also be an unfortunate coincidence, where the webcam would have died on that day anyway. Let's not jump to conclusions, I only say this to keep the mind open.

Do you also have a way to disable webcam in BIOS? Maybe it's worth to try that, disabling it there, booting up, and then rebooting again and enabling it back? Just wondering if this would somehow "reset" some internal failure...

vega-d commented 2 years ago

I'll honestly say that I have a very hard time imaging how a userspace app like wluma, even if executed via systemd, could damage a hardware device

My best idea is that because wluma polled it too fast and in too little windows? And that was harmful to it in some way? I gutted my laptop and tried plugging webcam in and out. Behavior changed in a slight way:

Boot the laptop
Use camera once, it enters that glitched state.
Unplug and plug the camera back in
Use camera again! It works, just fine! But enters a glitched state again.

I feel like I should get a sacrificial usb webcam and see if it dies too XD

maximbaz commented 2 years ago

My best idea is that because wluma polled it too fast and in too little windows? And that was harmful to it in some way?

At this moment I would not be surprised by anything 😅 For reference, what wluma does is takes a single camera frame in the lowest possible camera resolution once every 2 seconds... That doesn't seem like a lot to ask, compared to video chats that ask for lots of frames in highest possible resolution...? But who knows, you might be right 🤷‍♂️ Would love to get to the bottom of this!

vega-d commented 2 years ago

I booted windows portable edition from a usb stick, to totally reassure myself it's not some crazy linux issue, and I can confirm, even windows experiences this issue

vega-d commented 2 years ago

I have observed previously unnoticed behavior. If I "glitch" the webcam so the light is stuck on, then pull my laptop to C3 sleep - the webcam light goes off! It shows the same pattern with a full reboot, but instead of a complete reboot that's just a sleep.

I have a new theory - webcam is supposed to be powered down while not used, and something broke - now it doesn't turn off the power. Doing a reboot with a complete power down on USB rails (just reboot doesn't cut it) or doing sleep which also powers down USB power rails resets it to a normal state. Perhaps there could be done some piece of script trickery telling usb controller to stop supplying power to /dev/video0 when it's not in use? or perhaps it's missing some magic packet of powering down? The plot thickens!

maximbaz commented 2 years ago

This theory is easy to test, simply run watch -n 0.1 lsof /dev/video0 on a side and see if wluma shows up there. In my tests it's not there, meaning that we release webcam resources as soon as frame is captured, so there isn't much we can do, I would guess it's up to the driver to power-off a device that is not used by anyone.

vega-d commented 2 years ago

It dawned on me to check dmesg, and I found this being outputted when anything tries to access webcam when it's glitched:

Feb 03 00:45:17 dart kernel: uvcvideo 3-7:1.1: Failed to set UVC probe control : -110 (exp. 26).

maximbaz commented 2 years ago

This is quite google-able, with varying results - try perhaps some "quirks" as in this thread https://forums.linuxmint.com/viewtopic.php?p=1954908&sid=84b2937204ce2dcda89793bbc2acf2ab#p1954908

vega-d commented 2 years ago

try perhaps some "quirks"

no luck. I am substituting it with my phone right now, but in the long run I will be looking into replacing the camera. And also perhaps the screen, kinda want a glossy covering instead of matte one.

maximbaz commented 1 year ago

Hey @vega-d, have you ever had any further progress in this mysterious issue? I haven't heard any similar feedback from others, and to be honest, while I don't want to ignore an issue like this, to be honest I just don't know what to do about it, on wluma's side.

name-snrl commented 9 months ago

have you ever had any further progress

I seem to have encountered this problem as well, no time yet, maybe some ideas will come up over the weekend

maximbaz commented 4 weeks ago

Closing due to lack of activity, please leave a comment or open a new issue if still relevant.

maximbaz / wluma