[Dell XPS 9700] Emitter sometimes stops working after reboot

[ondrej-simon]

Output of linux-enable-ir-emitter boot status

○ linux-enable-ir-emitter.service - enable the infrared emitter
     Loaded: loaded (/lib/systemd/system/linux-enable-ir-emitter.service; enabled; vendor preset: enabled)
     Active: inactive (dead) since Mon 2022-11-07 07:21:19 CET; 4min 25s ago
    Process: 7517 ExecStartPre=/sbin/modprobe uvcvideo (code=exited, status=0/SUCCESS)
    Process: 7518 ExecStart=/usr/bin/linux-enable-ir-emitter run (code=exited, status=0/SUCCESS)
   Main PID: 7518 (code=exited, status=0/SUCCESS)
        CPU: 34ms

lis 07 07:21:19 ondrej-XPS-17-9700 systemd[1]: Starting enable the infrared emitter...
lis 07 07:21:19 ondrej-XPS-17-9700 systemd[1]: linux-enable-ir-emitter.service: Deactivated successfully.
lis 07 07:21:19 ondrej-XPS-17-9700 systemd[1]: Finished enable the infrared emitter.

*Output of `cat /etc/linux-enable-ir-emitter/`**

device=/dev/video2
unit=4
selector=9
size=4
control0=0
control1=0
control2=0
control3=0

Give more information if you have

I am using the script to enable IR Emittes on Dell XPS 9700. When I run the configure command, sometimes I need to press the no option 4-7 times until I find a configuration which displays enough illumination my face is recognized, this configuration is accepted using the yes option. I verify correct config by running sudo howdy test, to ensure my face is clearly visible.

Sometimes after reboot, it can happen after 1 reboot, or even multiple, something changes and (most likely) the driver I chose start behaving differently. Suddenly the image is too dark or flickering (although light conditions remain the same), and the laptop is no longer able to recognize me.

I noticed the same sometimes happens when I successfully run configure, verify correct driver selection and illumination using sudo howdy test, and then execute the boot disable and boot enable commands to refresh boot config.

Additional info

• Distro: Ubuntu
• Version: 22.04.1 LTS
• Version of linux-enable-ir-emitter: 4.1.5
• How did you install linux-enable-ir-emitter: PPA package (.deb)

[EmixamPP]

What is your dark_threshold in sudo howdy config ? In my case, I must set it to 80 in order to recognize me in any environment (in terms of lighting).

[ondrej-simon]

@EmixamPP , thank you for the quick reply. I do not think it is a problem with the darkness threshold. Look at the following two pictures extracted from sudo howdy test. They are taken 30 seconds apart from each other, with the same lighting conditions.

This first one is directly after boot:

and this second one is after I ran the following commands:

sudo linux-enable-ir-emitter boot disable
sudo linux-enable-ir-emitter boot enable

As you can see, in the first picture everything seems to work fine, but not in the second test.

[EmixamPP]

This is maybe a silly question, but just to be sure; this is normal to have dark images, followed by a clear image, because the emitter is blinking. And it could be that the second image has been taken when the transmitter is off ?

Nevertheless, I'm going to investigate by doing some tests. I'll keep you in touch.

In the meantime, you should increase the dark_threshold to 80 or more, that will fix the authentication issue.

[ondrej-simon]

@EmixamPP , I think this is something else. Basically, there are three scenarios I notice on my laptop, after I successfully configure the emitter using the configure command. When using sudo howdy test:

1. the emitter is not blinking, active camera warning light is displayed, and the picture is as bright and clear as the first picture I posted,
2. the emitter is not blinking, active camera warning light is displayed, and the picture is dark and unclear as in the second picture I posted,
3. the emitter is blinking (red lights next to camera), active camera warning light is displayed, and the image is blinking, brither, darker, but overall significantly darker than in 1st situation.

If you look at the two images I posted, you can see in the bottom left corner, they are also different resolutions. The illuminated one is 480x640, the darker one is 360x640, as if they were coming from different sources. Maybe this has something to do with it?

Either way, I greatly appreciate your effort and the time you take to answer! This tool is otherwise amazing, thanks a lot for it.

[EmixamPP]

If you look at the two images I posted, you can see in the bottom left corner, they are also different resolutions. The illuminated one is 480x640, the darker one is 360x640, as if they were coming from different sources. Maybe this has something to do with it?

Please can you give me the outputs of v4l2-ctl -d /dev/video0 --list-formats-ext and v4l2-ctl -d /dev/video2 --list-formats-ext?

Based on this, I think I could make a hypothesis about your problem.

[ondrej-simon]

@EmixamPP , of course, here you go:

sudo v4l2-ctl -d /dev/video0 --list-formats-ext 
ioctl: VIDIOC_ENUM_FMT
    Type: Video Capture

    [0]: 'MJPG' (Motion-JPEG, compressed)
        Size: Discrete 1280x720
            Interval: Discrete 0.033s (30.000 fps)
        Size: Discrete 960x540
            Interval: Discrete 0.033s (30.000 fps)
        Size: Discrete 848x480
            Interval: Discrete 0.033s (30.000 fps)
        Size: Discrete 640x480
            Interval: Discrete 0.033s (30.000 fps)
        Size: Discrete 640x360
            Interval: Discrete 0.033s (30.000 fps)
    [1]: 'YUYV' (YUYV 4:2:2)
        Size: Discrete 640x480
            Interval: Discrete 0.033s (30.000 fps)
        Size: Discrete 640x360
            Interval: Discrete 0.033s (30.000 fps)
        Size: Discrete 424x240
            Interval: Discrete 0.033s (30.000 fps)
        Size: Discrete 320x240
            Interval: Discrete 0.033s (30.000 fps)
        Size: Discrete 320x180
            Interval: Discrete 0.033s (30.000 fps)
        Size: Discrete 160x120
            Interval: Discrete 0.033s (30.000 fps)

---------------------------------------------------

v4l2-ctl -d /dev/video2 --list-formats-ext
ioctl: VIDIOC_ENUM_FMT
    Type: Video Capture

    [0]: 'GREY' (8-bit Greyscale)
        Size: Discrete 640x360
            Interval: Discrete 0.033s (30.000 fps)

[EmixamPP]

Can you confirm that you didn't modify the device used by howdy between the two images ?

As you pointed out, the resolution of the first image is 480x640, and you can see that your infrared camera (/dev/video2 here) only capture in 360x640, as with the second image.

In conclusion, the two images were not taken with the same camera part, the first one have been taken with the regular camera, and the second with the infrared one (thus you should increase the dark_threshold).

There are two possibilities I think :

1. Howdy is badly configured
2. Your infrared camera is sometimes on /dev/video2 and sometimes on /dev/video0. I have already read on the Arch wiki that this can happen. However, normally it is supposed to stay fixed after the boot. But from what you explain to me, the camera used, have change after the execution of the command linux-enable-ir-emitter boot disable/enable

Can you past your howdy configuration ? sudo howdy config

[EmixamPP]

The current version of linux-enable-ir-emitter does not take into account that the camera can change /dev/video link. I plan to implement this, but it will take me a long time.

Edit: but if that is your issue, we can try to do some sort of DIY to fix the problem.

[ondrej-simon]

@EmixamPP , here is the howdy config, I have not really changed it that much:

# Howdy config file
# Press CTRL + X to save in the nano editor

[core]
# Print that face detection is being attempted
detection_notice = false

# Do not print anything when a face verification succeeds
no_confirmation = false

# When a user without a known face model tries to use this script, don't
# show an error but fail silently
suppress_unknown = false

# Disable Howdy in remote shells
ignore_ssh = true

# Disable Howdy if lid is closed
ignore_closed_lid = true

# Disable howdy in the PAM
# The howdy command will still function
disabled = false

# Use CNN instead of HOG
# CNN model is much more accurate than the HOG based model, but takes much more
# computational power to run, and is meant to be executed on a GPU to attain reasonable speed.
use_cnn = false

[video]
# The certainty of the detected face belonging to the user of the account
# On a scale from 1 to 10, values above 5 are not recommended
# Lower is better
certainty = 4.0

# The number of seconds to search before timing out
timeout = 4

# The path of the device to capture frames from
# Should be set automatically by an installer if your distro has one
device_path = /dev/v4l/by-id/usb-CKFJH67Q313420022BB0_Integrated_Webcam_HD-video-index0

# Scale down the video feed to this maximum height
# Speeds up face recognition but can make it less precise
max_height = 320

# Set the camera input profile to this width and height
# The largest profile will be used if set to -1
# Automatically ignored if not a valid profile
frame_width = -1
frame_height = -1

# Because of flashing IR emitters, some frames can be completely unlit
# Skip the frame if the lowest 1/8 of the histogram is above this percentage
# of the total
# The lower this setting is, the more dark frames are ignored
dark_threshold = 50

# The recorder to use. Can be either opencv (default), ffmpeg or pyv4l2.
# Switching from the default opencv to ffmpeg can help with grayscale issues.
recording_plugin = opencv

# Video format used by ffmpeg. Options include vfwcap or v4l2.
# FFMPEG only.
device_format = v4l2

# Force the use of Motion JPEG when decoding frames, fixes issues with YUYV
# raw frame decoding.
# OPENCV only.
force_mjpeg = false

# Specify exposure value explicitly. This disables autoexposure.
# Use qv4l2 to determine an appropriate value.
# OPENCV only.
exposure = -1

[snapshots]
# Capture snapshots of failed login attempts and save them to disk with metadata
# Snapshots are saved to the "snapshots" folder
capture_failed = true

# Do the same as the option above but for successful attempts
capture_successful = true

[debug]
# Show a short but detailed diagnostic report in console
# Enabling this can cause some UI apps to fail, only enable it to debug
end_report = false

[ondrej-simon]

@EmixamPP , this may still be relevant, but I ended up working around it by adding more howdy models for the detection:

• looking up,
• looking down,
• looking right,
• looking left,
• tilting head right,
• tilting head left.

With these configurations (also including looking directly straight into the camera), the detection works much better and faster.

[EmixamPP]

Sorry for the delay. What happens is that the light image you see is your normal camera, and the dark image is your infrared camera (this is normal). But why you have a switch between the two, I don't know. The path that is indicated in your configuration (/dev/v4l/by-id/usb-CKFJH67Q313420022BB0_Integrated_Webcam_HD-video-index0) corresponds to a specific camera. Could you show me the return of ls /dev/v4l/by-id/ and ls /dev/v4l/by-path/.

So, indeed, having several profiles recorded allows to find a match better. But if before it was noted that the image was too dark, this implies that you have to increase the value of dark_threshould, since your emitter is not very powerful or the capture brightness is set to a low value. I'll experiment with 2-3 things when I have time regarding this.

[EmixamPP]

Feel free to reopen if needed