boltgolt / howdy

🛡️ Windows Hello™ style facial authentication for Linux
MIT License
5.58k stars 293 forks source link

Thank you developers for this! Have comments, questions, and requests. #457

Open phd21 opened 3 years ago

phd21 commented 3 years ago

Dear developers and maintainers,

Thank you developers for this excellent application! I have comments, questions, and requests.

Q: Does this system require an IR (infrared) webcam to work or does it work more securely with an IR webcam? Your list of supported webcams does show non-IR webcams like the Logitech C270 which I also have on another older computer.

Q: On this GitHub front page for Howdy under the section: A note on security "This package is in no way as secure as a password and will never be. Although it's harder to fool than normal face recognition, a person who looks similar to you or well-printed photo of you could be enough to do it. "

This is a little confusing to me and a little unnerving. If you have an infrared webcam, how could someone use a photo to login as a photo would have no heat signature, doesn't this software check for that, perhaps some depth checking?

I do not know how this software's facial recognition works in detail, but as for someone looking similar allowing a login, doesn't this software have various levels of details that it checks. Most people would not mind if it took a second longer to prevent this from happening. I read and saw during installation Fast, Balanced, and Secure options and I chose Balanced as was recommended. Would selecting the Secure option prevent similar faces from logging in? Are their detailed descriptions for each of these options, if so where?

My Background: I have been working as a computer professional for many years. I use Linux Mint Cinnamon version 20 and Linux KDE Neon both based on Ubuntu 20.04. My hardware specs are below. My computer came with MS Windows 10 with their Hello option active which I thought was pretty cool and convenient. A couple of days ago I decided to see if there are Linux facial recognition options available and came across various positive articles on this "Howdy" software.

How to Set Up Face Unlock on Ubuntu and Other Linux Distros https://itsfoss.com/face-unlock-ubuntu/

My System: Kernel: 5.4.0-52-generic x86_64 bits: 64 compiler: gcc v: 9.3.0 Desktop: Cinnamon 4.6.7 Distro: Linux Mint 20 Ulyana base: Ubuntu 20.04 focal Machine: Type: Convertible System: Dell product: Inspiron 7573 v: N/A serial: Mobo: Dell model: 01VKP1 v: A00 serial: UEFI: Dell v: 1.16.0 date: 02/17/2020

Integrated_Webcam_HD: Integrate (usb-0000:00:14.0-5) Bus 001 Device 003: ID 0bda:5692 Realtek Semiconductor Corp. Integrated_Webcam_HD

I have not tried this yet in my KDE Neon, but I will soon.

A couple of days ago, I decided to try this software in my Linux Mint Cinnamon v20. I installed the PPA (thank you for this), but on my first attempt, it failed with many Python deprecated messages and even locked up my computer requiring a forced restart. Like most newer systems, Python3 is installed as default although I also installed Python 2.

I removed Howdy (purged). I tried again and this time it appeared to finish, but whenever I tried to run any howdy command, it was not recognized. So, I checked your code and it appears it never ran the post-installation script therefore never created the "howdy" executable? I tried various Debian downloads and other installation options, no go, so I purged it again. Then later 2 nights ago, I re-tried the PPA, and this time it finished and I could run howdy commands like adding various facial pictures. I am not sure what happened before, perhaps it had trouble accessing my Internet, or something else?

I am active in the Linux Mint forums as member Phd21, and I would like to recommend this to people.

I have read some of the other posts and recommendations in GitHub for Howdy.

Requests and Recommendations: I too would like the option to run the verification again if it timed out rather than entering a password.

I use KeePassXC extensively which of course has a different password than my system and it sure would be great to open this with Howdy, Is there a way to do that or a plugin I need? My Android uses my fingerprint to unlock KeepPassXC which is convenient.

I also use KDE Neon with Kwallet as my primary OS, I also GPG encryption, so somehow integrating encryption and decryption capabilities into Howdy would be excellent even if it has to use its own secure database for various passwords.

Thank you again for this software and anything regarding this post. Phil phd21

boltgolt commented 3 years ago

This might be the longest issue i've read so far, i'll try to answer most of it

Does this system require an IR (infrared) webcam to work or does it work more securely with an IR webcam?

No, but Howdy is build with features specifically for IR cameras. For instance, it can filter out the blinking of the IR emitters

If you have an infrared webcam, how could someone use a photo to login as a photo would have no heat signature, doesn't this software check for that, perhaps some depth checking?

Right now Howdy does not make a 3D version of the image and there currently is no way to do so. Howdy does no have direct control over the emitters and does not know if the left or right emitter is firing. The face recognition is done by comparing the location of facial features (nose, eyes, etc) to a known model.

Would selecting the Secure option prevent similar faces from logging in? Are their detailed descriptions for each of these options, if so where?

These options set the certainty config option, which means the algorithm will accept less fuzzy matches when set to "secure". A photo of you will probably still pass if using a normal webcam because it IS your face.

I re-tried the PPA, and this time it finished and I could run howdy commands like adding various facial pictures. I am not sure what happened before, perhaps it had trouble accessing my Internet, or something else?

Those are some weird APT issues, i have not heard those problems before. Especially hanging the whole computer on installation is very weird.

I too would like the option to run the verification again if it timed out rather than entering a password.

Very hard to do, Howdy works within PAM and when PAM starts password auth Howdy can't be activated again.

I use KeePassXC extensively which of course has a different password than my system and it sure would be great to open this with Howdy, Is there a way to do that or a plugin I need?

If KeePassXC integrates with the central PAM system Howdy will be enabled automatically.

phd21 commented 3 years ago

Hi boltgolt,

Thank you for this application and for responding quickly.

Sorry for the long post. A long time ago, I was a software developer that provided support to people from all over, so I try to be thorough.

I did a lot of research into facial recognition and anti-spoofing last night and found some really great information that I will link in this post and replies.

I also installed Howdy into my KDE Neon which uses Ubuntu 20.04 Focal like Linux Mint 20 and I ran into the same installation and problems. The installation problem could be something to do with my hardware, ran very hot during install, but after removing the software (sudo apt purge howdy) and just reinstalling it (sudo apt install howdy), it worked except for the system login which was a major pain and still not working properly. Not sure if I should make a new post to this or add it to this one. I read a few other posts from KDE users and other users regarding this, #208, #233, #302, #28, etc... After adding images, and restarting or just logging out, I could not login while howdy was active, I could enter in the password, then the camera lights, but it would lock up.

After reading posts here, I ran these: sudo chmod -R 755 /lib/security/howdy/ Then: sudo howdy config And set no_confirmation to true

Afterward, during the initial system login I could enter a password and it would login, and then anything requiring a system password from then on works fine with Howdy. But, I still have to login with my password. It is as if Howdy is not given the first priority for login over the default system login processes or something does not know to check Howdy first. Still researching this to see if it has something to do with SDDM or PAM, ...

Thank you again, Phil phd21

phd21 commented 3 years ago

Hi boltgolt,

As for the accuracy of facial recognition and preventing spoofing, I found these articles. I was thinking that with the built-in Infrared camera, using some form of an algorithm for detecting a heat signature in the video stream or from a quick snapshot would eliminate anyone from using a photograph or image to bypass facial recognition. I don't know if that requires a special thermal IR camera (FLIR) or if typical IR webcams would work?

Windows Hello Facial Recognition Bypassed with a Photo - ExtremeTech https://www.extremetech.com/computing/261014-windows-hello-facial-recognition-bypassed-photo

*** Superb article on facial recognition and anti-spoofing and "liveness" detection with opencv code examples. Liveness Detection with OpenCV - PyImageSearch https://www.pyimagesearch.com/2019/03/11/liveness-detection-with-opencv/

flirimageextractor · PyPI https://pypi.org/project/flirimageextractor/

Convert IR image to temperature Python - Google Search https://www.google.com/search?client=ms-android-hmd-rev2&sxsrf=ALeKk03xYA2pdAMP4S9WgiWDWIdVSUl0xA:1603682398639&q=Convert+IR+image+to+temperature+Python&sa=X&ved=2ahUKEwjz9s-9ptHsAhX7hHIEHTCWBWkQ1QIwEXoECAwQAQ&cshid=1603682519561&biw=360&bih=560&dpr=3

Using Linux with USB 3.1 | FLIR Systems https://www.flir.com/support-center/iis/machine-vision/application-note/using-linux-with-usb-3.1/

Detecting people with a RaspberryPi, a thermal camera and machine learning | by Fabio Manganiello | Towards Data Science https://towardsdatascience.com/detecting-people-with-a-raspberrypi-a-thermal-camera-and-machine-learning-376d3bbcd45c

...