Fails when there are more than one face in the frame

[funnukes]

The program works great until there is one face but as soon as another face comes in frame, the program stops and gives this error. Is there a way to keep the program working even when there are multiple faces? It is fine if it can recognise only one and leave the other unlabelled.

I tried to contact you but it is hard to join fosstodon without an invite, I would really like if you can help me.

Error- Traceback (most recent call last): File "C:\Users\funnu\OneDrive\Desktop\Face and Object\wyzely_detect__main__.py", line 225, in main() File "C:\Users\funnu\OneDrive\Desktop\Face and Object\wyzely_detect__main.py", line 94, in main if face_details := utils.recognize_face( File "C:\Users\funnu\AppData\Local\Programs\Python\Python311\Lib\site-packages\wyzely_detect\utils\utils.py", line 141, in recognize_face path_to_image = Path(df.iloc[-1]["identity"]) File "C:\Users\funnu\AppData\Local\Programs\Python\Python311\Lib\site-packages\pandas\core\indexing.py", line 1153, in getitem__ return self._getitem_axis(maybe_callable, axis=axis) File "C:\Users\funnu\AppData\Local\Programs\Python\Python311\Lib\site-packages\pandas\core\indexing.py", line 1714, in _getitem_axis self._validate_integer(key, axis) File "C:\Users\funnu\AppData\Local\Programs\Python\Python311\Lib\site-packages\pandas\core\indexing.py", line 1647, in _validate_integer raise IndexError("single positional indexer is out-of-bounds") IndexError: single positional indexer is out-of-bounds

[slashtechno]

Interesting - I recall this occurring previously but I thought I had fixed it. I am unable to replicate the error on the latest version of the program. When testing, it identified the most-similar face and left the other one unlabeled. As far as I know, this is a limitation of deepface, the library this uses. Can you try using the latest version of the program?

Also:

I tried to contact you but it is hard to join fosstodon without an invite, I would really like if you can help me.

Fosstodon is federated with other Mastodon instances, meaning you can contact a Fosstodon user from another instance, and vice-versa. If you want to join Fosstodon specifically though, here's a one-time use invite: https://fosstodon.org/invite/r6aujeKd

[slashtechno]

File "C:\Users\funnu\AppData\Local\Programs\Python\Python311\Lib\site-packages\wyzely_detect\utils\utils.py", line 141, in recognize_face path_to_image = Path(df.iloc[-1]["identity"])

This looks to be an outdated install as line 141 was changed in commit b5d95ed and a try statement introduced to prevent this.

[funnukes]

Yea the error is fixed. But now when there is one known face, it labels it correctly but when another face comes in frame, it labels the new face with the label of the previous known face even when it does not match. I hope I am making sense, basically it forgets the older known face and removes the correct label and gives a false positive on the new face refusing to come back to the older face unless the new face is removed from the frame.

Any solutions? I know you mentioned that deepface only processes one face so I thought that it can work if there is only one face in the faces directory.

Also, can it be due to less accuracy of the recognition model? I currently have 10 images of my face and and just one folder. But I am using a webcam for the main program and the images were taken from my phone which is much higher quality. Would using the low res laptop webcam for photos be better?

[slashtechno]

It might be due to the default confidence threshold being too low. Try changing the command line flag --face-confidence-threshold - 0.6 may prevent false positives and minimize false negatives.

Regarding the folder structure, one image should work but more images will reduce false negatives, especially if the confidence threshold is set higher.

I'm not sure if the difference in sample picture quality makes a significant difference but you can always add the images from your laptop webcam to the folder with the images from your phone.

[funnukes]

changed the min_confidence to 0.6 from 0.3 and that has fixed the false positive. But the issue when there are more than one person, no labels appear. it is only labelled when only one face is in frame. Any way to make it work with multiple faces in frame? No way to solve the limitation of deepface only working when one face in frame?

[slashtechno]

But the issue when there are more than one person, no labels appear. it is only labelled when only one face is in frame. 

I was unable to replicate this in testing. Perhaps lower the confidence threshold? What is the output?

Also, did you end up adding images from your laptop webcam? Try that if you haven't already.

[funnukes]

Yea added 15 more images using the webcam. As you can see, if the phone is off or no other face then recognition works but as soon as another face is there, it goes away. Had to blur the face and name. I removed the images, I hope that is fine.

[funnukes]

the buffer value is 3 instead of 1. That won't cause this right?

[slashtechno]

the buffer value is 3 instead of 1. That won't cause this right?

It shouldn't, but maybe change it? I'll need to do some more testing since it's hard to debug when I can't reproduce the problem. Just to confirm, you're running the latest version of the program (not 0.1.1, but from source) and in your faces directory, you only have images of yourself, alone. Correct?

[slashtechno]

What does the program output in the console?

[funnukes]

Yes the faces folder only has my images, I have also uninstalled the and installed the last version which fixed the deepface iloc error before. The buffer value is also 1 now. Here is the output, it basically stops recognizing the face which it was recognizing as soon as the phone enters, it shows the face not recognized.

output- Python 3.11.5 (tags/v3.11.5:cce6ba9, Aug 24 2023, 14:38:34) [MSC v.1936 64 bit (AMD64)] on win32 Type "help", "copyright", "credits" or "license()" for more information.

= RESTART: C:\Users\funnu\AppData\Local\Programs\Python\Python311\Lib\site-packages\wyzely_detect__main__.py WARNING:tensorflow:From C:\Users\funnu\AppData\Local\Programs\Python\Python311\Lib\site-packages\keras\src\losses.py:2976: The name tf.losses.sparse_softmax_cross_entropy is deprecated. Please use tf.compat.v1.losses.sparse_softmax_cross_entropy instead.

No .env file found NVIDIA GeForce GTX 1650 Set CUDA device Video resolution: 640.0x480.0 Beginning video capture... representations_arcface.pkl does not exist Cosine similarity: 0.4231752158819977, filname: image4.jpeg, to_return: {'label': 'funnukes', 'x1': 230, 'y1': 226, 'x2': 417, 'y2': 413} First detection of funnukes ever First detection of person in this detection window Cosine similarity: 0.37019352537111416, filname: image4.jpeg, to_return: {'label': 'funnukes', 'x1': 178, 'y1': 184, 'x2': 376, 'y2': 382} Cosine similarity: 0.3521785029272865, filname: image4.jpeg, to_return: {'label': 'funnukes', 'x1': 192, 'y1': 184, 'x2': 389, 'y2': 381} Cosine similarity: 0.3460421836836043, filname: image4.jpeg, to_return: {'label': 'funnukes', 'x1': 195, 'y1': 182, 'x2': 397, 'y2': 384} Cosine similarity: 0.3648836876275703, filname: image4.jpeg, to_return: {'label': 'funnukes', 'x1': 189, 'y1': 180, 'x2': 396, 'y2': 387} Detected funnukes for 2 seconds ntfy_url is None. Not sending notification. Set ntfy_url to send notifications Detected person for 2 seconds ntfy_url is None. Not sending notification. Set ntfy_url to send notifications Cosine similarity: 0.37141645942934265, filname: image4.jpeg, to_return: {'label': 'funnukes', 'x1': 192, 'y1': 184, 'x2': 393, 'y2': 385} Cosine similarity: 0.3830823940444342, filname: image4.jpeg, to_return: {'label': 'funnukes', 'x1': 191, 'y1': 181, 'x2': 399, 'y2': 389} Cosine similarity: 0.3744115563422624, filname: image4.jpeg, to_return: {'label': 'funnukes', 'x1': 192, 'y1': 185, 'x2': 399, 'y2': 392} Cosine similarity: 0.37443268812069463, filname: image4.jpeg, to_return: {'label': 'funnukes', 'x1': 193, 'y1': 186, 'x2': 397, 'y2': 390} Cosine similarity: 0.38956564043828956, filname: image4.jpeg, to_return: {'label': 'funnukes', 'x1': 193, 'y1': 186, 'x2': 396, 'y2': 389} Cosine similarity: 0.3976306391336132, filname: image4.jpeg, to_return: {'label': 'funnukes', 'x1': 193, 'y1': 190, 'x2': 392, 'y2': 389} Cosine similarity: 0.40990400811145244, filname: image4.jpeg, to_return: {'label': 'funnukes', 'x1': 188, 'y1': 187, 'x2': 394, 'y2': 393} Cosine similarity: 0.41267670773291, filname: image4.jpeg, to_return: {'label': 'funnukes', 'x1': 186, 'y1': 183, 'x2': 398, 'y2': 395} Cosine similarity: 0.37998461785039284, filname: image4.jpeg, to_return: {'label': 'funnukes', 'x1': 190, 'y1': 181, 'x2': 396, 'y2': 387} Cosine similarity: 0.36534293107021953, filname: image4.jpeg, to_return: {'label': 'funnukes', 'x1': 189, 'y1': 180, 'x2': 397, 'y2': 388} Cosine similarity: 0.3880701599324292, filname: image4.jpeg, to_return: {'label': 'funnukes', 'x1': 190, 'y1': 182, 'x2': 392, 'y2': 384} Cosine similarity: 0.3359099206411722, filname: image4.jpeg, to_return: {'label': 'funnukes', 'x1': 176, 'y1': 182, 'x2': 372, 'y2': 378} Cosine similarity: 0.3827298438911173, filname: image4.jpeg, to_return: {'label': 'funnukes', 'x1': 184, 'y1': 188, 'x2': 368, 'y2': 372} Cosine similarity: 0.41272611997956765, filname: image4.jpeg, to_return: {'label': 'funnukes', 'x1': 182, 'y1': 195, 'x2': 359, 'y2': 372} Cosine similarity: 0.40859915195789087, filname: image3.jpeg, to_return: {'label': 'funnukes', 'x1': 193, 'y1': 195, 'x2': 379, 'y2': 381} Cosine similarity: 0.479878614309209, filname: image3.jpeg, to_return: {'label': 'funnukes', 'x1': 187, 'y1': 191, 'x2': 387, 'y2': 391} Cosine similarity: 0.41965852923242697, filname: image3.jpeg, to_return: {'label': 'funnukes', 'x1': 195, 'y1': 197, 'x2': 387, 'y2': 389} Cosine similarity: 0.4526909544742743, filname: image3.jpeg, to_return: {'label': 'funnukes', 'x1': 192, 'y1': 196, 'x2': 387, 'y2': 391} Cosine similarity: 0.4809249520452421, filname: image3.jpeg, to_return: {'label': 'funnukes', 'x1': 199, 'y1': 203, 'x2': 386, 'y2': 390} Cosine similarity: 0.4275441827423673, filname: image3.jpeg, to_return: {'label': 'funnukes', 'x1': 190, 'y1': 195, 'x2': 386, 'y2': 391} Cosine similarity: 0.38699897160677144, filname: image4.jpeg, to_return: {'label': 'funnukes', 'x1': 170, 'y1': 184, 'x2': 327, 'y2': 341} Cosine similarity: 0.3901264649459262, filname: image4.jpeg, to_return: {'label': 'funnukes', 'x1': 173, 'y1': 185, 'x2': 324, 'y2': 336} Cosine similarity: 0.4100696911041224, filname: image4.jpeg, to_return: {'label': 'funnukes', 'x1': 170, 'y1': 186, 'x2': 324, 'y2': 340} Cosine similarity: 0.4270905933997525, filname: image4.jpeg, to_return: {'label': 'funnukes', 'x1': 168, 'y1': 184, 'x2': 327, 'y2': 343} Cosine similarity: 0.4011384292249478, filname: image4.jpeg, to_return: {'label': 'funnukes', 'x1': 167, 'y1': 184, 'x2': 327, 'y2': 344} Cosine similarity: 0.4216763104562996, filname: image4.jpeg, to_return: {'label': 'funnukes', 'x1': 167, 'y1': 183, 'x2': 328, 'y2': 344} Cosine similarity: 0.39840114759507894, filname: image4.jpeg, to_return: {'label': 'funnukes', 'x1': 167, 'y1': 183, 'x2': 329, 'y2': 345} Cosine similarity: 0.4131230597829707, filname: image4.jpeg, to_return: {'label': 'funnukes', 'x1': 167, 'y1': 182, 'x2': 327, 'y2': 342} Cosine similarity: 0.3926352561622587, filname: image4.jpeg, to_return: {'label': 'funnukes', 'x1': 168, 'y1': 184, 'x2': 325, 'y2': 341} Cosine similarity: 0.4358690293159505, filname: image4.jpeg, to_return: {'label': 'funnukes', 'x1': 164, 'y1': 183, 'x2': 327, 'y2': 346} Cosine similarity: 0.4083853859226595, filname: image4.jpeg, to_return: {'label': 'funnukes', 'x1': 194, 'y1': 191, 'x2': 359, 'y2': 356} Cosine similarity: 0.46954080752629457, filname: image4.jpeg, to_return: {'label': 'funnukes', 'x1': 198, 'y1': 194, 'x2': 374, 'y2': 370} Cosine similarity: 0.4547253816632323, filname: image4.jpeg, to_return: {'label': 'funnukes', 'x1': 199, 'y1': 193, 'x2': 380, 'y2': 374} Cosine similarity: 0.4494667467114325, filname: image4.jpeg, to_return: {'label': 'funnukes', 'x1': 202, 'y1': 195, 'x2': 377, 'y2': 370} Cosine similarity: 0.4387763423003679, filname: image4.jpeg, to_return: {'label': 'funnukes', 'x1': 196, 'y1': 194, 'x2': 377, 'y2': 375} Cosine similarity: 0.4557596362997517, filname: image4.jpeg, to_return: {'label': 'funnukes', 'x1': 198, 'y1': 194, 'x2': 377, 'y2': 373} Cosine similarity: 0.42464543417622813, filname: image4.jpeg, to_return: {'label': 'funnukes', 'x1': 196, 'y1': 195, 'x2': 378, 'y2': 377} Cosine similarity: 0.43628880195071007, filname: image4.jpeg, to_return: {'label': 'funnukes', 'x1': 197, 'y1': 195, 'x2': 376, 'y2': 374} Cosine similarity: 0.46680656088242267, filname: image4.jpeg, to_return: {'label': 'funnukes', 'x1': 195, 'y1': 195, 'x2': 378, 'y2': 378} Cosine similarity: 0.43987501260743456, filname: image4.jpeg, to_return: {'label': 'funnukes', 'x1': 196, 'y1': 194, 'x2': 382, 'y2': 380} Detected funnukes for 2 seconds Face present but not recognized Face present but not recognized Face present but not recognized Face present but not recognized Face present but not recognized Face present but not recognized Face present but not recognized Face present but not recognized Face present but not recognized Face present but not recognized Face present but not recognized Face present but not recognized Face present but not recognized Face present but not recognized Face present but not recognized Face present but not recognized Face present but not recognized Face present but not recognized Face present but not recognized Face present but not recognized

[slashtechno]

Okay, I think I know what the problem is. On line 154 of utils.py, when a face isn't recognized, return None occurs. I think changing this to continue should fix it.

[funnukes]

Damn..man. It worked like a charm after changing it to continue

[slashtechno]

Perfect! Do you want to make a PR? If not, I'll try to later.

[funnukes]

PR as in? I am new to github

[slashtechno]

PR is an acronym for pull request. You first "fork" the repository, such as this; make changes, preferably on another branch; and create a pull request to "merge" your changes with the existing repository.

[funnukes]

Sure

[funnukes]

Sent the PR