matiasdelellis / facerecognition

Nextcloud app that implement a basic facial recognition system.
GNU Affero General Public License v3.0
510 stars 46 forks source link

Unknown faces in database not being shown for identification #562

Closed boyddensmore closed 2 years ago

boyddensmore commented 2 years ago

Hi! Thanks for all the awesome work you've put into this. Having an easier way to see the faces of my loved ones has been incredibly positive for me.

Expected behaviour

I should be prompted to name all unknown faces for my user.

Actual behaviour

I am prompted for many faces but there are about 30% of faces that are unknown but I'm not prompted for a name. I've made VERY sparing use of Ignore, probably about 15 times for those 30,000 images for roughly 2-3 faces each.

Steps to reproduce

  1. Install, select model 1, analyze all images (30,000)
  2. In Settings, assign names to unknown faces
  3. While browsing images, continually find images with faces marked as Unknown
  4. Query the database directly to find faces linked to a person with a NULL name, find about 5000 unknown faces

This is the query I'm running:

select distinct path
from oc_facerecog_images images
    left join oc_facerecog_faces faces on faces.image = images.id
    left join oc_facerecog_persons persons on persons.id = faces.person
    left join oc_filecache files on files.fileid = images.file
where persons.name is null
    and is_valid = 1
    and is_visible = 1

Server configuration

Logs

Background task log with debug.

sudo -u apache php occ -vvv face:background_job ``` 1/10 - Executing task CheckRequirementsTask (Check all requirements) 2/10 - Executing task CheckCronTask (Check that service is started from either cron or from command) 3/10 - Executing task LockTask (Acquire lock so that only one background task can run) 4/10 - Executing task DisabledUserRemovalTask (Purge all the information of a user when disable the analysis.) 5/10 - Executing task StaleImagesRemovalTask (Crawl for stale images (either missing in filesystem or under .nomedia) and remove them from DB) 6/10 - Executing task CreateClustersTask (Create new persons or update existing persons) Clusters already exist, estimated there is no need to recreate them 7/10 - Executing task AddMissingImagesTask (Crawl for missing images for each user and insert them in DB) 8/10 - Executing task EnumerateImagesMissingFacesTask (Find all images which don't have faces generated for them) 9/10 - Executing task ImageProcessingTask (Process all images to extract faces) NOTE: Starting face recognition. If you experience random crashes after this point, please look FAQ at https://github.com/matiasdelellis/facerecognition/wiki/FAQ 10/10 - Executing task UnlockTask (Release obtained lock) ```
boyddensmore commented 2 years ago

My minimum confidence for grouping is 0.9, here is a breakdown of the confidence for these images. Interesting that many faces have a confidence greater than 1.

select round(confidence, 2), count(*)
from oc_facerecog_images images
    left join oc_facerecog_faces faces on faces.image = images.id
    left join oc_facerecog_persons persons on persons.id = faces.person
    left join oc_filecache files on files.fileid = images.file
where persons.name is null
    and persons.user = 'boyd'
group by round(confidence, 2)
order by round(confidence, 2) asc
+----------------------+----------+
| round(confidence, 2) | count(*) |
+----------------------+----------+
|                 0.00 |       13 |
|                 0.01 |       20 |
|                 0.02 |       15 |
|                 0.03 |       15 |
|                 0.04 |       12 |
|                 0.05 |       22 |
|                 0.06 |       11 |
|                 0.07 |       20 |
|                 0.08 |       14 |
|                 0.09 |       14 |
|                 0.10 |       17 |
|                 0.11 |       20 |
|                 0.12 |       18 |
|                 0.13 |       14 |
|                 0.14 |       15 |
|                 0.15 |       13 |
|                 0.16 |       17 |
|                 0.17 |       15 |
|                 0.18 |       20 |
|                 0.19 |       15 |
|                 0.20 |       25 |
|                 0.21 |       19 |
|                 0.22 |       14 |
|                 0.23 |       18 |
|                 0.24 |       13 |
|                 0.25 |       13 |
|                 0.26 |       15 |
|                 0.27 |       20 |
|                 0.28 |       11 |
|                 0.29 |       24 |
|                 0.30 |       23 |
|                 0.31 |       14 |
|                 0.32 |       15 |
|                 0.33 |       16 |
|                 0.34 |       22 |
|                 0.35 |       23 |
|                 0.36 |       20 |
|                 0.37 |       20 |
|                 0.38 |       25 |
|                 0.39 |       13 |
|                 0.40 |       16 |
|                 0.41 |       21 |
|                 0.42 |       27 |
|                 0.43 |       24 |
|                 0.44 |       13 |
|                 0.45 |       13 |
|                 0.46 |       21 |
|                 0.47 |       27 |
|                 0.48 |       21 |
|                 0.49 |       13 |
|                 0.50 |       16 |
|                 0.51 |       18 |
|                 0.52 |       10 |
|                 0.53 |       22 |
|                 0.54 |       16 |
|                 0.55 |       16 |
|                 0.56 |       19 |
|                 0.57 |       28 |
|                 0.58 |       23 |
|                 0.59 |       22 |
|                 0.60 |       26 |
|                 0.61 |       23 |
|                 0.62 |       24 |
|                 0.63 |       19 |
|                 0.64 |       30 |
|                 0.65 |       15 |
|                 0.66 |       26 |
|                 0.67 |       28 |
|                 0.68 |       27 |
|                 0.69 |       35 |
|                 0.70 |       27 |
|                 0.71 |       28 |
|                 0.72 |       29 |
|                 0.73 |       25 |
|                 0.74 |       40 |
|                 0.75 |       26 |
|                 0.76 |       27 |
|                 0.77 |       22 |
|                 0.78 |       32 |
|                 0.79 |       33 |
|                 0.80 |       23 |
|                 0.81 |       38 |
|                 0.82 |       37 |
|                 0.83 |       51 |
|                 0.84 |       29 |
|                 0.85 |       53 |
|                 0.86 |       32 |
|                 0.87 |       41 |
|                 0.88 |       49 |
|                 0.89 |       45 |
|                 0.90 |       63 |
|                 0.91 |       33 |
|                 0.92 |       42 |
|                 0.93 |       34 |
|                 0.94 |       35 |
|                 0.95 |       50 |
|                 0.96 |       54 |
|                 0.97 |       76 |
|                 0.98 |       73 |
|                 0.99 |       88 |
|                 1.00 |      137 |
|                 1.01 |      138 |
|                 1.02 |      174 |
|                 1.03 |      220 |
|                 1.04 |      289 |
|                 1.05 |      321 |
|                 1.06 |      304 |
|                 1.07 |      258 |
|                 1.08 |      193 |
|                 1.09 |      138 |
|                 1.10 |      116 |
|                 1.11 |       76 |
|                 1.12 |       58 |
|                 1.13 |       62 |
|                 1.14 |       37 |
|                 1.15 |       20 |
|                 1.16 |       25 |
|                 1.17 |       11 |
|                 1.18 |       15 |
|                 1.19 |       12 |
|                 1.20 |        9 |
|                 1.21 |        1 |
|                 1.22 |        1 |
|                 1.23 |        2 |
|                 1.24 |        3 |
|                 1.25 |        1 |
|                 1.26 |        1 |
+----------------------+----------+

Also, only 54 images have errors.

MariaDB [nextcloud]> select count(*) from oc_facerecog_images where error is not null;
+----------+
| count(*) |
+----------+
|       54 |
+----------+
guystreeter commented 2 years ago

The facerecognition page only shows "clusters" of faces, where it has decided those faces are sufficiently similar that they must all be the same person. I don't know the minimum number of "similar" faces required to constitute a cluster. You can have more than one face of a given individual, but it the algorithm doesn't think they are similar enough to each other, they won't be grouped.

matiasdelellis commented 2 years ago

Just comment that... https://github.com/matiasdelellis/facerecognition/blob/master/lib/Controller/ClusterController.php#L185-L186

.,, and it will show all the faces even individual... 😉

boyddensmore commented 2 years ago

This is very helpful detail, thank you so much, both of you. I've commented out that block and am on the long, slow road to identifying several thousand more faces haha. Thanks again for all your hard work, I hope to be able to contribute soon 😊