Not scanning the group folder

kabaga commented 1 year ago

Hey, Thanks for reporting issues back to Nextcloud Face Recognition. Please, try to complete this report in detail so we can help you easier. :smile:

Make sure you read all the documentation, and the FAQ, and that the issue has not been reported before. :wink:

Expected behaviour

I have all my photos and videos located in /Memories which is a group folder that I created. The group folder can be viewed on the root folder on the users using it. However, the absolute path locate is in /data/__groupfolders/1/. The issue that I am having is when I ran the scan. Face Recognition does not scan the group folder that my wife and I are sharing.

Actual behaviour

It just scans the /data/<user-id>/files/

Steps to reproduce

Install the Face Recognition app
Set the php.ini memory_limit to 2048M

Ran the prep commands:

sudo -u www-data php occ face:setup --model 1
sudo -u www-data php occ face:setup --model 3
sudo -u www-data php occ face:setup --model 4
sudo -u www-data php occ face:setup --memory 1024M --model 4
sudo -u www-data php occ config:app:set facerecognition handle_shared_files --value true
sudo -u www-data php occ face:background_job -t 900

Server configuration

Operating system: Debian 12
Pdlib version: php8.2
How is DLib installed: Make sure it is working correctly with this tool Compiled by following the Hard Way guide
How is PDlib installed: Make sure it is working correctly with this tool Compiled by following the Hard Way guide
PHP version: 8.2
Web server: NGINX
Database: mariadb
Nextcloud version: 27.0.2

Client configuration

Browser:
Firefox
Operating system: Debian 12

Logs

Background task log with debug.

sudo -u apache php occ -vvv face:background_job

``` 1/8 - Executing task CheckRequirementsTask (Check all requirements) System: Linux System memory: 4065071104 PHP Memory Limit: 2147483648 Clustering backend: pdlib Image Backend: Imagick 2/8 - Executing task CheckCronTask (Check that service is started from either cron or from command) 3/8 - Executing task DisabledUserRemovalTask (Purge all the information of a user when disable the analysis.) yielding yielding yielding yielding 4/8 - Executing task StaleImagesRemovalTask (Crawl for stale images (either missing in filesystem or under .nomedia) and remove them from DB) Skipping stale images removal for user admin as there is no need for it Skipping stale images removal for user user3 as there is no need for it Skipping stale images removal for user user4 as there is no need for it Skipping stale images removal for user user5 as there is no need for it 5/8 - Executing task CreateClustersTask (Create new persons or update existing persons) Skipping cluster creation, not enough data (yet) collected. For cluster creation, you need either one of the following: * have 1000 faces already processed * or you need to have 95% of you images processed Use stats command to track progress yielding Skipping cluster creation, not enough data (yet) collected. For cluster creation, you need either one of the following: * have 1000 faces already processed * or you need to have 95% of you images processed Use stats command to track progress yielding Skipping cluster creation, not enough data (yet) collected. For cluster creation, you need either one of the following: * have 1000 faces already processed * or you need to have 95% of you images processed Use stats command to track progress yielding Face clustering will be created for the first time. 358 faces found for clustering 201 clusters found after clustering yielding 6/8 - Executing task AddMissingImagesTask (Crawl for missing images for each user and insert them in DB) Skipping image scan for user admin that has disabled the analysis Skipping image scan for user user3 that has disabled the analysis Skipping image scan for user user4 that has disabled the analysis Skipping full image scan for user user5 7/8 - Executing task EnumerateImagesMissingFacesTask (Find all images which don't have faces generated for them) yielding 8/8 - Executing task ImageProcessingTask (Process all images to extract faces) NOTE: Starting face recognition. If you experience random crashes after this point, please look FAQ at https://github.com/matiasdelellis/facerecognition/wiki/FAQ ```

Web server error log

``` Insert your webserver log here ```

Nextcloud log (data/nextcloud.log)

Nextcloud log

``` Insert your Nextcloud log here ```

Browser log

``` Insert your browser log here, this could for example include: a) The javascript console log b) The network log c) ... ```

matiasdelellis commented 1 year ago

sudo -u www-data php occ config:app:set facerecognition handle_group_files --value true

kabaga commented 1 year ago

That worked, and it says on the admin settings "The analysis is finished - 18477 images were analyzed". However, when I go to People in Memories, it is still blank.

kabaga commented 1 year ago

I'll take it back. The people is now showing up. I just need to logout and log back in.

vwbusguy commented 9 months ago

What I'm seeing happen is that when I set handle_group_files to true and then triggered the re-analysis, it picked up and analyzed the shared folders, but doesn't seem to pick up new pictures once that analysis is complete during background jobs. I see face recognition of new photos I take from that point, but not newer ones from my wife's account that are shared with mine.

Newer photos in shared folders just say this when viewed in Memories:

matiasdelellis commented 8 months ago

Hi @vwbusguy

I haven't tested the external files in a while, but it should work... Try the files:scan command

[matias@nube nextcloud]$ sudo -u apache php occ files:scan user_id

vwbusguy commented 8 months ago

It's not "external files" in the Nextcloud nomenclature sense, but another user on the same Nextcloud instance that has shared with another local user, so handle_shared_files would be relevant here (I have shared and group enabled) as opposed to handle_external_files, I would think?

I'll try that when I get home from work, but is there anyway to keep it analyzing new pictures in shared folders automatically after the initial scan is complete?

matiasdelellis commented 8 months ago

Sooooo sorrrrrrrrrrrrrrrrryyyyyyyyyyyyy.. 😞 😥 🤦🏻‍♂️

It seems that I never updated the documentation but the setting is handle_group_files 🤦🏻

Please try it..

ps.: Now I see that I already mentioned it before. But hey... the documentation is missing and someone should try it carefully!

vwbusguy commented 1 month ago

What I've found is that if full_image_scan_done is false, then it processes shared/group folders fine, but for some reason it stops evaluating new pictures in shared folders once this is eventually set back to true.

My workaround for the timebeing has been to add a cron task to flip that to false for my user every couple of hours so that the facerecognition service will then find the new shared/group images on the next service run:

0 */2 * * * sudo -u apache php /srv/nextcloud/occ user:setting scott facerecognition full_image_scan_done false

matiasdelellis / facerecognition