search

nextcloud / recognize

👁 👂 Smart media tagging for Nextcloud: recognizes faces, objects, landscapes, music genres
https://apps.nextcloud.com/apps/recognize
GNU Affero General Public License v3.0
539 stars 45 forks source link

Face recognition queue not being processed #608

Closed BurningTheSky closed 1 year ago

BurningTheSky commented 1 year ago

Which version of recognize are you using?

3.3.4

Enabled Modes

Object recognition, Face recognition

TensorFlow mode

Normal mode

Which Nextcloud version do you have installed?

25.0.2

Which Operating system do you have installed?

Linux 5.19.17-Unraid x86_64

Which Docker container are you using to run Nextcloud? (if applicable)

nextcloud:latest

How much RAM does your server have?

32

What processor Architecture does your CPU have?

x86_64

Describe the Bug

Running a fresh Docker container with Recognize installed on Unraid 6.11.5. I'm not getting any errors in the GUI on the recognize page, I have green ticks for the following:

The object queue seems to have processed recenlty but I get the following warning: Face recognition: 19799 Queued files, Last classification: 16 hours ago

Cron is being triggered every 5 minutes from Unraid but I have also manually triggered a recrawl and the cron.php from inside the container but it doesn't seem to do anything

Expected Behavior

Queue to be processed with every cron run?

To Reproduce

I have a GPU in the server and tried adding that to the docker using the variables below to see if I could enable GPU processing but I get told "Successfully loaded libtensorflow in Node.js, but couldn't load GPU. Make sure CUDA Toolkit and cuDNN are installed and accessible, or turn off GPU mode. " so I have disabled that again.

NVIDIA_VISIBLE_DEVICES: NVIDIA_DRIVER_CAPABILITIES: all

Debug log

{"reqId":"Z3pVrMTtiOhO5r8kbdLJ","level":0,"time":"January 03, 2023 10:35:02","remoteAddr":"","user":"--","app":"cron","method":"","url":"/cron.php","message":"Run OCA\\Recognize\\BackgroundJobs\\StorageCrawlJob job with ID 413","userAgent":"--","version":"25.0.2.3","data":{"app":"cron"}}
{"reqId":"Z3pVrMTtiOhO5r8kbdLJ","level":0,"time":"January 03, 2023 10:35:02","remoteAddr":"","user":"--","app":"cron","method":"","url":"/cron.php","message":"Finished OCA\\Recognize\\BackgroundJobs\\StorageCrawlJob job with ID 413 in 0 seconds","userAgent":"--","version":"25.0.2.3","data":{"app":"cron"}}
{"reqId":"Z3pVrMTtiOhO5r8kbdLJ","level":0,"time":"January 03, 2023 10:35:02","remoteAddr":"","user":"--","app":"cron","method":"","url":"/cron.php","message":"CLI cron call has selected job with ID 2","userAgent":"--","version":"25.0.2.3","data":{"app":"cron"}}
{"reqId":"Z3pVrMTtiOhO5r8kbdLJ","level":0,"time":"January 03, 2023 10:35:02","remoteAddr":"","user":"--","app":"cron","method":"","url":"/cron.php","message":"CLI cron call has selected job with ID 414","userAgent":"--","version":"25.0.2.3","data":{"app":"cron"}}
{"reqId":"Z3pVrMTtiOhO5r8kbdLJ","level":0,"time":"January 03, 2023 10:35:03","remoteAddr":"","user":"--","app":"cron","method":"","url":"/cron.php","message":"Run OCA\\Recognize\\BackgroundJobs\\StorageCrawlJob job with ID 414","userAgent":"--","version":"25.0.2.3","data":{"app":"cron"}}
{"reqId":"Z3pVrMTtiOhO5r8kbdLJ","level":0,"time":"January 03, 2023 10:35:03","remoteAddr":"","user":"--","app":"cron","method":"","url":"/cron.php","message":"Finished OCA\\Recognize\\BackgroundJobs\\StorageCrawlJob job with ID 414 in 0 seconds","userAgent":"--","version":"25.0.2.3","data":{"app":"cron"}}

I did get this error once but it seems to not be repeating: {"reqId":"tg5TKGprdeODYq7d2rk5","level":1,"time":"January 03, 2023 10:25:03","remoteAddr":"","user":"--","app":"no app in context","method":"","url":"/cron.php","message":"Unable to generate a URL for the named route \"contacts.contacts.directcircle\" as such route does not exist.","userAgent":"--","version":"25.0.2.3","exception":{"Exception":"Symfony\\Component\\Routing\\Exception\\RouteNotFoundException","Message":"Unable to generate a URL for the named route \"contacts.contacts.directcircle\" as such route does not exist.","Code":0,"Trace":[{"file":"/var/www/html/lib/private/Route/Router.php","line":361,"function":"generate","class":"Symfony\\Component\\Routing\\Generator\\UrlGenerator","type":"->","args":["contacts.contacts.directcircle",["IeOAY8qWPrJnSvTp2zY8pk5Z6DSGmY2"],1]},{"file":"/var/www/html/lib/private/Route/CachingRouter.php","line":58,"function":"generate","class":"OC\\Route\\Router","type":"->","args":["contacts.contacts.directcircle",["IeOAY8qWPrJnSvTp2zY8pk5Z6DSGmY2"],false]},{"file":"/var/www/html/lib/private/URLGenerator.php","line":103,"function":"generate","class":"OC\\Route\\CachingRouter","type":"->","args":["contacts.contacts.directcircle",["IeOAY8qWPrJnSvTp2zY8pk5Z6DSGmY2"]]},{"file":"/var/www/html/apps/circles/lib/Model/ModelManager.php","line":561,"function":"linkToRoute","class":"OC\\URLGenerator","type":"->","args":["contacts.contacts.directcircle",["IeOAY8qWPrJnSvTp2zY8pk5Z6DSGmY2"]]},{"file":"/var/www/html/apps/circles/lib/Model/Circle.php","line":733,"function":"generateLinkToCircle","class":"OCA\\Circles\\Model\\ModelManager","type":"->","args":["IeOAY8qWPrJnSvTp2zY8pk5Z6DSGmY2"]},{"file":"/var/www/html/apps/circles/lib/Model/Circle.php","line":855,"function":"getUrl","class":"OCA\\Circles\\Model\\Circle","type":"->","args":[]},{"function":"jsonSerialize","class":"OCA\\Circles\\Model\\Circle","type":"->","args":[]},{"file":"/var/www/html/apps/circles/lib/Service/FederatedUserService.php","line":1293,"function":"json_encode","args":[{"__class__":"OCA\\Circles\\Model\\Circle"}]},{"file":"/var/www/html/apps/circles/lib/Service/FederatedUserService.php","line":1067,"function":"cacheSingleCircle","class":"OCA\\Circles\\Service\\FederatedUserService","type":"->","args":[{"__class__":"OCA\\Circles\\Model\\FederatedUser"},{"__class__":"OCA\\Circles\\Model\\Circle"}]},{"file":"/var/www/html/apps/circles/lib/Service/FederatedUserService.php","line":979,"function":"getSingleCircle","class":"OCA\\Circles\\Service\\FederatedUserService","type":"->","args":[{"__class__":"OCA\\Circles\\Model\\FederatedUser"},true]},{"file":"/var/www/html/apps/circles/lib/Service/FederatedUserService.php","line":533,"function":"fillSingleCircleId","class":"OCA\\Circles\\Service\\FederatedUserService","type":"->","args":[{"__class__":"OCA\\Circles\\Model\\FederatedUser"},true]},{"file":"/var/www/html/apps/circles/lib/ShareByCircleProvider.php","line":565,"function":"getLocalFederatedUser","class":"OCA\\Circles\\Service\\FederatedUserService","type":"->","args":["burningsky"]},{"file":"/var/www/html/lib/private/Share20/Manager.php","line":1406,"function":"getSharedWith","class":"OCA\\Circles\\ShareByCircleProvider","type":"->","args":["burningsky",7,null,-1,0]},{"file":"/var/www/html/apps/files_sharing/lib/MountProvider.php","line":97,"function":"getSharedWith","class":"OC\\Share20\\Manager","type":"->","args":["burningsky",7,null,-1]},{"file":"/var/www/html/lib/private/Files/Config/MountProviderCollection.php","line":134,"function":"getMountsForUser","class":"OCA\\Files_Sharing\\MountProvider","type":"->","args":[{"__class__":"OC\\User\\User"},{"__class__":"OC\\Files\\Storage\\StorageFactory"}]},{"file":"/var/www/html/lib/private/Files/SetupManager.php","line":226,"function":"addMountForUser","class":"OC\\Files\\Config\\MountProviderCollection","type":"->","args":[{"__class__":"OC\\User\\User"},{"__class__":"OC\\Files\\Mount\\Manager"},{"__class__":"Closure"}]},{"file":"/var/www/html/lib/private/Files/SetupManager.php","line":318,"function":"OC\\Files\\{closure}","class":"OC\\Files\\SetupManager","type":"->","args":["*** sensitive parameters replaced ***"]},{"file":"/var/www/html/lib/private/Files/SetupManager.php","line":227,"function":"setupForUserWith","class":"OC\\Files\\SetupManager","type":"->","args":[{"__class__":"OC\\User\\User"},{"__class__":"Closure"}]},{"file":"/var/www/html/lib/private/legacy/OC_Util.php","line":111,"function":"setupForUser","class":"OC\\Files\\SetupManager","type":"->","args":[{"__class__":"OC\\User\\User"}]},{"file":"/var/www/html/custom_apps/recognize/lib/BackgroundJobs/ClassifierJob.php","line":65,"function":"setupFS","class":"OC_Util","type":"::","args":["burningsky"]},{"file":"/var/www/html/custom_apps/recognize/lib/BackgroundJobs/ClassifyLandmarksJob.php","line":33,"function":"runClassifier","class":"OCA\\Recognize\\BackgroundJobs\\ClassifierJob","type":"->","args":["landmarks",[1,1]]},{"file":"/var/www/html/lib/public/BackgroundJob/Job.php","line":78,"function":"run","class":"OCA\\Recognize\\BackgroundJobs\\ClassifyLandmarksJob","type":"->","args":[[1,1]]},{"file":"/var/www/html/lib/public/BackgroundJob/TimedJob.php","line":103,"function":"start","class":"OCP\\BackgroundJob\\Job","type":"->","args":[{"__class__":"OC\\BackgroundJob\\JobList"}]},{"file":"/var/www/html/lib/public/BackgroundJob/TimedJob.php","line":93,"function":"start","class":"OCP\\BackgroundJob\\TimedJob","type":"->","args":[{"__class__":"OC\\BackgroundJob\\JobList"}]},{"file":"/var/www/html/cron.php","line":152,"function":"execute","class":"OCP\\BackgroundJob\\TimedJob","type":"->","args":[{"__class__":"OC\\BackgroundJob\\JobList"},{"__class__":"OC\\Log"}]}],"File":"/var/www/html/3rdparty/symfony/routing/Generator/UrlGenerator.php","Line":143,"message":"Unable to generate a URL for the named route \"contacts.contacts.directcircle\" as such route does not exist.","exception":{},"CustomMessage":"Unable to generate a URL for the named route \"contacts.contacts.directcircle\" as such route does not exist."}}

marcelklehr commented 1 year ago

Normal mode

On Unraid you'll be using the linuxserverio container, correct? I believe you'll need to enable WASM mode for recognize to work on Alpine linux (which is what linuxserverio is using, afaik)

Unable to generate a URL for the named route \"contacts.contacts.directcircle\" as such route does not exist.

This seems to be unrelated.

BurningTheSky commented 1 year ago

No, I'm using the official one I believe: https://registry.hub.docker.com/_/nextcloud. I will give WASM a go.

That's that I thought but I thought I'd add it just in case.

Is it possible to pass the GPU through to the container?

BurningTheSky commented 1 year ago

Also, I had managed to get around 500 photos tagged for people not using WASM somehow

marcelklehr commented 1 year ago

Also, I had managed to get around 500 photos tagged for people not using WASM somehow

That's strange

marcelklehr commented 1 year ago

Is it possible to pass the GPU through to the container?

Yes, there's even a docker image, ready to run recognize on GPU: https://github.com/bugsyb/recognize_docker

BurningTheSky commented 1 year ago

I'll take a look at that, never built an image on unraid before and portainer is complaining about needing BuildKit for that docker file.

BrainStone commented 1 year ago

I'm having a similar issue. I initially enabled all recognition settings. And in the begining everything worked fine. But after restarting my docker container it now is not processing anything.

Did not find anything in the logs

marcelklehr commented 1 year ago

But after restarting my docker container it now is not processing anything.

Can you check the oc_jobs table in the database?

BrainStone commented 1 year ago

Sure. Here are all jobs related to Recognize 

"id","class","argument","last_run","last_checked","reserved_at","execution_duration","argument_hash","time_sensitive"
"117908","OCA\Recognize\BackgroundJobs\ClassifyMovinetJob","{""storageId"":3,""rootId"":119}","1672842002","1672842002","1672842002","1458","730a97d1d54fe7363538dbe02ab6964a","0"
"141828","OCA\Recognize\BackgroundJobs\ClassifyFacesJob","{""storageId"":3,""rootId"":119}","1672842308","1672842308","1672842308","505","730a97d1d54fe7363538dbe02ab6964a","0"
"142245","OCA\Recognize\BackgroundJobs\StorageCrawlJob","{""storage_id"":3,""root_id"":119,""override_root"":121,""last_file_id"":415720,""models"":[""faces""]}","0","1672841704","0","0","9cfa4212eadeb7fba3f3cf1a9e1372b5","1"
"142246","OCA\Recognize\BackgroundJobs\StorageCrawlJob","{""storage_id"":3,""root_id"":119,""override_root"":121,""last_file_id"":226654,""models"":[""faces""]}","0","1672842308","0","0","25d98aeb780b2a5be8f4e18d8dcf77b2","1"
marcelklehr commented 1 year ago

@BrainStone So, nextcloud is currently under the impression that Movinet and face classifcation are still running, i believe after 12h it will automatically attempt to run them again

BrainStone commented 1 year ago

Yeah round about at the 12h mark they started running again. Seems like when I stopped the container the task didn't shut down properly.

Not a huge deal then, but certainly fixable I think

marcelklehr commented 1 year ago

certainly fixable I think

I don't know a way to fix this in recognize. If you kill the cron process things will go wrong. After 12h it'll recover, that's the fix.