CatalogueOfLife / backend

Complete backend of COL ChecklistBank
Apache License 2.0
15 stars 11 forks source link

Stalled sector syncs cannot be deleted #793

Open gdower opened 4 years ago

gdower commented 4 years ago

At some point, something went wrong with the sector sync queue which resulted in a bunch of stalled waiting sector syncs that are not getting processed:

https://api.catalogue.life/dataset/3/sector/sync?limit=25&offset=0

I want to cancel them, so I manually sent a DELETE /dataset/3/sector/2945/sync request which returns a 204 No Content, but does not actually seem to do anything because the 2945 sector is still waiting as seen above.

gdower commented 4 years ago

DELETE /dataset/3/sector/2945/sync HTTP/1.1 Host: api.catalogue.life Authorization: Basic [redacted] User-Agent: insomnia/2020.3.3 Accept: /

< HTTP/1.1 204 No Content < Date: Thu, 30 Jul 2020 18:46:16 GMT < Server: Apache/2.4.6 (CentOS) OpenSSL/1.0.2k-fips < Cache-Control: must-revalidate,no-cache,no-store

gdower commented 4 years ago

Here's the [logs](https://logs.gbif.org/app/kibana#/context/AWyLa4mQHCKcR6PFXu4X/logs/AXOhC-EJHCKcR6PFH0x1?_a=(columns:!(level,datasetKey,service,logger_name,message),filters:!(),predecessorCount:5,sort:!('@timestamp',desc),successorCount:5)&_g=(refreshInterval:(display:Off,pause:!f,value:0),time:(from:now-1h,mode:quick,to:now))), although there is no error.

mdoering commented 4 years ago

Yes, that's annoying. But they dont do any harm. Still it would be good to get rid of them. A startup of the backend should clear them up, but apparently it doesn't.

If you don't mind I put this on the waiting list as its not causing problems, its just annoying. I can quickly kill them manually in the db if you want me to

gdower commented 4 years ago

I checked and those waiting sectors can still be synced, so it definitely can wait.