scientist-softserv / adventist_knapsack

Apache License 2.0
1 stars 0 forks source link

Bulkrax importers stuck pending in prod post-Knapsack #170

Closed KatharineV closed 2 months ago

KatharineV commented 2 months ago

I can get Bulkrax importers to run on staging, but everything I've launched on a production tenant today is stuck pending. It's been a few hours, and nothing is moving. This may be related to scientist-softserv/adventist_knapsack#172 and the importers that won't update, but I'm making a separate ticket for now, just in case.

See these examples on ADL and SDAPI:

https://sdapi.b2.adventistdigitallibrary.org/importers/439?locale=en https://sdapi.b2.adventistdigitallibrary.org/importers/438?locale=en

https://adl.b2.adventistdigitallibrary.org/importers/311?locale=en

ShanaLMoore commented 2 months ago

I believe things are working, just rather slowly. There are about 7 jobs that are taking FOREVER to finish. We have a thread count of 10 so that really means when we start a new import, only 3 threads becomes available giving the appearance of a slow or stuck ingest. We're still looking into this. 👀

image

Tested using a small import: https://april-test.b2.adventistdigitallibrary.org/importers/2?locale=en

image

image