search

pulibrary / dpul

Princeton's digital collections: Digital PUL
https://dpul.princeton.edu/
Apache License 2.0
15 stars 0 forks source link

Keep indexing records when one times out on validation #1529

Closed hackartisan closed 5 months ago

hackartisan commented 5 months ago

advances pulibrary/figgy#6329

hackartisan commented 5 months ago

Fixing it this way means that we'll declare the indexing job failed even while a bunch of the resources are enqueued, actually before they're all enqueued even, and someone might retry while this exhibitreindex job is still running. I think there's a way to do this where we instead move the valid? check down the stack and have the spawned job fail, which will record the timeout error in the usual way and still allow all the resources to be enqueued, and still skip indexing for properly invalid resources (i.e. resources that are private or whatnot).