search

ucldc / rikolti

calisphere harvester 2.0
BSD 3-Clause "New" or "Revised" License
7 stars 3 forks source link

Don't raise 409 errors from OpenSearch #1119

Open amywieliczka opened 1 month ago

amywieliczka commented 1 month ago

Resolves #1095

Typically we see a conflict because OpenSearch is still trying to update documents at the time that we issue a query for all outdated documents, so we think there are some outdated documents when in reality, some documents just haven't finished updating yet.

I think it's safe to ignore a 409, but I'd like feedback on doing so. In both places that delete_by_query is called, the response is printed directly following, so it will be in the logs even if we do get a 409.

amywieliczka commented 1 month ago

Converting this back to a draft - it is very odd that we're seeing this 409 behavior because we have refresh=True on the bulk update request.