search

BiologicalRecordsCentre / ABLE

Assessing ButterfLies in Europe project repository
2 stars 3 forks source link

Issue: Validated Records of moths keep coming back to "Pending" menu #708

Closed Helder-Cardos closed 4 months ago

Helder-Cardos commented 4 months ago

Records, after being validated, return to the "Pending" menu.

DavidRoy commented 4 months ago

hi Helder, could you give some example occurrence IDs so we can investigate. Records will return to pending if they are edited.

Helder-Cardos commented 4 months ago

Hi David,

I just validated these right now, and they returned just after to "Pending", this issue was also reported to me by other verifiers, on our group.

ID: 36952528 ID: 36951324 ID: 36951321

DavidRoy commented 4 months ago

@andrewvanbreda could you investigate the logs on this. Anything affecting verification is urgent. Please liase with John if needed

johnvanbreda commented 4 months ago

I've checked the first 2 examples, both are correctly set to verified in the database and on Elasticsearch. However, the Logstash pipelines that update data in Elasticsearch were not running between yesterday afternoon and 10am BST this morning so they may have been outdated in Elasticsearch until then.

When a record is verified, 2 updates are performed to set the verification status - both the raw PostgreSQL copy and the Elasticsearch copy used for reporting are updated at that point in time. Shortly after, the Logstash pipeline will also copy over any changes - normally not necessary for a verification status change, though it may have fixed the data issue in these instances. My suggestion then would be to explore whether the verification tool is failing to set the identification.verification_status on Elasticsearch at the time of verification, so the records have the appearance of staying in the pending queue even though they are actually verified. I've just tested this on my local verification test setup and also on iRecord's live verification and could not reproduce the problem.

andrewvanbreda commented 4 months ago

Hi @johnvanbreda OK thanks, as I had just investigated, and was just about to call you in as I couldn't see anything wrong in logs or database. Seeing as it looks like one possible cause of the issue may have been rectified at 10am, perhaps a good starting point is if @Helder-Cardos could you see if you are still getting the issue? as the examples above are before that time.

Thanks

Andy

Helder-Cardos commented 4 months ago

@andrewvanbreda . I just rechecked it and the records are already accepted!! Several on that situation were inserted yesterday, and they are also "Accepted" now.

Thank you!