search

MI-DPLA / combine

Combine /kämˌbīn/ - Metadata Aggregator Platform
MIT License
26 stars 11 forks source link

Record validity is not updated correctly after JobValidation removed #262

Closed ghukill closed 5 years ago

ghukill commented 5 years ago

Appears there might be a race condition where RecordValidations are removed from Mongo in tasks.py, but Spark immediately queries and finds they still exist, which throws off the re-write and does not update validity of Records.

If no other validations run, Job will show that no Records are invalid, but that's not the case.

ghukill commented 5 years ago

Fixed - needed to update previously False to True validity