Update DB After Validate

[btylerburton]

User Story

In order to keep an accurate count of the state of our datasets, datagovteam wants to update the Harvesting database with the results of the Validate operation.

Acceptance Criteria

[ACs should be clearly demoable/verifiable whenever possible. Try specifying them using BDD.]

• [ ] GIVEN a record needs to validated to conform to DCAT-US spec AND that operation is successful THEN that dataset should be passed along to the sync operation

• [ ] GIVEN a record needs to validated AND that operation is unsuccessful THEN we should log that failure AND we should create a new error in the harvest_error table AND that dataset should be added to a list to be marked as "status: error" in the harvest_records db

Background

[Any helpful contextual notes or links to artifacts/evidence, if needed]

Security Considerations (required)

[Any security concerns that might be implicated in the change. "None" is OK, just be explicit here!]

Sketch

• [x] Loop over records to be deleted
• [x] Create an error if necessary at runtime
• [x] Log the result of the validate operation

[rshewitt]

we have a FK reference on a nullable field in harvest errors. the FK reference field ( harvest_record.id) is a primary key which can't be null.

[btylerburton]

Blocked pending conversation about db config