Fix deletion in JSON Harvester

EricSoroos commented 5 years ago

Deletion in the JSON harvester is broken in 2 ways

The json harvester crashes in the import stage when it can't access the harvest object id: None, because the id is queued before it is generated in obj.save()
All of the harvest objects for the deleted dataset are set to not current before the crash, so the relationship between the dataset and the harvester is lost, so the dataset will live on in CKAN when it has been deleted from the source.

This patch fixes the crash by saving the harvest object prior to using it's id, and only clearing the harvest objects for the deleted item after it has been deleted.

EricSoroos commented 5 years ago

Looks like tests for ckan=master are failing for unrelated reasons.

metaodi commented 5 years ago

@EricSoroos can you write a test for your change?

EricSoroos commented 5 years ago

Sorry, I can't get the test suite to run locally based on the README:

To run the tests, do:

    nosetests --nologcapture --ckan --with-pylons=test.ini ckanext

ckan / ckanext-dcat

Fix deletion in JSON Harvester #157