ckan / ckanext-dcat

CKAN ♥ DCAT
163 stars 142 forks source link

Improve the error handling in the gather and import stage #95

Closed metaodi closed 6 years ago

metaodi commented 6 years ago
metaodi commented 6 years ago

I added the traceback as we most likely have an error in a custom profile, so this makes the it easier pinpoint the source of the error (in my case I was hunting a KeyError somewhere in my custom profile)

metaodi commented 6 years ago

I added two test cases to test the new error handling

  1. The user-contributed profiles are used (parse_dataset), hence we don't have control over the code. If an exception is raised, the harvester should fail gracefully incl. a nice error msg in the log
  2. The IDCATRDFHarvester provides extension points to the harvester. As we don't have control over the code behind, the same as above applies: if an exception is raised, the harvester should fail gracefully (possibly only for one harvest_object) and log an error msg.
amercader commented 6 years ago

Thanks @metaodi!