search

ckan / ckanext-dcat

CKAN ♥ DCAT
164 stars 142 forks source link

Harvester crashes with missing title #254

Open jze opened 11 months ago

jze commented 11 months ago

I had a harvester run abort today with the following error message:

[ckanext.harvest.model] Error when processsing dataset: KeyError('title') / Traceback (most recent call last):
  File "/usr/lib/ckan/default/src/ckanext-dcat/ckanext/dcat/harvesters/rdf.py", line 218, in gather_stage
    dataset['name'] = self._gen_new_name(dataset['title'])
KeyError: 'title'

I have not yet figured out which record contains the error and what it looks like. However, the harvest run should not crash completely due to a single incorrect dataset.

seitenbau-govdata commented 11 months ago

Thanks @jze . Yes, of course. I agree. Having a try/except around each dataset or checking for an existent title and creating a gather_error would be the better way in this case. Would you mind submitting a PR with the changes needed?