MI-DPLA / combine

Combine /kämˌbīn/ - Metadata Aggregator Platform
MIT License
26 stars 11 forks source link

stastic harvests identifiers: error on capture, results in error in Record #204

Closed ghukill closed 6 years ago

ghukill commented 6 years ago

When running static harvests, you have the ability to include an XPath expression to find a value to use as the record_id for a Record. However, it appears that if that process fails, it is written to the Record's error column with the error that occurred.

This is helpful in many ways, but the error is not necessarily with the document, which that column is usually reserved for, it's only the identifier. This results in skipping those Records for indexing to ElasticSearch.

Either:

ghukill commented 6 years ago

This brings up a further point of how to handle errors after the fact:

ghukill commented 6 years ago

More tolerance for static harvesting. AmbiguousIdentifier errors still result in success == True for Record, allowing indexing.

Other errors still result in success being False.