Consider extraction errors as expected operational errors without crashing the engine - Githubissues

OpenTermsArchive / engine

Tracks contractual documents and exposes changes to the terms of online services.

https://opentermsarchive.org

European Union Public License 1.2

111 stars 30 forks source link

Consider extraction errors as expected operational errors without crashing the engine #1069

Closed Ndpnt closed 4 months ago

Ndpnt commented 5 months ago

Changed

Consider extraction errors as expected operational errors without crashing the engine

Added

Added label empty content to reported issues on GitHub when server returns empty content or when PDF content is unextractable
Added label invalid selector to reported issues on GitHub when CSS selector is invalid

Fixed

Prevent engine crashes caused by invalid URLs in source document links
Ensure the CLI option --types is not ignored
Fix counts of terms when they are specified via the CLI option --types
Fixed display issues for service ID and terms type associated with errors

Other technical improvements:

Make logger silent in test env
Also allow passing no params to the track function
Ensure all errors from fetch function will bubble as FetchDocumentError
Ensure all errors from extract function will bubble as ExtractDocumentError
Minor improvements in logged messages