delphi-hub / delphi-crawler

Delphi's crawling and processing engine to extract facts on open-source software
https://delphi.cs.uni-paderborn.de/
Apache License 2.0
5 stars 11 forks source link

Add a crawling pass that retries components which have faced errors before #32

Open bhermann opened 6 years ago

bhermann commented 6 years ago

Is your feature request related to a problem? Please describe. After #31 is implemented we are very aware of components that faced errors before. However, there is no process that picks up the components which failed.

Describe the solution you'd like Add a crawler source that feeds from the list of errors in the elasticsearch index. In order to avoid infinite retries a circuit breaker should be established here.

Describe alternatives you've considered None

Additional context None