mozilla-services / foxsec-pipeline

Log analysis pipeline utilizing Apache Beam
Mozilla Public License 2.0
25 stars 9 forks source link

Improve error handling to reduce pipeline stalls #507

Closed kkleemola closed 2 years ago

kkleemola commented 3 years ago

Unhandled exceptions cause the runner to retry messages infinitely when in streaming mode. If the error is deterministic (i.e. because it's from parsing a message), the pipeline will stall requiring manual intervention. These types of errors should be caught and outputted (either as a log message we can alert on or as an error message with the original event so we can track down problems easier).

kkleemola commented 2 years ago

Partially solved with #553