dlvenable
commented
2 weeks ago I have tried to reproduce this in order to find the root cause. It does not appear to be exactly from throwing the exception on the routes, but possibly some side effect.
I have this pipeline to attempt to reproduce, but it does not stall.
entry-pipeline:
workers: 1
delay: 10
source:
http:
buffer:
bounded_blocking:
buffer_size: 2
batch_size: 10000
sink:
- pipeline:
name: second-pipeline
- stdout:
second-pipeline:
workers: 1
delay: 10
source:
pipeline:
name: entry-pipeline
buffer:
bounded_blocking:
buffer_size: 2
batch_size: 10000
processor:
routes:
- bad_route: 'this will throw an exception'
sink:
- stdout:
routes: ['bad_route']
kkondaka
Describe the bug
If the routes for a sink fail, such as when the expression is invalid, the process worker running will stop running. This will lead to Data Prepper running without any process workers.
The buffer will fill up and Data Prepper will have effectively been shutdown.
To Reproduce
Expected behavior
I expect that Data Prepper will continue to run. One difficulty is what to do with the data. We could drop it or send it incorrectly somewhere.
Ideally, we can use the new
_defaultroute if available. Environment (please complete the following information):Data Prepper 2.8
Additional context
This is a very similar issue to #4103, but is manifest through failures in the router and/or sinks.