Fancy stream processing made operationally mundane. This repository is a fork of the original project before the license was changed.
1.06k
stars
71
forks
source link
add: Strict Mode option to Pipeline configuration #174
Open
gregfurman opened 2 days ago
Motivation
We need a
strict_mode
to allow for a processing step to return and log an error, with the subsequent processors not being run.Related to https://github.com/warpstreamlabs/bento/pull/166 -- where we opted for nacking at the failed
processor
instead of at the output layer.Can be used as follows:
Changes
strict_mode
in thepipeline
block for.output
.generate
input component always enablingauto_replay_nacks
.TODO
input.processors
andoutput.processors
to also have their results checked for errors.input
to stop new data being read in.