elastic / logstash-devutils

An assortment of tooling/libraries to make Logstash core and plugin development and releasing a bit easier.
Apache License 2.0
18 stars 27 forks source link

sample spec helper: default to validating output (not mutated input) #97

Closed yaauie closed 3 years ago

yaauie commented 3 years ago

The queue of processed events is not a reliable record of the results of a pipeline's processing. It fails to account for events that are created by filters (such as split, clone, some uses of the ruby filter, etc.), contains events that have been cancelled and were not routed to outputs, and relies on the current implementation detail of the queue holding references to the events that are later mutated.

By default, the sample helper should validate the actual result of the provided event(s) to the pipeline, as received by the outputs.

In order to reduce the risk of one of our plugins somehow needing to rely on queue state for validation cascading to a reversion of this change, we also provide an option for plugins to explicitly opt to validate the queue (supply validation_source: :queue to the sample helper).