Define an error handling behavior of Serializer.SerializeBatch function

[dialogbox]

Feature Request

Currently, the serializer interface doesn't define error handling behavior of SerializeBatch function.

If an error is occurred in the middle of processing multiple metrics, whether of not return the rest of succeeded metrics is up to the implementation.

Proposal:

Define the expected behaviour and add it in the comment/document.

Current behavior:

Undefined.

It made each serializer uses different strategy.

For example,

• influx: https://github.com/influxdata/telegraf/blob/master/plugins/serializers/influx/influx.go#L114-L122
  • Ignore certain type of error and return nil if others
• graphite: https://github.com/influxdata/telegraf/blob/master/plugins/serializers/graphite/graphite.go#L102-L111
  • returns nil for any single error
• wavefront: https://github.com/influxdata/telegraf/blob/master/plugins/serializers/wavefront/wavefront.go#L102-L105
  • A little unclear but it looks like it ignores all the errors

Desired behavior:

Having well defined set of behavior including

• Whether of not stop processing all the metrics if error occurs from one of the metrics
• Whether of not log the error if we ignore the error
  • Maybe we can change the return error type to []error
• Possibly having a config flag for changing the behavior

Use case:

This is important to reduce possible confusions of end users and plugin writers.

[powersj]

next steps: review how this might look if we want to adopt this

@srebhan you had a conversation about this in #8828, did you have thoughts here?

[srebhan]

@powersj I think that having inconsistent behavior is not a good thing. Furthermore, a serializer might decide to just drop the complete batch resulting in throwing away more data than needed. So I tend to agree that we should define what the behavior is and allow a way to only drop the faulty parts if possible. The same problem might also arise in parsers where you maybe can parse certain information (e.g. from an array or a bunch of text-lines) even-though there is an error elsewhere.