splunk / pipelines

Concurrent processing pipelines in Go.
MIT License
21 stars 8 forks source link

Consider Option-based Pipeline stages #14

Closed kalexmills-splunk closed 2 years ago

kalexmills-splunk commented 2 years ago

Currently, the only way to handle optional stages is to have the calling code explicitly handle nils and use pointer types. An alternative could be to have "Option" variants of relevant stages. For example:

// OptionMap calls f for every value received from in and, sends any non-nil results to the output channel.
func OptionMap[S, T any](ctx context.Context, in <-chan T, f func(S) *T) <-chan T

This could also be used to implement FilterMap; which can omit certain inputs based on arbitrary logic embedded in f.