The decoder currently expects all data passed to FLBPluginFlush/FLBPluginFlushCtx to be a 2 length slice of [timestamp, map data]. But this is not always true and when it is not, the decoder panics, crashes and loses whatever data is pending in memory.
I do not know why this case occurs, but the debugging I have done shows that sometimes, "data" is just a 0 value int64. This is likely something coming from fluent-bit itself rather than any of the go code. It is worth noting that some of the native C plugins in fluent-bit are also doing content checking of the data value, so I am guessing this issue isn't unique to the plugin proxy.
This PR implements some basic sanity checks on the structure of the flushed data and skips anything that isn't a 2 len slice. I am not an expert on Go, nor an expert on fluent-bit internals, so let me know if there is a better way.
Currently, I cannot run my go plugin in production without this patch, as it crashes every few minutes or so.
PettitWesley
commented
4 years ago
Related to:
https://github.com/fluent/fluent-bit-go/issues/34 https://github.com/fluent/fluent-bit-go/issues/29
The decoder currently expects all data passed to FLBPluginFlush/FLBPluginFlushCtx to be a 2 length slice of [timestamp, map data]. But this is not always true and when it is not, the decoder panics, crashes and loses whatever data is pending in memory.
I do not know why this case occurs, but the debugging I have done shows that sometimes, "data" is just a 0 value int64. This is likely something coming from fluent-bit itself rather than any of the go code. It is worth noting that some of the native C plugins in fluent-bit are also doing content checking of the data value, so I am guessing this issue isn't unique to the plugin proxy.
This PR implements some basic sanity checks on the structure of the flushed data and skips anything that isn't a 2 len slice. I am not an expert on Go, nor an expert on fluent-bit internals, so let me know if there is a better way.
Currently, I cannot run my go plugin in production without this patch, as it crashes every few minutes or so.