Changes behavior to check if a stream exists first before attempting to create it.
Fixes a bug where if a user manually deletes and recreates a stream then the plugin fails to make any progress
Testing:
auto_create_stream false
[x] Start fluent-bit with no log stream and stream not created
[x] Create stream and logs start getting populated in the stream
[x] Delete the stream and recreation isn't attempted
[x] Create stream again and logs start getting populated in the stream
auto_create_stream true
[x] Start fluent-bit with no log stream and stream is created and populated
[x] Delete the stream and stream gets created and populated
[x] Delete and create the stream and logs start getting populated in the stream
[x] Start fluent-bit with log stream created and stream gets populated
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.
Bug explained
The bug that was discovered causes the plugin to not make progress when a user deletes and creates a stream. This bug was existent before the changes but was just noticed more easily due to streams not being auto created.
When this delete + create happens the plugin will use an incorrect (old) sequenceToken and will get a InvalidSequenceTokenException. It attempts to recover from this by fetching the token from the error message by parsing the next token from the error message. With a new stream no sequence token is expected and the parsing from the error message fails. To fix we detect if it is a new stream and use nil nextSequenceToken insted.
Error for normal sequence token error:
An error occurred (InvalidSequenceTokenException) when calling the PutLogEvents operation: The given sequenceToken is invalid. The next expected sequenceToken is: 49621219915356076984497278709410931028996559626100016802
Error for new stream sequence token error:
An error occurred (InvalidSequenceTokenException) when calling the PutLogEvents operation: The given sequenceToken is invalid. The next expected sequenceToken is: null
Issue #, if available:
Description of changes:
This change:
auto_create_streamconfigTesting: auto_create_stream false
auto_create_stream true
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.
Bug explained
The bug that was discovered causes the plugin to not make progress when a user deletes and creates a stream. This bug was existent before the changes but was just noticed more easily due to streams not being auto created.
When this delete + create happens the plugin will use an incorrect (old)
sequenceTokenand will get aInvalidSequenceTokenException. It attempts to recover from this by fetching the token from the error message by parsing the next token from the error message. With a new stream no sequence token is expected and the parsing from the error message fails. To fix we detect if it is a new stream and usenilnextSequenceToken insted.Error for normal sequence token error:
An error occurred (InvalidSequenceTokenException) when calling the PutLogEvents operation: The given sequenceToken is invalid. The next expected sequenceToken is: 49621219915356076984497278709410931028996559626100016802Error for new stream sequence token error:
An error occurred (InvalidSequenceTokenException) when calling the PutLogEvents operation: The given sequenceToken is invalid. The next expected sequenceToken is: null