KinesisLogWriter auto-create reports an error but works

[kdgregory]

The integration tests that auto-create Kinesis streams fail, because of an error recorded in the logs. The actual messages are written as expected.

This did not happen for the August 2023 release, and there are no changes to the Kinesis log-writer on the current development branch.

After investigation (see this Python gist), I have discovered the following:

• Immediately calling PutRecords on a newly-created, newly-active stream will fail when using the stream name.
• The second and subsequent calls succeed.
• Calling with ARN rather than name succeeds.
• A 0.5 second sleep after the stream becomes active also allows the call to succeed.
• A 0.25 second sleep won't.

This appears to be an AWS bug. I've submitted a ticket to Support (and now have to deal with their delaying tactics), and also tried a back-door. However, since it holds up the release of minor but useful functionality, I'm considering work-arounds.

[kdgregory]

I could always disable the assertion, or check that it isn't the "expected" exception, but that seems too hacky to consider.

Writing the tests so that they pre-create streams is a bad idea, since it won't test the auto-create functionality.

I could retrieve the stream ARN post-create, and use it for all subsequent actions. However, this will require a bunch of changes, which could introduce its own bugs.

The 0.5 second sleep seems to be the least-hacky solution. In real-world use, it shouldn't affect very many people. And I can flag with a TODO in case AWS fixes the bug.

[kdgregory]

The sleep caused unit tests to fail (no surprise, really). It looks like I'll need to either (1) wait for AWS to fix their regression, or (2) switching the log-writer to use ARN rather than name. Given my current time availability, #1 might be what happens.

For what it's worth, I opened case 171569048300857 with AWS, and they confirmed that the problem was on their side, but no time to fix.