search

TylerBrock / saw

Fast, multi-purpose tool for AWS CloudWatch Logs
MIT License
1.4k stars 77 forks source link

"watch" did not return data for some log streams in a group #65

Open sc0ttdav3y opened 3 years ago

sc0ttdav3y commented 3 years ago

Hi,

This is a really cool tool — thanks.

This may be a question on usage, or it may be an AWS API limitation or a bug in saw, I'm not sure. I hope you can give me some guidance.

Use-case

I'm running a serverless.com project, and I run saw against a log group set up against a lambda function like this:

saw watch /aws/lambda/my-function-name

Expected

When I invoke my lambda function, I expect to see the log output appear in saw for all invocations.

Actual

saw works beautifully when I invoke my lambda function so it only runs once at a time.

But when I invoke the same function multiple times in parallel (my test was to invoke the function 12 times), saw doesn't seem to detect all the log streams created in the log groups and it only returns the contents of one of the streams — the last one.

If I immediately run saw get /aws/lambda/my-function-name --start -5m then I can see all the logs mixed together as I'd expected to see in watch mode.

Extra info

Lambda creates one stream per invocation. I can invoke my lambda function using InvocationType: Event which runs each invocation asynchronously in parallel. My test runs 12 in parallel.

When I do this, I see all the 12 invocation logs in AWS Console, but saw only returns the contents for 1 of the invocations. It appears to be the last one I invoked.

pkit commented 3 years ago

It just misses even in the middle of log streams too, pretty unreliable.