search

aws / aws-toolkit-vscode

Amazon Q, CodeCatalyst, Local Lambda debug, SAM/CFN syntax, ECS Terminal, AWS resources
https://marketplace.visualstudio.com/items?itemName=AmazonWebServices.amazon-q-vscode
Apache License 2.0
1.42k stars 358 forks source link

Cloudwatch Logs: customize time range for "Load more" #1298

Open rli opened 3 years ago

rli commented 3 years ago

via IDE feedback:

Hi there!

I was trying to use AWS Toolkit for Visual Studio Code to look up CloudWatch logstream with 1-week range of logs data. I was hoping this will provide me more intuitive and easier experience to export that much logs and save them to disk than using AWS CloudWatch Console (in web). However, I was quickly disappointed to see that I can only load CloudWatch logstream with 1h increments (pagination default in AWS Toolkit for VS Code).

I am struggling to quickly get 1-week range of logs exported so I can attach it to the support case we have open with another 3rd-party service (MongoDB Cloud Health Manager). So at this point, both AWS CloudWatch Console (web) and AWS Toolkit for VSCode have failed me in this task.

It would be really beneficial and good feature if you could add settings to the AWS Toolkit (VSCode plugin) that allows users to configure the pagination ranges when grabbing CloudWatch logstream data. For example, in my task, I wouldn't mind doing 7 clicks to load more logs if I was able to get 1-day range of logs loaded with each click. However, without this setting being available as configuration option of the AWS Toolkit and with default being 1h range of logs, I would need to do 24x7 clicks and wait in between each. Additionally, I have to do this for 3 nodes of the distributed cluster (replica group) so I would have to do this 3x7x24 times, which is not pleasant experience. I am going to look at the AWS CLI APIs next for CloudWatch and see if that can be of more help, but I wanted to let you know that this would be nice configurable option in AWS Toolkit as well.

rli commented 3 years ago

@azec-pdx,

Thanks for the detailed writeup of your use case! Using the console to export data to S3 sounds like a better solution for what you're trying to do, but we will consider your suggestion as well.

justinmk3 commented 3 years ago

Instead of a global setting, I could imagine a Load custom range... next to the existing Load more... codelens, which then shows a relative/absolute picker.

bryceitoc9 commented 3 years ago

However, I was quickly disappointed to see that I can only load CloudWatch logstream with 1h increments (pagination default in AWS Toolkit for VS Code).

The pagination that VS Code using is the default 10k lines/1 MB of entries (whichever is smaller); this is also the max page size: https://docs.aws.amazon.com/AmazonCloudWatchLogs/latest/APIReference/API_GetLogEvents.html#API_GetLogEvents_RequestParameters . Unfortunately, there is no time range associated with the pagination.

We're thinking a good next step for the implementation would either be a time range picker (as @justinmk3 said) or a FilterLogEvents/Insights implementation for more detailed search. Would either of these be of interest to you?

adminy commented 2 years ago

The load previous / next buttons don't do anything, I have to manually pick again.

Maybe a collapsed logs file view would be the ideal experience. with an expand all logs button at the top.

justinmk3 commented 1 year ago

The time range features of the "Search Log Group" feature help with this. (Not released yet; living in branch: https://github.com/aws/aws-toolkit-vscode/tree/feature/cwl )