Closed recumbent closed 4 years ago
Hello,
thank you for reaching out about this. Automated pagination is not enabled for FilterLogEvents
because the operation returns multiple lists. This is unfortunately not a supported case for automated pagination.
We will keep this use case in mind for future improvements.
@matteo-prosperi thank you for responding.
So my understanding is clear - the challenge is that the underlying API returns multiple lists - specifically in this case a list of events and a list of searched log streams - and as a consequence automated paging is not (currently) supported.
This is the reason for the difference between the behavior of the CLI and the Tools for Powershell.
Makes sense.
@recumbent You can have a look here for documentation about our current pagination approach https://aws.amazon.com/blogs/developer/output-pagination-with-aws-tools-for-powershell/.
I am also looking into improvements to address this issue but I am not able to commit to anything yet.
Hello, thanks for opening this issue. We changed the pagination algorithm to be more robust and flexible in the AWS.Tools modules starting with today's release (3.3.618). You can find more details in https://github.com/aws/aws-tools-for-powershell/issues/61.
At this time, we are not considering to port these changes to the AWSPowerShell and AWSPowerShell.NetCore modules due to backward compatibility concerns.
If I make a call to the aws cli along the lines of the following:
aws logs filter-log-events --log-group-name '/my/log/group' --start-time "$(date -d "-1 day" +%s)000" --filter-pattern '{ $.level != INFO }'
Then I see all the messages matching the filter for the last 24 hours (for my typical use case about half a dozen).
If I make an equivalent call using the tools for powershell
Get-CWLFilteredLogEvent -LogGroupName "/my/log/group" -StartTime $filterStart -Filter "{ $.level != INFO }"
(where $filterStart is an equivalent start time to the above).I see rather fewer messages. I also get a continuation token (took me way too long to work that out...)
Expected Behavior
The behavior should be consistent across the CLI and the tools for Powershell - so in this case powershell should return all of the filtered messages for the time period in the same way that the CLI does
Current Behavior
Powershell returns a
NextToken
where the CLI does not and lists 4 searched log streams (per call) where the CLI lists 10 in a single result.Context
Firstly it seems to me that the purpose of using "filtered" is to be able to more trivially find the interesting data from larger list of log messages. If I need to start worrying about NextTokens for what is ultimately 7kb of data then it ceases to be trivial.
Secondly I'm far more comfortable with Powershell than Bash (and more able to further play with the output), but need to be able to work on either - to be able to switch seamlessly between the two is helpful. If the behavior of the two toolkits is inconsistent then its much harder to do so.
Your Environment Include as many relevant details about the environment where the bug was discovered.
Get-AWSPowerShellVersion
: 3.3.553.0$PS$PSVersionTable.OS
: Microsoft Windows 10.0.18362$PSVersionTable.PSEdition
: Core 6.2.2