Open seittema opened 3 years ago
@seittema - Thank you for your post. I am able to reproduce the issue. In the handle_first_request function we are assigning all value to empty except the primary result key(In this case its Items). https://github.com/boto/botocore/blob/develop/botocore/paginate.py#L384 That's why we are getting count as 0. Marking this as bug.
Describe the bug When using a query paginator for dynamodb, it appears that it returns invalid count data if you pass in a StartingToken. We have a process that runs on lambda and for very large data sets we return the LastEvaluatedKey so we can make a subsequent request and continue on. It appears that when you make a pagination call and pass in the StartingToken from a previous request, the first calls Count and ScannedCount are 0 rather than the correct count from the actual call.
Steps to reproduce Here is a sample script showing the invalid data being returned. Load a DynamoTable with at least 4 records with the same PK but different sort keys. The first call with no StartingToken prints the correct 2 counts, but the second call when passing in the previous LastEvaluatedKey returns 0 count even though 2 items are returned.
Output: ItemCount: 2 ScannedCount: 2 ItemLength: 2 ItemCount: 0 ScannedCount: 0 ItemLength: 2
Expected behavior I expect the Count and ScannedCount to accurately reflect what DynamoDB is returning. It appears to be something in the paginate _handle_first_request function since the parsed data has the correct values before being passed into this function where it then gets reset to 0