search

pynamodb / PynamoDB

A pythonic interface to Amazon's DynamoDB
http://pynamodb.readthedocs.io
MIT License
2.43k stars 426 forks source link

Query or count all items on table #1037

Open ofekfeller-glossai opened 2 years ago

ofekfeller-glossai commented 2 years ago

Hi all, anyone else having issues trying to count all items on table? i have a model that i created a table from. I'm trying to useModelName.count() but i keep getting 0 even though i have some items there. When sending a specific key to ModelName.count(key) i get the correct results, but i want to count all items.

I tried to query all items and count but it seems that i must set the primary key to query, so this walkaround isn't relevant.

Ill be glad for help here if someone dealt with this one before, Thanks!

MacHu-GWU commented 2 years ago

@ofekfeller1 same issue here

ikonst commented 2 years ago

Agree, we could've documented it better: https://pynamodb.readthedocs.io/en/latest/quickstart.html#counting-items

Alternatively, you can retrieve the table item count by calling the count method without filters:

should mention that we're relying on the table description's ItemCount in this case, and it might not reflect recent changes, as DynamoDB's docs say:

ItemCount The number of items in the specified table. DynamoDB updates this value approximately every six hours. Recent changes might not be reflected in this value.

However, what we're not allowing right now is to do a precise table-wide count, which DynamoDB allows (albeit expensively) by making a Scan request with Select='COUNT'. Is this really something you'd be interested in?

MacHu-GWU commented 2 years ago

@ikonst thank you for providing details, I don't know that Dynamodb Count item doesn't reflect recently changed. I think a better way is to just select all item and return hash_key only. Although it is expensive, but it works.

ikonst commented 2 years ago

In the DynamoDB tables we have, there are often billions of rows, so a full scan only makes sense as one-off maintenance operation. Many tables are smaller, but still the point in using DynamoDB is for data that can scale massively, at which point full scans are not viable.

I think one of the best properties of DynamoDB is that its API prompts one to be explicit about the complexity of DB operations.

MacHu-GWU commented 4 months ago

@ikonst I think we can close this issue now. It is not a bug.