search

stevearc / flywheel

Object mapper for Amazon's DynamoDB
MIT License
128 stars 25 forks source link

Does flywheel automatically deals with the 1MB limit per API call in the queries? #20

Open rcalsaverini opened 10 years ago

rcalsaverini commented 10 years ago

According to this page, the queries on dynamoDB are limited to 1MB per result set and any aditional results can be retrieved by doing extra API calls using the LastEvaluatedKey value that is returned in the previous call.

Does flywheel deals with this "pagination" automatically?

Thanks.

stevearc commented 10 years ago

Flywheel automatically handles pagination. This has been confirmed with unit tests when the number of results exceeds the limit, but the 1MB data limit has not been explicitly tested.

Govinda-Fichtner commented 8 years ago

@stevearc What does that exactly mean? If I have more data than 1 MB for a query Flywheel will fetch it automatically under the hood with multiple requests?

And from the perspective of using the all() method - does this mean that I get a list with all results including those above 1MB back?

I feel that the documentation about [Queries](http://flywheel.readthedocs.io/en/latest/topics/queries.html} should describe this case and the behaviour in detail.

stevearc commented 8 years ago

Short answer: yes

Under the hood you're going to get back a ResultSet when you perform a query. This is an iterator that will perform successive queries for you until there are no more results. It completely abstracts away the pagination so you should never have to worry about it.

all() will iterate to the end and give you a list of models that were built from the results. Compare with gen(), which returns a generator that will perform the successive fetches on-demand.

Govinda-Fichtner commented 8 years ago

@stevearc Thanks a lot for the clarification!