:limit not working with query function

[johnboy14]

I'm trying to paginate results but my query doesn't seem to limit the result set as expected. Just to be clear I'm querying a table with a composite key of :id and :timestamp. The table has 25 items with the same :id.

(far/query client-opts table {:id [:eq id]} {:limit 10})                                            

Another thing. How do I retrieve the LastEvaluatedKey in my results.

[wit-0-bit]

Hello @johnboy14,

I know its been a while, but I ran into this today and I might have found the cause so I figured I would share in case someone else runs into this.

You might want to check out the span-reqs option. By default, it is set {:max 5} and splits the query amongst a maximum of 5 requests. When max is set to values other than 1, it seems to be a little fuzzy about how many actual results to return. For example, setting it to {:max 2} and setting :limit 2 I received 4 results back. The result count was consistent with :limit when I tried something like:

(far/query client-opts table {:id [:eq id]} {:limit 10 :span-reqs {:max 1}})  

I am not 100% sure that is what you are experiencing, but it might be worth checking out.

About your other question, you can get the LastEvaluatedKey out of the result's meta, where it is stored as last-prim-kvs. For example:

(let [query-results (far/query client-opts table {:id [:eq id]}  {:limit 10 :span-reqs {:max 1}})]
  (:last-prim-kvs (meta query-results)))

I hope this helps

[johnboy14]

I eventually worked out I needed to use the :span-reqs option with a limit value but I wasn't aware that I could pull the :last-prim-kvs using the meta function, so thanks for that.

[belucid]

This one OK to close @johnboy14 ? Or do you have any suggestions for the docs?

[smoes]

I ran into the very same problem and I'd still consider this a bug.

Expanding docs would be a patch and fine as a first step, but not the correct solution in my opinion. DynamoDB's query/scan behaves differently concerning the limit param, doesn't it? I would definitely expect the number I pass to limit to be the maximal number of returned items.

[kipz]

Would a :span-reqs/max default of 0 (i.e. do nothing) be what people would expect there? We could also have an option to make that infinite. In either case, it might make sense to return a lazy-sequence (if we're not already).

I'd be quite happy for the :limit to simply be the value that is sent to dynamodb, and not necessarily be the limit on number of results returned by faraday, but the current default of span-reqs/max of 5 is probably never what I want (either 0 or infinite are more common for me).