This feature on model.getItems doesn't seem to work
DynamoDB limits the number of items you can get to 100 or 1MB of data for a single request. Vogels automatically handles splitting up into multiple requests to load all items.
The reason being is that when it detects that the request was capped (by UnprocessedKeys not being null/empty), it doesn't construct the subsequence request properly, resulting in:
Missing required key 'RequestItems' in params
After we get UnprocessedKeys (due to 1mb limit), it tries to request further results automatically. However it doesn't build the subsequent request properly, as you can see here:
https://github.com/clarkie/dynogels/blob/master/lib/batch.js#L13
it is supposed to set RequestItems: as the base key, whereas it's not.
This feature on
model.getItems
doesn't seem to workDynamoDB limits the number of items you can get to 100 or 1MB of data for a single request. Vogels automatically handles splitting up into multiple requests to load all items.
The reason being is that when it detects that the request was capped (by UnprocessedKeys not being null/empty), it doesn't construct the subsequence request properly, resulting in:
Missing required key 'RequestItems' in params
I believe this is a related Vogels issue: https://github.com/ryanfitz/vogels/issues/166
I've tracked down the source of the issue here: https://github.com/clarkie/dynogels/blob/master/lib/batch.js#L46
After we get UnprocessedKeys (due to 1mb limit), it tries to request further results automatically. However it doesn't build the subsequent request properly, as you can see here: https://github.com/clarkie/dynogels/blob/master/lib/batch.js#L13 it is supposed to set
RequestItems:
as the base key, whereas it's not.This results in: Original Request:
subsequent ones:
note the missing RequestItems in the second one.