mapbox / dyno

simple dynamodb client
MIT License
78 stars 28 forks source link

Safer batch writes #82

Closed rclark closed 9 years ago

rclark commented 9 years ago

Refs #80

This adds aggregation of unprocessed items across multiple batch requests that a single dyno.putItems or dyno.deleteItems request might make. The scenario that is tested:

Similar test scenario for dyno.deleteItems. Note that this is only tested for the case of user error, where you have handed dyno some invalid records. I don't have a good way to simulate a batch write request that say, makes 12 requests successfully, then fails due to networking problems, resulting in an additional 13 unprocessed items.

If any requested operations fail because the table's provisioned throughput is exceeded or an internal processing failure occurs, the failed operations are returned in the UnprocessedItems response parameter

Dynalite will never let me exceed provisioned throughput to test this.

cc @emilymcafee @willwhite

rclark commented 9 years ago

@willwhite curious if I should merge this before/after your work on #41? I'd rather deal with resolving any conflicts myself so you can keep moving.