stevearc
commented
9 years ago Ah, yeah this is because BatchGetItem has no ordering guarantees for the return values. I guess I didn't account for that when I wrote the refresh implementation...whoops. The only reason you're seeing the problem now is probably just jitter or some other small implementation change that amazon made to the backend. The solution is pretty straightforward. I'll see if I can get it fixed tomorrow
Today I ran some old code that uses Engine.refresh() on a list of models, all from the same table, and the call fails with "AttributeError: Cannot change an item's primary key!". This code worked a few months ago, so I dug into how Flywheel implements refresh().
The cause of the error is that Flywheel expects batch_get() to return my results in the order the keys were specified, but DynamoDB seems to be returning them in a random order. I'm watching the JSON response from DynamoDB and they really are in a different order each time. I can reproduce this with a small number of small items, which easily fit in one response and require no paging. Also, I have no UnprocessedKeys in my results.
Did DynamoDB change its behavior recently? I see a comment in the 0.4.5 branch that asserts we'll get the results in order, and I'm pretty sure refresh() used to work in my program, so I'm currently stumped.
DynamoDB's docs don't have a lot to say about result ordering. It mentions attributes aren't ordered, but doesn't explicitly talk about result items.