nervous-systems / hildebrand

Asynchronous DynamoDB client for Clojure & Clojurescript/Node
The Unlicense
66 stars 10 forks source link

Unprocessed items do not seem to be returned. #16

Closed Guthur closed 7 years ago

Guthur commented 8 years ago

It appears that the unprocessed data is ignored in the transform result, returning only the hildebrand/error via the ex-info.

https://github.com/nervous-systems/hildebrand/blob/master/src/hildebrand/internal.cljc#L18

This is very undesirable as the one would most likely wish to retry the unprocessed items.

moea commented 8 years ago

From reading the code, it looks like the ExceptionInfo instance created by that line will have the underlying data associated with it. Calling ex-data on the exception ought to yield a map with an :unprocessed key containing the unprocessed items. If that's not the case, then this is a bug.

moea commented 8 years ago

Yeah, I guess there is a bug, actually.

this line should be returning {:hildebrand/error (assoc data :type type :message message)} - which'd cause ex-data to DTRT.

Guthur commented 8 years ago

@moea Yeah, I was thinking that was the initial intention, the assoc to the ex-data. I was actually going to update it accordingly on a fork for my own needs, I can open a PR from that.

Guthur commented 8 years ago

There is a slight complication; the unprocessed data and hildebrand's chosen format for batch requests are not isomorphic. One nice thing about the underlying AWS API is that the unprocessed response can be sent untouched as a new batch request this is not the case with hildebrands format, it will need to be transformed accordingly.

{:table-name ({:put-request {:item {:name "foo"}}})} => {:put {:table-name ({:name "foo"})}}

moea commented 8 years ago

It totally makes sense to change the representation of :unprocessed. I'd welcome a PR, else I can tackle it, though it may take me a little while.