nervous-systems / hildebrand

Asynchronous DynamoDB client for Clojure & Clojurescript/Node
The Unlicense
66 stars 10 forks source link

BigNumber conversion #10

Open boymaas opened 8 years ago

boymaas commented 8 years ago

Great work on the libs, it is a pleasure using them. I am running into one thing and that is the BigNumbers conversion. When I start doing arithmetic, say fe. adding 0.1 to a 200 stored as a Number type in dynodb, I get "200.1" as a string the following will become "200.1.1" etc etc.

As a quick solution, as I store deeply nested data, I just do a deepwalk converting all BigNumbers to regular number and all is normal again except for the fact I don't like to do a deep-walk at the edges of my system.

Is this a case foreseen, and by design, or not, and what is a way to go around that?

moea commented 8 years ago

Thanks! That doesn't like fun, and definitely isn't the intended behaviour. Can you show an example?

;; Using inc
(hildebrand.core/update-item!! 
  creds :eulalie-test-table {:name "Moe" :age 30} {:shoe [:inc 1]} {:return :all-new})
;; {:name "Moe", :shoe 3N, :age 30N}
;; Using inc w/ double
(hildebrand.core/update-item!! 
  creds :eulalie-test-table {:name "Moe" :age 30} {:shoe [:inc 0.1]} {:return :all-new})
;; {:name "Moe", :shoe 3.1M, :age 30N}
;; Using add /w double
(hildebrand.core/update-item!!
  creds :eulalie-test-table {:name "Moe" :age 30} {:shoe [:add 0.1]} {:return :all-new})
;; {:name "Moe", :shoe 3.2M, :age 30N}
;; Nested inc
(hildebrand.core/put-item!! 
  creds :eulalie-test-table {:name "Moe" :age 30, :x {:y {:z 0}}})
(hildebrand.core/update-item!! 
  creds :eulalie-test-table
 {:name "Moe" :age 30} {:x {:y {:z [:inc 0.1]}}} {:return :all-new})
;; {:name "Moe", :age 30N, :x {:y {:z 0.1M}}}
boymaas commented 8 years ago

Yes I can. I have another update pattern, I am not using update-item!. I am doing a get-item! and put-item! and doing the increments in clojurescript (nodejs). As hildebrand converts to BigNumber when pulling data out of dynamo, I get to work with BigNumber in the clojurescript domain which does not handle the arithmetic correctly.

Example (out of my head):

(def item (hildebrand.core/get-item! .....)) 
;; => {:numeric-field BigNumber 200 }

(update item :numeric-field + 0.1) 
;;=> {:numeric-field String "200.1"}
;; after the first arithmetic the field gets converted to a string
;; as there is no protocol on BigNumber to handle arithmetic

(update item :numeric-field + 0.1) 
;;=> {:numeric-field String "200.1.1"}
moea commented 8 years ago

Sorry, I'm a little slow on the uptake, I was travelling - I didn't know anyone other than me was using the cljs support . Working on browser support at the moment.

Yeah, so as you mentioned, no mechanism for handling arithmetic via + using bignumber.js - you'd have to do something like:

(update item :numeric-field #(.plus % 0.1))

I don't know there's any real way of changing that, either in Javascript or Clojurescript, without changing the approach to handling numbers. If your numbers are small, and small things only ever happen to them, I can see the ubiquitous use of bignumber being tedious - however there would also be problems if we chose when to return bignumbers, and when not. If this is a pain for you, I could add a feature to allow either the "credentials" map (for all requests performed with that map), or the optional final map arg to the Dynamo functions (for per-operation behaviour) to contain a key which will convert all incoming numbers from Dynamo into Numbers, throwing an error if any of them are unrepresentable due to precision.

boymaas commented 8 years ago

Well like they say, I am the other guy :-) Your solution would cover my usecase, credentials and potential override per invocation would be the most flexible solution I would also like to have a "force" option as I want to be able to ignore the errors in the "unrepresentable case". In this case I just want my numbers as js/Number. :+1: