taoensso / carmine

Redis client + message queue for Clojure
https://www.taoensso.com/carmine
Eclipse Public License 1.0
1.16k stars 132 forks source link

Key order in maps affect map identity #284

Closed oelrich closed 1 year ago

oelrich commented 1 year ago

Hello!

Using com.taoensso/carmine {:mvn/version "3.3.0-RC1"} (or com.taoensso/carmine {:mvn/version "3.2.0"}) I get the following results:

(car/wcar my-wcar-opts
          (car/set {:a 1 :b 2} "Good")
          (car/get {:a 1 :b 2})
          (car/get {:b 2 :a 1}))
; => ["OK" "Good" nil]

When I expected something simliar to the result when using sets:

(car/wcar my-wcar-opts
          (car/set #{[:a 1] [:b 2]} "Good")
          (car/get #{[:a 1] [:b 2]})
          (car/get #{[:b 2] [:a 1]}))
; => ["OK" "Good" "Good"]
ptaoussanis commented 1 year ago

Hi @oelrich,

What you're seeing is the result of two things interacting:

  1. Nippy being used to auto serialize your Clojure data structures to binary strings for storage in Redis.
  2. The above binary strings being sensitive to the order of keys in small maps, due to implementation details of Clojure's small (array) maps.

Note that while (= {:a 1 :b 1} {:b 1 :a 1}) is true in Clojure, their sequential representations (relevant here due to serialization) are actually different:

(seq {:a 1 :b 1}) ; => ([:a 1] [:b 1])
(seq {:b 1 :a 1}) ; => ([:b 1] [:a 1])

Depending on your specific needs, you have a few options:

  1. Don't use serialized Clojure maps like this, but use Redis's own hashmap structure.
  2. Use a sorted map, or other data structure that'll have a consistent seq order (e.g. set).
  3. Consider using hashes of your Clojure data rather than serialized copies. Clojure's hash fn is already designed to account for these kinds of cases.

Hope that helps!

oelrich commented 1 year ago

Yes, that helped. Thank you!