Change with freeze/thaw behavior from 1.8 => 1.9 (or nippy 2.11.1 => 2.12.0)?

[rwilson]

I'm seeing data stored with faraday 1.8 failing to thaw with faraday 1.9.

Putting Data:

• I start with an item map, e.g. {:id 100 :version 1 :data {:foo "bar"}}
• When in DDB, I want the :id and :version keys raw (they're DDB compatible types), but data to be serialized as it can be large and contain clojurey types. So, I explicitly freeze the :data value with nippy-tools/wrap-for-freezing
• The updated map is passed as the item to faraday's put-item

So, that's the data as stored in DDB.

Getting Data:

• 1.8: in the result of a get-item call, the :data value is already thawed.
• 1.9: in the result of a get-item call, the :data value appears still serialized, i.e. #object["[B" 0x607e94b3 "[B@607e94b3"].

I figured maybe whatever auto-thawed the :data value changed, but it seems that the frozen value can't be thawed directly either. Trying to call nippy/thaw on that value results in the errors:

clojure.lang.ExceptionInfo: Unrecognized type id (6). Data frozen with newer Nippy version?
    type-id: 6
clojure.lang.ExceptionInfo: Thaw failed against type-id: 6
    type-id: 6
clojure.lang.ExceptionInfo: Thaw failed against type-id: 112
    type-id: 112
clojure.lang.ExceptionInfo: Thaw failed: Decryption/decompression failure, or data unfrozen/damaged.
    opts: {:compressor :auto, :encryptor :auto}

Previous Dependencies (working version)

[com.taoensso/faraday "1.8.0"]
[com.taoensso/encore "2.64.1"] ;; pulled in by timbre 4.7.0, I think higher than faraday 1.8 spec
[com.taoensso/nippy "2.11.1"] ;; pulled in by carmine 2.13.1, faraday 1.8 had nippy 2.9.1

Current Dependencies (not yet working version)

[com.taoensso/faraday "1.9.0"]
[com.taoensso/encore "2.67.2"]
[com.taoensso/nippy "2.12.0"]

I first noticed it break after upgrading carmine from 2.13.1 => 2.14.0, which I now suspect is because that change pulled in latest encore (2.67.2) and nippy (2.12.0). In debugging that, I saw a faraday update was available that also used those versions, so I tried v1.9 to no avail.

Rolling back to the previous dependency versions restores the working behavior, so the data itself appears uncorrupted.

I looked over the nippy changelog for 2.12.0, which clearly states breaking changes, but didn't see anything obvious since I'm not using nippy directly except for that single wrap-for-freezing call.

Is there a faraday usage of nippy that is subject to the nippy 2.12 breaking changes? Or do you have any other thoughts?

[rwilson]

It seems to be the nippy upgrade. By excluding nippy from the faraday dependency and explicitly including the previous nippy release 2.11.1, I'm observing the previous behavior.

To be clear, this is the dependency set I just tested:

com.taoensso/encore "2.67.2"
com.taoensso/nippy "2.11.1"
com.taoensso/faraday "1.9.0"

Since I clearly mismatched the nippy dependency faraday declares, I'm not sure whether I subtly broke something else. But, it narrowed the breaking change down to nippy.

[ptaoussanis]

Hi Ryan, thanks for the terrific report.

The info here helped me immediately identify the problem. It's a bug with Nippy v2.12.0: seems I accidentally nixed the definitions of two old (deprecated) type ids. (type-id 6 in this case).

It's a trivial fix, just need a few minutes to cut a new Nippy release. Will update here.

Cheers!

[ptaoussanis]

Just pushed [com.taoensso/nippy "2.12.1"] to Clojars. That should do the trick, but I'll wait on confirmation from you to close this?

Cheers!

[rwilson]

Tried out 2.12.1, works as expected. So, all looks good now. Thanks!

[ptaoussanis]

Great, thanks for the confirmation. Sorry about the trouble.