freezing large clojure map with java int arrays

taoensso / nippy

The fastest serialization library for Clojure

https://www.taoensso.com/nippy

Eclipse Public License 1.0

1.04k stars 60 forks source link

freezing large clojure map with java int arrays #125

Closed drorbemet closed 3 years ago

drorbemet commented 4 years ago

Hi, first thanks a lot for nippy. Now how can nippy be configured or adapted to support freezing about 10GB of clojure maps with java int arrays? With freeze-to-file I am getting an error about an unfreezable int array object as soon as the size of the value reaches 2GB. Did I miss something here?

ptaoussanis commented 3 years ago

Hi there, sorry for the delay replying.

By default, int arrays will be serialized using Java's Serializable interface. I'm not sure if that has a limitation re: size. If so, an alternative would be to use extend-freeze, extend-thaw to implement support via Nippy's native protocol.

Hope that helps!