taoensso / nippy

The fastest serialization library for Clojure
https://www.taoensso.com/nippy
Eclipse Public License 1.0
1.04k stars 60 forks source link

Support for object-array #113

Closed isaksky closed 5 years ago

isaksky commented 5 years ago

I'm seeing if I can make nippy work with datascript without having to take a slow / inefficient approach, and object-array seems to be among the types that are stopping things.

Does this sound reasonable to add support for? If so, should it get its own set of type ids like byte arrays do? (E.g., objects-[0, sm, md, lg]?)

Btw, I see that int-arrays work, though I couldn't immediately see how (there is no [I class check like how there is a [B check for byte arrays). Is it via java serialize, edn, or something else?

ptaoussanis commented 5 years ago

Hi Isak,

I'm unfortunately not familiar enough with Datascript to be sure what you mean by "work with" in this context, or what the various possibilities are.

But assuming there's a decent motivation for supporting object array serialization, I can comment on the id set question. The value of the [0, sm, md] prefixes is strictly as a space micro-optimization for common collection types (esp. when the collections are typically small).

One reasonable option would be to just to define a single object id type, implicitly as type lg. It'll be a little space inefficient for small arrays, but probably not to any degree that matters.

If there's ever a motivation to further optimize the space usage, you could introduce the other variants in a non-breaking way by just renaming id-object to id-object-lg at that time.

Does that make sense?

I'd probably prefer that route, just as a safe way of gauging demand for this type.

Btw, I see that int-arrays work, though I couldn't immediately see how (there is no [I class check like how there is a [B check for byte arrays). Is it via java serialize, edn, or something else?

Not sure off hand. You could check the byte-array sequence to identify the type id being used.

Hope that helps!

isaksky commented 5 years ago

Makes complete sense, thanks!

ptaoussanis commented 5 years ago

Great, merging manually in a moment- thanks Isak!