search

taoensso / nippy

The fastest serialization library for Clojure
https://www.taoensso.com/nippy
Eclipse Public License 1.0
1.04k stars 60 forks source link

Generalise to allow serialising from/to any DataInput/DataOutput. #34

Closed cursive-ide closed 10 years ago

cursive-ide commented 10 years ago

This is a simple patch allowing nippy to serialise to/from any DataInput/DataOutput, not just DataInputStream/DataOutputStream.

ptaoussanis commented 10 years ago

Hi Colin, thanks for this - looks good!

I'm rather conservative with changes to Nippy - would like an opportunity to test this out properly to rule out any undesired side-effects.

Unfortunately a little tied up with work at the moment, so might not get a chance to look at this real soon. Will keep your PR open and definitely come back to this, but can't promise when that'll be - couple weeks maybe?

Would suggest a fork if you need this in prod in the meantime, and I'll come back to you once it's in master!

Thanks again, cheers! :-)

cursive-ide commented 10 years ago

Sure, no problem - I'm actually making more significant changes to Nippy for my own purposes, but that was a generally-useful part I isolated out. So I'm already using a fork - no rush on my end.

ptaoussanis commented 10 years ago

Great, thanks.

I'm actually making more significant changes to Nippy for my own purposes

Am curious - were you running into any limitations with Nippy that could be corrected master-end?

cursive-ide commented 10 years ago

No, it's more a very specific use case. I'm looking at using Nippy to serialise index data in Cursive which is quite an unusual usage pattern. I'm serialising lots of small objects and I just rebuild all the indexes when the format changes so I've removed the header, and I don't need compression or encryption and I don't want to have to distribute the dependencies so I've stripped all that out. And to make it easier to modify and maintain I've also removed all the legacy stuff. IntelliJ also uses a StubOutput (derived from DataOutput) for serialising some of its indexes which has support for deduplicating strings, which I'll probably use for keywords.

Basically, I want the smallest, most efficient library for serialising a relatively known set of small objects - apart from this change they're not generally useful.

ptaoussanis commented 10 years ago

Okay, gotcha - thanks for the info.

Happy hacking :-)

ptaoussanis commented 10 years ago

Closing, addressed in an upcoming release.

ptaoussanis commented 10 years ago

Hey Colin,

Just pushed 2.6.0-alpha1 as an early experimental release - includes your DataInput/Output PR. Have also stripped most of the legacy stuff and repackaged it as a headerless mode since that's primarily what it was offering anyway.

Feedback welcome if you've got any other suggestions.

Cheers!

cursive-ide commented 10 years ago

Thanks Peter, I'll take a look when I get a chance. I'm still not using my fork in real code, I'll check 2.6.0 in case that'll work for me instead - I'd rather use an official version when possible.