search

revelytix / carbonite

Clojure library for serializing Clojure data using Kryo
Apache License 2.0
59 stars 29 forks source link

Added default Kryo behavior to kryo-extend multimethod. #1

Closed sritchie closed 12 years ago

sritchie commented 12 years ago

The current implementation doesn't necessarily allow setRegistrationOptional to behave as expected (where unknown java classes are serialized using the FieldSerializer.) This commit implements Kryo's default behavior as a separate function, register-generic, and adds a default dispatch method to kryo-extend that calls register-generic if no custom serialization made it.

Patch here: https://gist.github.com/1425255

sritchie commented 12 years ago

Let me know what you guys think! I also wanted to ask your opinion on setting optional registration with a keyword argument.

sritchie commented 12 years ago

Actually, this makes me think of a potential problem. I was reading the Kryo page and cake across this note:

When a class is registered, it is assigned an ordinal number. This integer is used in the serialized bytes to identify what class to instantiate when the object is deserialized. By using an integer, a class can be represented very efficiently, usually with just a byte. The downside is that the ordinals must be identical when the class is deserialized. To do this, the exact same classes must be registered in the exact same order when the object is deserialized.

could it be that kryo-extend will cause issues between different compilations of the same source, or if additional multimethods are added between serialization and deserialization?

puredanger commented 12 years ago

On the original patch, this makes sense and I will apply it.

On the ordering issue - there is an implicit assumption in the current Kryo setup that assumes that the reader and writer have registered the same classes in the same order. If you are explicitly registering all your classes and encapsulating that setup in a single function called by both sides, there is no isssue. (This is what we do at Revelytix). If, however, reading and writing code registers them in different orders, there could certainly be a problem.

There is a solution to this in Kryo, which is to call [register](http://kryo.googlecode.com/svn/api/com/esotericsoftware/kryo/Kryo.html#register(java.lang.Class, com.esotericsoftware.kryo.Serializer, boolean)) and pass true for the last argument - that will send the class name string on the first serialization so the receiving side can register it properly (or at least that's my understanding).

sritchie commented 12 years ago

Cool, got it. Just tested it out and it looks like the class name is stored the first time it appears in each object. Thanks!