Additions to the API: Add copyInto, readInto to Serializer

[GoogleCodeExporter]

Currently, Serializer.read and Serializer.copy methods create a new instance in 
first place, then do the work, and finally return that instance.

It whould be great to have also the equivalents:

  public void copyInto(Kryo kryo, T original, T destination);
  public void readInto(Kryo kryo, Input input, T destination);

(and the read and copy methods should be rewritten to use internally these).

This methods would help in several situations...

For instance, it Kryo could be used internally in the standard Externalization 
(externalRead would call Serializer.readInto).

This use-case (Kryo inside standard Serialization) will be a big improvement 
inapplication using standard JMS, RMI or remote EJBs. In fact, I also suggest 
to document it in the main page of the Project.

Cheers!

Original issue reported on code.google.com by serverpe...@gmail.com on 8 Jul 2012 at 8:26

[GoogleCodeExporter]

Kryo2 initially separated object creation from deserialization. There was 
Serializer#create to return a new instance and then Serializer#read to populate 
that instance. This proved awkward for immutable instances and was confusing 
for users. Eventually it was changed to what we have now, which is just 
Serializer#read that creates and populates as needed.

While I agree they would be very useful, I don't see a clean way to implement 
copyInto and readInto. Separating create and populate proved to be messy, so 
going back to that doesn't have much appeal. I hesitate to add new methods to 
Serializer, since it is subclassed so many times. Also I try to keep the stack 
calls to a minimum, since the stack limits the depth of object graphs that can 
be handled, though this last issue is minor.

Possibly using Kryo in Externalizable could be done differently. 
FieldSerializer could be subclassed to allow the object instance to be set, 
which it would return in FieldSerializer#create, instead of creating a new 
instance. In readExternal you'd have to get an instance to Kryo somehow (maybe 
through an ObjectInput implementation?), set the FieldSerializer instance to 
"this", and then let Kryo do the deserialization. This sounds a bit messy, but 
possibly could be streamlined with a nice API.

Original comment by nathan.s...@gmail.com on 11 Jul 2012 at 3:31

[GoogleCodeExporter]

Hi Nate,

I had already implemented a proof of concept in my Library/Project exactly
how you say, in order to use the benefits of Kryo but inside "standard"
Serialization and Clonation. Writing and reading to a persistent JMS queue
through a standard JMS API, now really flyes!!

What I've done:

1.- Created a XSerializer interface:

    public interface XSerializer<T extends Object> {
        public void readInto(Kryo kryo, Input input, T destination);
    }

2.- Create "XFieldSerializer extends FieldSerializer implements XSerializer"
and implement that method copy-and-pasting from FieldSerializer code :-((((

3.- Create another custom-List-serializer of my own (needed for a special
kind of list in my Project), whick also implements XSerializer

4.- When creating Kryo instances, register XFieldSerializer.class as the new
default serializer

5.- Created ObjectOutputWrapperStream and ObjectInputWrapperStream clases
which wrap/delegate tu ObjectOutput / ObjectInput, and set these stream
wrappers as the underlying streams of the Output and Input instances used.

6.- My classes now "implements Externalizable, Cloneable", and uses a
thread-local instance of Kryo in writeExternal, readExternal, and clone.

The defensive trick in readExternal  is to ensure that the serializer chosen
by Kryo implements XSerializer, otherwise forcing to use the new "default"
XFieldSerializer.

Surely this could be made more elegant.

This is only optimal as long as the main object in the
serialization/clonation graph is one handled by these new XSerializers
created. For now this is sufficient for my concrete case...
In other case, I'm afraid I will have to make my eXtensions to the rest of
the serializers...

Ciao

-----Mensaje original-----
De: kryo@googlecode.com [mailto:kryo@googlecode.com] 
Enviado el: mi�rcoles, 11 de julio de 2012 17:32
Para: serverperformance@gmail.com
Asunto: Re: Issue 79 in kryo: Additions to the API: Add copyInto, readInto
to Serializer

Original comment by serverpe...@gmail.com on 11 Jul 2012 at 4:08

[GoogleCodeExporter]

You shouldn't have to copy paste code from FieldSerializer. 
FieldSerializer#read looks like:

public T read (Kryo kryo, Input input, Class<T> type) {
   T object = create(kryo, input, type);
...

So you can just write something like:

Object destination;
protected T create (Kryo kryo, Input input, Class<T> type) {
   return destination;
}
public void readInto(Kryo kryo, Input input, T destination) {
   this.destination = destination;
   return read(kryo, input, null);
}

Other "generic" serializers (CollectionSerializer, MapSerializer, etc) should 
have a similar create method.

You could avoid the thread local look up in read/writeExternal and clone by 
keeping the Kryo reference in your ObjectInput/Output wrappers.

I'm open to more ideas on how we could make using Kryo with Java serialization 
cleaner. I have a feeling that there isn't a good way to integrate it nicely, 
so hopefully we can make your current approach cleaner.

Original comment by nathan.s...@gmail.com on 11 Jul 2012 at 4:30

[GoogleCodeExporter]

Thank you for the quick response!

This would prevent using the same Kryo instance / serializers for a
standard/direct Kryo usage (in my Project I am using Kryo directly whenever
posible, but integrating with Java serialization (mainly for JMS and RMI
integration).
Or maybe I am losing something...

by keeping the Kryo reference in your ObjectInput/Output wrappers.
Good catch, I'll try it :-)

Original comment by serverpe...@gmail.com on 11 Jul 2012 at 4:43

[GoogleCodeExporter]

Hmm, in that case maybe try:

Object destination;
protected T create (Kryo kryo, Input input, Class<T> type) {
   return destination != null ? destination : super.create(kryo, input, type);
}
public void readInto(Kryo kryo, Input input, T destination) {
   this.destination = destination;
   read(kryo, input, null);
   this.destination = null;
}

Original comment by nathan.s...@gmail.com on 11 Jul 2012 at 4:56

[GoogleCodeExporter]

I will try it (but tomorrow... :-)

Original comment by serverpe...@gmail.com on 11 Jul 2012 at 5:44

[GoogleCodeExporter]

It work :-)

A litte bit dirty, and still depends on the actual implementation detalis of 
FieldSerializer (read invokes create, copy invokes createCopy) but better than 
before.

Please assure that future evolutions doesn't break this compatibility (if 
possible, of course)

Thanks!

Original comment by serverpe...@gmail.com on 12 Jul 2012 at 9:13

[GoogleCodeExporter]

Can you pleas add the createCopy method in CollectionSerializer (and use it 
inside copy, of course). It exists in ohter generic serializers: 
FieldSerializer and MapSerializer (I have had no time to search in others)

Tnx!

Original comment by serverpe...@gmail.com on 12 Jul 2012 at 1:04

[GoogleCodeExporter]

Hi again. One question: how can I replace default serializers? I mean, for 
instance, for replacing the default serializer for collection, I'm invoking:

        kryoInstance.addDefaultSerializer(Collection.class, XCollectionSerializer.class);

But, looking at the code, it seems that Kryo.getDefaultSerializer searches in 
the defaultSerializers list in order starting from 0. So... it won't find my 
replacement as there as the default one is found before. True?

How can I do this replacements?

Original comment by serverpe...@gmail.com on 12 Jul 2012 at 1:50

[GoogleCodeExporter]

Added CollectionSerializer#createCopy.

addDefaultSerializer always puts your additions *before* the default default 
serializers, so it will find yours first. This may be a little problem, because 
order is important. If you add a default serializer for Collection.class, it 
goes first, which means it is before Collections.EMPTY_LIST.getClass() and the 
other Collections that are there by default. I guess I'll wait until someone 
really has the need for full control of the list, I hate to add more APIs when 
it isn't needed. Of course you can just copy paste the addDefaultSerializer 
calls from the Kryo class and customize them.

Original comment by nathan.s...@gmail.com on 12 Jul 2012 at 5:38