It is not possible to serialize classes without a zero-argument-constructor

[GoogleCodeExporter]

What steps will reproduce the problem?
1. generate a class without default constructor
2. serialize it with kryo

What is the expected output? What do you see instead?
expected: no output
output: Exceptions about missing zero-argument constructor

What version of the product are you using? On what operating system?
0.92

why not using the internal way of creating an empty object?
example source attached

Original issue reported on code.google.com by kp86...@googlemail.com on 30 Nov 2009 at 3:04

Attachments:

• createObject.java

[GoogleCodeExporter]

You can modify newInstance method in Serializer class to solve problem:

        try {
            Constructor<T> cons = null;
            cons = type.getDeclaredConstructor(new Class[0]);
            cons.setAccessible(true);
            return cons.newInstance(new Object[0]);
        } catch (Exception e) {
            throw new RuntimeException("Class cannot be created (missing 
no-arg constructor): " + type.getName(), e);
        }

Original comment by yipen...@gmail.com on 1 Dec 2009 at 3:13

[GoogleCodeExporter]

To properly support serializing a class that has no zero-arg constructor, 
whatever
constructor does exist must be called with proper parameters. This is not easily
done. How would Kryo know which constructor to use if there are multiple?

Neither of the two proposed solutions allow serializing a class that has no 
zero-arg
constructor.

To do it, you will need to implement a Serializer for the specific class you 
want to
serialize. Eg:
http://code.google.com/p/kryo/source/browse/trunk/test/com/esotericsoftware/kryo
/serialize/SerializerTest.java#207
Note that you can make use of existing serializers.

Original comment by nathan.s...@gmail.com on 6 Jan 2010 at 6:31

• Changed state: WontFix

[GoogleCodeExporter]

I want to explain the Java 'original' way to serialize/deserialize Objects.

Given that we have the following class hierarchy:
Object -> A -> B -> CS -> DS -> ES
(The classes with S implement Serializable)
at creating the Object ES with new ES(...) the constructors will be called in 
this
order: Object, A, B, CS, DS, ES

at deserializing the Object ES you have to look for the first class which is not
Assignable to Serializable. This class must to have a zero-arg constructor! 
Then with
that constructor and the class info from ES, an Object will be created for that 
given
class.
The constructors that will be called are only: Object, A, B

The Solution to the original Problem would be to "interpret" a registered class 
as
"Serializable" and look for the first class which is not registered, which must
contain a zero-arg constructor.

On the long run it would be better to not use registered classes and instead
implement some marker-interface.

Some sample code for a possible solution attached

Original comment by kp86...@googlemail.com on 26 Feb 2010 at 2:58

Attachments:

• Serializer.java

[GoogleCodeExporter]

[deleted comment]

[GoogleCodeExporter]

[deleted comment]

[GoogleCodeExporter]

Java's default serialization has a lot of problems and "gotchas". One of these 
is
that it provides an extralinguistic mechanism to construct classes. When the 
default
deserialization occurs, it doesn't call a constructor to create an instance of a
class. This must be carefully planned for and increases the chances of bugs and
security holes.

Even if we wanted Kryo to support object construction without calling 
constructors, 
the mechanism Java's default serialization uses to do so is not made available 
for
our use. It is not possible any other way. To create an instance of a class, a
constructor must be called.

The code you posted doesn't compile, and I'm not really sure where you are 
going with
it. [edit] Ahh, I see from the code in your original post that you are using
sun.reflect.ReflectionFactory. I don't think this is appropriate for Kryo, 
since it
is a general purpose library. Even if using a sun.* class was acceptable, I 
don't
like the "magic backdoor" that the default serialization uses.

It is not hard to write a serializer for problematic classes. Please see the
"testNoDefaultConstructor" method in the SerializerTest.java file linked above. 
I
have just now added a more complex example to that test showing how to use
FieldSerializer with a class lacking a zero argument constructor.

Original comment by nathan.s...@gmail.com on 27 Feb 2010 at 9:23

[GoogleCodeExporter]

You wrote:
> Even if we wanted Kryo to support object construction without calling 
constructors, 
> the mechanism Java's default serialization uses to do so is not made 
available for
> our use. It is not possible any other way. To create an instance of a class, a
> constructor must be called.
With a sun jdk it's possible via sun.reflect.ReflectionFactory:

final Constructor<?> constructor = 
ReflectionFactory.getReflectionFactory().newConstructorForSerialization( cls, 
Object.class.getDeclaredConstructor( new Class[0] ) );
constructor.setAccessible( true );
return (Collection<Object>) constructor.newInstance( new Object[0] );

For other jvms there should be similar possibilities.

Cheers,
Martin

Original comment by martin.grotzke on 24 Mar 2010 at 11:30

[GoogleCodeExporter]

Again, I don't feel sun.* classes are appropriate for use in a general purpose
library. Even if they were, the "magic backdoor" that the default serialization 
uses
is a bad idea.

Original comment by nathan.s...@gmail.com on 24 Mar 2010 at 11:33

[GoogleCodeExporter]

This (ReflectionFactory) btw. also solves deserialization of private classes...

I'd also be interested in deserialization of classes with no default 
constructor and 
private classes, I'm still interested in kryo to integrate as serialization 
strategy 
into the memcached-session-manager.

Cheers,
Martin

Original comment by martin.grotzke on 24 Mar 2010 at 11:34

[GoogleCodeExporter]

ReflectionFactory doesn't call a constructor to create an instance of a
class. This must be carefully planned for and increases the chances of bugs and
security holes. In fact, the serialization proxy pattern is recommended 
(Effective
Java by Joshua Bloch, item 78) to bypass the complexities and potential 
problems with
Java's built-in serialization. IMO, if you have to go to extra trouble anyway, 
you
might as well just write a Kryo serializer that handles reading/writing 
constructor
parameters and non-zero arg construction.

If you really want ReflectionFactory functionality, Kryo serializers are 
extensible.
I believe comment 3 above provides the code to extend FieldSerializer to create
instances with ReflectionFactory.

Regarding using Kryo for memcached-session-manager, I haven't forgotten about 
you! We
have come part way as Kryo now supports serializing unregistered classes 
(though I
feel it still needs some refinement). We don't yet have forward/backward
compatibility, but it is on my todo list.

Original comment by nathan.s...@gmail.com on 25 Mar 2010 at 12:26

[GoogleCodeExporter]

Great, thanx for this info! Really looking forward to using kryo! :-)

Original comment by martin.grotzke on 25 Mar 2010 at 8:17

[GoogleCodeExporter]

I'm thinking loud here regarding "Best effort discover, record and invoke of 
appropriate constructor to call". People can add to this idea/steps and see if 
we 
can do better than invoke newInstance() and fail if there isn't a no-arg 
constructor. 

In writeObject():
- If class has no arg constructor do the usual, if not then below

- Identify a least argument constructor whose parameters (by type and name) can 
be 
mapped uniquely to the object fields.
   1) Look for all one arg constructors and inspect the fields (type and names) and 
see if any of them can uniquely be mapped to the constcutor param. If so, that 
is 
the candidate constructor to be recorded for use by readObject(). 
   2) If found, serialize the Constructor object (name, params) with the values of 
the params to be passed for construction to the buffer (may be some 
standardized 
marker can separate constructor, constructor params from the object fields).
   3) If not, repeat (1) and (2) for two arg, three arg and so on progressively till 
all declaredConstructors are exhausted.

readObject():
- If after reading the Class info, if there is a Standard marker that indicates 
a 
constructor related data then read the Constructor info, read params and invoke 
the 
constructor. 
- Once the object is constructed with no exceptions/errors, go about 
de-serializing 
the object.

Feedback/criticism welcome

Thanks,
-Prasad

Original comment by vvoo...@gmail.com on 26 Mar 2010 at 7:19

[GoogleCodeExporter]

That could work in some situations, but fails when there are two fields the 
same type
as a single argument constructor, or if multiple constructor arguments are of 
the
same type. Even when the proposed approach can be applied, there is no 
guarantee that
the chosen fields really should be applied to the constructor. I think we have 
to be
sure. If we guess then sometimes we will be wrong, which will cause a major and
possibly difficult to detect problem.

The official way to map fields to properties is with the
java.beans.ConstructorProperties annotation, which is Java 1.6+ only. A second
mechanism would be needed both for Java 1.5 and for third party classes that 
don't
use the annotation. I suppose it would work something like
FieldSerializer#setConstructorFields(String...). However, it seems that this is 
only
slightly less work than just writing a small serializer to construct the object.
Still, if users would find this useful and feel it best solves the problem, I 
can
reopen this bug and implement this proposed solution.

Original comment by nathan.s...@gmail.com on 27 Mar 2010 at 12:33

[GoogleCodeExporter]

I'd vote for Kryo supporting the @ConstructorProperties annotation. Those of us 
using Java 1.6 can use this. Others have nothing to lose w.r.t current 
functionality. Also, I'm not too sure if implicitly deriving the constructor 
props 
by type and name match will be an issue in most applications. The object (being 
constructed) by itself may not have much impact as its state will be updated 
from 
the serialized state after the construction. It can however have side effects 
viz. 
initializing the state of another object or opening a resource not needed etc. 
If we 
provide a way to turn-on/off the implicit derivation of constructor props then 
applications that don't see such side effects can turn-on and others can keep 
it 
off.  

Original comment by vvoo...@gmail.com on 29 Mar 2010 at 4:14

[GoogleCodeExporter]

Will look into some sort of constructor properties support.

Original comment by nathan.s...@gmail.com on 5 Apr 2010 at 3:50

• Changed state: Accepted

[GoogleCodeExporter]

how about a zero arg private constructor? this could be accessed using 
reflection and
would allow an effectively immutable object to be serialized without requiring a
public zero arg constructor that would violate immutability.

Original comment by pfirm...@bigpond.net.au on 7 Apr 2010 at 8:54

[GoogleCodeExporter]

I think that is quite a clever idea! Support for private constructors is in 
SVN, r98.
Thanks!

People probably still want support for mapping fields to constructor parameters 
for
classes were the source cannot be modified. This will get implemented 
eventually.

Original comment by nathan.s...@gmail.com on 7 Apr 2010 at 9:47

[GoogleCodeExporter]

Wow that's got to be the fastest response i've seen ever! Certainly encurages
comments. Could I suggest another method?

private void defensivelyCopyState();

It could be called using reflection, optionally allowing an object to make 
defensive
copies of internal mutable state, eg: guard against stolen references, so 
contained
mutable objects can be effectively immutable.

N.B. I've just found your project, looking for a new serialization framework 
after
realising ObjectStreamClass prevents ClassLoaders from being garbage collected, 
while
trying to work out how to handle package versioning and ClassLoader isolation 
for
remote objects.

Thanks,

Peter

Original comment by pfirm...@bigpond.net.au on 7 Apr 2010 at 9:13

[GoogleCodeExporter]

So Kryo would call defensivelyCopyState, if it exists? When would this happen? 
One
issue with using setAccessible is that it is not available in some 
environments, such
as applets.

I've thought of adding an interface named Deserialization, which would have a
"deserialized" method which is called after an object has been completely
deserialized. Currently the only place to take action after deserialization is 
in the
registered serializer.

Original comment by nathan.s...@gmail.com on 8 Apr 2010 at 12:14

[GoogleCodeExporter]

Yes, if it exists, if not or where the environment didn't support it (if 
private), it
wouldn't be called. The only consequence of not calling it would be that the 
state
would not be defensively copied if it wasn't called, the references would still 
be
valid. I wonder if it would be possible to register a null serializer that 
calls the
method when no serializer exists and or insert a serializer into the inheritance
hierachy for serializers that calls the method if it exists? 

By the sound of your Deserializer interface you've already thought about this 
and it
sounds like a good decision.

If the method is public the implementor should set a volatile boolean variable 
to
ensure it is only executed once, especially where immutability is used for 
thread safety.

I'm going to have to study your code to get a better understanding.

Cheers & Thanks,

Peter.

Original comment by pfirm...@bigpond.net.au on 8 Apr 2010 at 2:47

[GoogleCodeExporter]

N.B Looks like you've done heaps of work here, the code looks really clean too. 
 I
see you've got a Serializer class, could it be used to execute your 
Deserializer method?

Original comment by pfirm...@bigpond.net.au on 8 Apr 2010 at 3:00

[GoogleCodeExporter]

Let's move the "deserialized" method discussion over to the discussion group:
http://groups.google.com/group/kryo-users

Original comment by nathan.s...@gmail.com on 8 Apr 2010 at 3:18

[GoogleCodeExporter]

Original comment by nathan.s...@gmail.com on 10 Oct 2010 at 2:50

• Added labels: Type-Enhancement
• Removed labels: Type-Defect

[GoogleCodeExporter]

Is it possible yet to deserialize classes without 0-arg ctors? Ran into this 
via 
http://stackoverflow.com/questions/7590557/simple-hassle-free-zero-boilerplate-s
erialization-in-scala-java-similar-to-pyth. (Bear in mind that your users 
aren't necessarily the authors of the classes they want to serialize, so even 
if we were willing to pollute our codebase with superfluous 0-arg ctors, it 
would not be possible for third-party code.)

Original comment by yaa...@gmail.com on 3 Oct 2011 at 8:05

[GoogleCodeExporter]

Also added as comment on SO: If you're using a sun/oracle jvm you can use 
http://github.com/magro/kryo-serializers for deserializing objects without a 
0-arg constructor: just change "new Kryo()" to "new 
KryoReflectionFactorySupport()"

Original comment by martin.grotzke on 3 Oct 2011 at 8:19

[GoogleCodeExporter]

v2 supports Objenesis, which can create classes without a zero argument 
constructor.

Original comment by nathan.s...@gmail.com on 17 Apr 2012 at 10:20

• Changed state: Fixed