Dynamic serialization and deserialization via class interface

[GoogleCodeExporter]

I have created a patch against gson-2.1 that adds code for allowing classes to 
define serialization and deserialization by implementing a simple interface. 
The benefit of this approach is that there is no longer a need to register 
special handlers for each class that requires custom serialization. It can be 
used as follows:

public class Foo implements JsonSerialization, 
JsonDeserializedBy<FooDeserializer> {
    public String name;
    public JsonElement serialize (JsonSerializationContext jsc) {
        return new JsonPrimitive(name);
    }
}

public class FooDeserializer implements JsonDeserializer<Foo> {
    public Foo deserialize(JsonElement je, Type type, JsonDeserializationContext jdc) {
        Foo f = new Foo();
        f.name = je.getAsString();
        return f;
    }
}

...
Foo foo = new Foo();
foo.name = "foo";
gson.toJson(foo); // "foo"
gson.fromJson(gson.toJson(foo), Foo.class).name; // "foo"

With my patch, a Gson object now checks the argument sent to toJson to 
determine if it implements JsonSerialization. If so, it calls its 
serialize(jsc) method to retrieve a JsonElement for that class.

When fromJson is called, the given class is inspected to determine if it 
implements the JsonDeserializer interface. If so, the class specified in the 
template parameter (in this case FooDeserializer) is used for deserialization 
by creating a new instance of that class (using the no-argument constructor), 
and then deserialize is called as with any JsonDeserializer object.

This patch adds the two new interfaces (JsonSerialization and 
JsonDeserialization) and adds a few lines of code to Gson in order to provide 
the aforementioned functionality. I'm not sure I chose the appropriate location 
to insert the functionality, and the code is pretty ugly, but it seems to work 
pretty well in my limited test cases.

I'll license the patch under the same license as gson, in case anyone is 
concerned about that.

Original issue reported on code.google.com by mint...@everlaw.com on 12 Jan 2012 at 1:59

Attachments:

• gson-2.1-dynamic_serialization.patch

[GoogleCodeExporter]

Sorry, the example code has a small mistake, as I renamed one of the interfaces 
before submitting the patch. Where it says "JsonDeserializedBy", it should say 
"JsonDeserialization". The patch and the rest of my post are accurate.

Original comment by mint...@everlaw.com on 12 Jan 2012 at 2:03

[GoogleCodeExporter]

That's extremely clever!

FYI, if you're willing to make a single call to 
GsonBuilder.registerTypeAdapterFactory(), I don't think you need to make any 
changes to Gson 2.1 to make this work.

Original comment by jessewil...@google.com on 12 Jan 2012 at 6:03

[GoogleCodeExporter]

[deleted comment]

[GoogleCodeExporter]

Will that actually work, though? From what I could tell looking at the code, a 
TypeAdaptor is registered against a specific type that is later retrieved with 
a map lookup. Since my method relies on any object simply implementing an 
interface, doesn't that require a change to Gson?

The alternative would be for each object to register itself with some 
globally-used Gson object. I preferred the interface approach.

Original comment by mint...@everlaw.com on 18 Jan 2012 at 7:06

[GoogleCodeExporter]

It'll work, but you need to register a TypeAdapterFactory, not a TypeAdapter. 
The factory lets you support any type.

Original comment by jessewil...@google.com on 21 Jan 2012 at 2:06

[GoogleCodeExporter]

Thank you for the continued guidance, Jesse. A TypeAdapterFactory is definitely 
the right way to do this, but I've run into a few issues with accessibility in 
implementing it. I've attached my TypeAdapterFactory and the interfaces 
associated with it, and I would appreciate any suggestions you can give me on a 
better approach. I am aware that using ReflectiveTypeAdapter in the way that I 
have is something of a hack (I should be following an approach like 
TreeTypeAdapter's delegate()), but I don't think it changes the idea much.

There are a few calls to my own custom class called "Reflection". I am not 
going to include it in the attachment, but I'll specify the methods instead:

Class classOfType(Type t)
Return the upper bound on t. If (t instanceof Class), it's simply ((Class) t). 
For a ParameterizedType, e.g., List<String> would be List.class. For something 
like "? extends Comparable", it's Comparable.class.

Class[] getTypeParameters(Class implClass, Class genClass)
Ascends and descends the class hierarchy between implClass and genClass to 
return the array indicating genClass's instantiated type parameters as 
specifically as possible. For any parameter that doesn't resolve completely, 
the behavior follows that of classOfType(). (In its usage here, it's used to 
find the actual class DESERIALIZER when a class implements 
JsonDeserialization<DESERIALIZER>.)

T newInstance(Class<T> c)
Just like c.newInstance() except that exceptions are rethrown as a 
RuntimeException and the accessibility is set to true before invoking the 
constructor (so that non-public constructors can be invoked).

Field getAccesibleField(Class c, String fieldName)
Calls c.getDeclaredField(fieldName), rethrowing exceptions as a 
RuntimeException and setting the resulting Field's accessibility to true and 
before returning it.

Original comment by mint...@everlaw.com on 6 Feb 2012 at 3:19

Attachments:

• factory.zip

[GoogleCodeExporter]

Yeah, you probably shouldn't have to do that much work. See the 
TypeAdapterFactory documentation for an example that includes delegation:

http://google-gson.googlecode.com/svn-history/r1110/trunk/gson/docs/javadocs/com
/google/gson/TypeAdapterFactory.html

For serialization, you should delegate to the concrete class of the type being 
serialized. For deserialization, you should delegate to the adapter of the 
concrete class that implements your interface. You can get both type adapters 
using the Gson instance passed in to create().

Original comment by jessewil...@google.com on 6 Feb 2012 at 4:46

[GoogleCodeExporter]

I'm sorry, but I'm having trouble following your suggestion. My mechanism 
allows a class to implement one or both of JsonSerialization and 
JsonDeserialization<DESERIALIZER>. If Obj obj implements the former, calls to 
gson.toJson(obj) execute a callback to 
obj.serialize(gson.serializationContext). If Obj implements the latter, calls 
to gson.fromJson(Obj.class) effectively execute 
DESERIALIZER.newInstance().deserialize(JsonElement je, Obj.class, 
gson.deserializationContext).

I don't see how I can handle that with a delegate. I understand that I can 
avoid my usage of ReflectiveTypeAdapter, but that's a more minor issue. The 
issue I'm running up against is that in implementing my own TypeAdapterFactory, 
I can't technically access gson.de/serializationContext without hacking around 
the Java security system. This leads me to believe I'm doing something wrong.

In the case where I wish to override the default de/serialization, I don't 
believe that I can perform the delegation you suggest because I'm not actually 
registering a type adapter for every type implementing JsonSerialization and 
JsonDeserialization (my two custom interfaces). Is there some hook to perform 
that registration without resorting to hacks that use the Reflections package 
to find every subclass of the aforementioned interfaces and register a 
TypeAdapter for each?

The point of my factory is to allow objects to simply implement an interface 
instead of having to register themselves with a canonical Gson instance. That 
means that I can't "delegate to the concrete class of the type being 
serialized" for classes that implement JsonSerialization because I need to 
ensure that the object's serialize(jsc) method is called instead. I also don't 
think I can delegate for deserialization because the deserializer is not 
registered with any Gson instance.

Am I missing something simple?

Original comment by mint...@everlaw.com on 6 Feb 2012 at 9:15

[GoogleCodeExporter]

Got it. The JsonSerializationContext/JsonDeserializationContext APIs aren't 
present nor necessary for streaming type adapters implementing the TypeAdapter 
interface. Instead that interface uses 'Gson' which provides a superset of the 
functionality of JsonSerializationContext and JsonDeserializationContext.

If you want, change your interface to take a Gson instance instead. I posted 
another big TypeAdapterFactory example on issue 43; you may want to read it 
through.

Original comment by jessewil...@google.com on 7 Feb 2012 at 5:02

[GoogleCodeExporter]

Perfect! Thank you very much. I'll reply here soon with a cleaner version of my 
InterfaceTypeAdapterFactory mechanism. Perhaps it will turn out to be something 
worth including in trunk after a few iterations.

Original comment by mint...@everlaw.com on 7 Feb 2012 at 5:24

[GoogleCodeExporter]

[deleted comment]

[GoogleCodeExporter]

[deleted comment]

[GoogleCodeExporter]

Thanks a lot for working through this issue with me. Attached is an 
implementation of the mechanism I discussed. It uses the Drink example you 
provided in  issue 43 . My goal was to make it as simple as possible for an 
implementor to perform the conversion to and from Json.

It's also available on github: https://github.com/BMintern/gson-interface

In order for it to work, it must be registered with the Gson instance:
Gson gson = new GsonBuilder()
        .registerTypeAdapterFactory(new InterfaceAdapterFactory())
        .create();

Note that I changed the deserializer class to implement JsonDeserializes 
instead of JsonDeserializer. I felt that the method signature defined there was 
more consistent with JsonSerialization (and slightly easier to use).

Note also that this provides a dead-simple way for a class to recursively 
serialize itself.

Original comment by mint...@everlaw.com on 9 Feb 2012 at 12:02

Attachments:

• gson-interface.zip

[GoogleCodeExporter]

Nice work!

FYI Gson team, mintern's library is a use case for a public getNextAdapter() 
method.

Original comment by limpbizkit on 9 Feb 2012 at 1:05

[GoogleCodeExporter]

Thanks, I'm glad you like it! Unfortunately, it also uses TypeToken(Type) in a 
gross way. I'd be interested in hearing an alternative. See 
GsonContext.nextAdapter(Type).

Original comment by mint...@everlaw.com on 9 Feb 2012 at 1:14

[GoogleCodeExporter]

Looks interesting. mintern/Jesse, do you have any take on the performance 
impact of this? I remember that in the past, annotations caused us quite a bit 
of hit on performance. Does this reflection have any such issues?

Original comment by inder123 on 11 Feb 2012 at 1:31

[GoogleCodeExporter]

I did not perform any performance tests on my mechanism. I can say that in the 
case of large class hierarchies where none actually implements 
JsonDeserialization, the Reflection.getTypeParameters(...) call will visit 
every ancestor of the type. I expect that this could be trivially improved by 
first checking JsonDeserialization.class.isAssignableFrom(...) before calling 
getTypeParameters.

Based on my (somewhat limited) understanding of Gson, the most expensive part 
will be called just once per (de/serialized) type per Gson instance. That is, 
the first time a user calls a Gson method on a type, the reflection 
introspection will slow it down somewhat. Subsequent operations, however, 
should be as fast as they currently are.

Original comment by mint...@everlaw.com on 13 Feb 2012 at 7:25

[GoogleCodeExporter]

We're going to publish the getNextAdapter API in Gson 2.2, though I believe we 
may rename it to getDelegateAdapter().

mintern, any further action you'd like us to take here?

Original comment by limpbizkit on 18 Mar 2012 at 6:08

[GoogleCodeExporter]

That's a good start, but I found in the implementation of my interface that I 
sometimes needed to call getNextAdapter(..., Type). Currently, getNextAdapter 
only accepts a TypeToken as an argument. The solution, then, is to either 
implement getNextAdapter(..., Type) or to make the TypeToken(Type) constructor 
public. A justification follows.

I provide a pair of methods (called thisToJson and thisFromJson) that allow one 
to use Gson to perform the de/serialization while avoiding infinite recursion. 
As you're aware, this makes use of getNextAdapter.

In some cases, however, I have to call getNextAdapter with a different 
type--for example, when using thisFromJson to construct a subclass. In this 
case, I need to call getNextAdapter with a TypeToken other than the one with 
which my TypeAdapter was constructed.

For an example of what I'm talking about, see:

https://github.com/BMintern/gson-interface/blob/master/InterfaceAdapterFactory.j
ava#L137

called by:

https://github.com/BMintern/gson-interface/blob/master/GsonContext.java#L85

called by:

https://github.com/BMintern/gson-interface/blob/master/InterfaceExample.java#L81

Original comment by mint...@everlaw.com on 19 Mar 2012 at 9:29

[GoogleCodeExporter]

Can you use TypeToken.get(type) ?

Original comment by limpbizkit on 11 Apr 2012 at 8:46

[GoogleCodeExporter]

Yes, I can. I don't know how I missed that. Thanks! This bug can be closed.

Original comment by mint...@everlaw.com on 12 Apr 2012 at 8:58

[GoogleCodeExporter]

This is a very useful addition.  Worked fine inside our own codebase.  I think 
it would make a good addition to the core library.

Original comment by a...@shapeways.com on 13 Apr 2012 at 8:49

[GoogleCodeExporter]

Thanks! I'm glad it was useful.

Some notes based on our usage:

1. If a class has a non-static inner class that extends Runnable, it results in 
an infinite loop during serialization. This is a problem with Gson in general, 
as far as I can tell. Implementing JsonSerialization and avoiding the use of 
thisToJson allows one to avoid this problem.

2. As Gson appears to be moving toward a streaming JSON interface, I'm not sure 
how much sense my interface makes. Streaming is clearly superior for large 
datasets, but it adds a lot of complexity when you just want to serialize a 
simple object.

3. Make sure you have the latest version. I made a change nearly a month ago 
that improved efficiency and fixed the problem outlined in (1) above. Just 
today I committed the change suggested by comment #20 above.

4. My package introduces several new classes that might be confusing in the 
general case (JsonSerialization, JsonDeserialization, JsonDeserializes, 
GsonContext, InterfaceTypeAdapter, InterfaceAdapterFactory). Some of my 
GsonContext functionality could potentially be rolled into the Gson object 
itself, but otherwise the departure from standard Gson means that I can't 
simplify the interfaces all that much.

5. Requiring an object to be deserialized by a separate class that implements 
JsonDeserialization is a bit cumbersome, but there's not a good way around it 
as far as I can tell. One pattern that eases the indirection is to make the 
deserializer be a static inner class of the class it is deserializing. In order 
to do that, in YourClass.java you'll have to import 
YourClass.YourClassDeserializer, where YourClass implements 
JsonDeserialization<YourClassDeserializer>, and YourClassDeserializer 
implements JsonDeserializes<YourClass>.

I hope that helps. Definitely report any issues you have on my github... I'm 
generally pretty responsive.

https://github.com/BMintern/gson-interface

Original comment by mint...@everlaw.com on 13 Apr 2012 at 9:15

[GoogleCodeExporter]

Original comment by limpbizkit on 2 Sep 2012 at 9:50

• Changed state: Done