Default Deserialization Strategy for Dates

[oleersoy]

Currently an exception is thrown when the date is specified in milliseconds. It would be great if gson would assume that this is the default, or would at least provide a simple builder option for it. More details here:

http://stackoverflow.com/questions/30652314/gson-datetypeexception-when-converting-date-in-typed-in-milliseconds/30653568#30653568

Cheers,

• Ole

[oleersoy]

Just wanted to add some more information. I'm testing a REST update method. First I submit a new contact. Then I update the contact and use gson to create the JSON string sent in the update. When I do this gson serializes the date to:

'Jun 5, 2015 10:38:13 AM'

This results in the following Spring Boot RestController Exception:

{"timestamp":1433519208523,"status":400,"error":"Bad Request","exception":"org.springframework.http.converter.HttpMessageNotReadableException","message":"Could not read JSON: Can not construct instance of java.util.Date from String value 'Jun 5, 2015 10:38:13 AM': not a valid representation (error: Failed to parse Date value 'Jun 5, 2015 10:38:13 AM': Can not parse date \"Jun 5, 2015 10:38:13 AM\": not compatible with any of standard forms (\"yyyy-MM-dd'T'HH:mm:ss.SSSZ\", \"yyyy-MM-dd'T'HH:mm:ss.SSS'Z'\", \"EEE, dd MMM yyyy HH:mm:ss zzz\", \"yyyy-MM-dd\"))\n at [Source: java.io.PushbackInputStream@446e776b; line: 1, column: 123](through reference chain: com.mydemandchain.site.models.Contact["created"]); nested exception is com.fasterxml.jackson.databind.exc.InvalidFormatException: Can not construct instance of java.util.Date from String value 'Jun 5, 2015 10:38:13 AM': not a valid representation (error: Failed to parse Date value 'Jun 5, 2015 10:38:13 AM': Can not parse date \"Jun 5, 2015 10:38:13 AM\": no* Closing connection 0 t compatible with any of standard forms (\"yyyy-MM-dd'T'HH:mm:ss.SSSZ\", \"yyyy-MM-dd'T'HH:mm:ss.SSS'Z'\", \"EEE, dd MMM yyyy HH:mm:ss zzz\", \"yyyy-MM-dd\"))\n at [Source: java.io.PushbackInputStream@446e776b; line: 1, column: 123](through reference chain: com.example.models.Contact["created"])","path":"/api/v1/contact"}

[inder123]

See my answer at: http://stackoverflow.com/questions/30652314/gson-datetypeexception-when-converting-date-in-typed-in-milliseconds/30653568#30653568

[oleersoy]

Hello @inder123

Thank you for pointing out the alternate solutions. Could we add a simple option to the builder for searialization that specifies one of the three standard forms that Spring Boot lists in the exception?

(\"yyyy-MM-dd'T'HH:mm:ss.SSSZ\" \"yyyy-MM-dd'T'HH:mm:ss.SSS'Z'\" \"EEE, dd MMM yyyy HH:mm:ss zzz\" \"yyyy-MM-dd\"))

Also it would be great if GSON had a more intelligent deserialization that mirrors that of Spring Boot Marshalling and perhaps incorporates formats from other frameworks as well.

[inder123]

Can you just set this pattern in GsonBuilder.setDateFormat?

[oleersoy]

I did not know about setDateFormat - Thanks for pointing it out. It looks like it's pretty easy to use. I'll have to play around with it. It would be "Refreshing" though if GSON defaulted to one of the standards that would make it work out of the box for Spring, Jersey, Apache Wink, Drop Wizard, etc.

[inder123]

We would love to change the default Date format in Gson, but can't figure out a way without breaking backward compatibility. If you have ideas, do let us know.

[oleersoy]

Perhaps a profile feature that if set guarantees "Effortless" searialization / deserialization of pojos between gson and various frameworks. Something like GsonBuilder.setProfile('Spring')?

On 06/11/2015 12:19 PM, inder123 wrote:

We would love to change the default Date format in Gson, but can't figure out a way without breaking backward compatibility. If you have ideas, do let us know.

— Reply to this email directly or view it on GitHub https://github.com/google/gson/issues/650#issuecomment-111211865.

[oleersoy]

Just adding a cross reference to one of the Spring tickets I submitted for this: https://jira.spring.io/browse/SPR-13100

At a minimum it would be great it GSON strongly declares in the getting started that it does not serialize to any of the standard date formats, so that developers are on guard. I'm sure a lot of us think that something as simple date serialization/deserialization will work fine in 2015 :).

[inder123]

Instead of a profile like that, may be we should create other dependent projects that configure Gson the right way for a framework.

So, gson-spring could be that subproject that provides a GsonBuilder configuration matched with Spring. Feel free to start one as a separate github project.

[oleersoy]

First I liked the idea. After thinking it through though both our ideas feel "Hackish". When looking for a library I think most of us look for simplicity first. It's possible to provide a standards based json conversion library that's simple to use, just works, and is default go to solution for all developers, so that should be the end goal. Gson is already 99% there.

[inder123]

The challenge is to avoid dragging in dependencies from these frameworks. Gson is around 150kb and is used in android apps where size matters. We don't want to get it bloated.

[oleersoy]

Could GSON support a standards based date format in the next release and provide backward compatibility for those who absolutely have to have it? Encouraging developers to move to a standards based date format should be a good thing.

[oleersoy]

Also this issue has been marked as closed, but I suspect that when the next developer tries to send an object with a date over the wire using REST and GSON they will end up right back here. Could this be reopened?

[inder123]

Backwards compatibility is an absolute requirement, so the default behavior will NOT change. We have been bitten in production by this before.

We have considered providing a different GsonBuilder constructor that makes many of the "good" decisions automatically. Not settled on it yet.

[oleersoy]

OK - Can this issue remain open? It's still causing code bloat and surprising errors for the part of the world that works with standard dates. Also do you not agree that encouraging developers to migrate to a standard date format is a good thing? It's really the insistence on not using a standard that's causing the issue here.

On 06/22/2015 09:51 AM, inder123 wrote:

Backwards compatibility is an absolute requirement, so the default behavior will NOT change. We have been bitten in production by this before.

We have considered providing a different GsonBuilder constructor that makes many of the "good" decisions automatically. Not settled on it yet.

— Reply to this email directly or view it on GitHub https://github.com/google/gson/issues/650#issuecomment-114139374.

[inder123]

I can keep the issue open but I see no easy resolution. We made a mistake in Gson early on, and are living with it. No way to fix it without breaking existing clients. And that breaking is very real. We TRIED changing it to a "better" format when Gson was internal to Google AND we broke a bunch of production systems. Now that Gson is open source and is used extensively, we WILL certainly break a very large number of systems. That is unacceptable.

[oleersoy]

Thanks - I appreciate that. At least if it stays open we can try to find a good solution in the long run.

I'm a little shocked that you are saying that making improvements broke Google production systems. I have heard that Google's hiring targets extremely talented individuals and that the processes is very rigorous. Surely Google engineers understand semver guidelines:

http://semver.org/

If someone introduces a version of any library that is a major release, this should expect and be aware of compatibility changes. If they update their build using a new major version of GSON, then they broke the build, not GSON. That's the way it has worked for a long time now. Personally (And I do understand that change is hard and that Google engineers probably have other things to worry about than refactoring code to fit a new date format) I think the right thing to do is to plan for a new major release that will have compatibility changes in it with respect to standardizing on a date format. Just give everyone plenty of heads up that this change is coming in the next major release. I suspect that 95% of the current GSON client base will be fine with the current version of GSON, and won't need to update to the next major version. GSON is pretty stable anyways right? If someone absolutely has to have a new feature, it could always be backported and maintained in a new branch using the previous major release number and while just updating the minor version. If there is one standard date format used, no one can complain. The ball is in their park. As long as this is not the case everyone has to keep dancing around this issue. Even if it's as simple as using the builder, this requires additional learning and will cause slight code bloat and developers care about code bloat. Just look at the debates around using the commonjs vs amd module format within various javascript communities.

The opportunity is here to just make marshalling and unmarshalling dates something we all can put zero thought into.

Thanks again for keeping it open. I hope we find an elegant solution.

On 06/22/2015 01:15 PM, inder123 wrote:

I can keep the issue open but I see no easy resolution. We made a mistake in Gson early on, and are living with it. No way to fix it without breaking existing clients. And that breaking is very real. We TRIED changing it to a "better" format when Gson was internal to Google AND we broke a bunch of production systems. Now that Gson is open source and is used extensively, we WILL certainly break a very large number of systems. That is unacceptable.

— Reply to this email directly or view it on GitHub https://github.com/google/gson/issues/650#issuecomment-114205293.

[swankjesse]

FYI, if you ask Gson 2.0+ to parse this format, it will: 2015-06-22T19:11:00Z. That means that we could plausibly switch a hypothetical Gson 3.x to write in that format, and 2.x clients wouldn't crash reading it. (The hypothetical Gson 3.x would also need to be able to read Gson 2.x's output.)

[inder123]

Good point, Jesse. I think a large part of the Gson user base has likely moved away from 1.x versions. So, this is a plausible fix. I will reopen the issue.

[oleersoy]

So if a new release of GSON attempts to read multiple date formats, before throwing an exception (Like Spring does), then we have solved half the problem. The current GSON date format can be included in this strategy, so that any GSON messaging application will work with the most popular standard date formats.

It sounds like serializing to a date format that is a standard is what is what is going to cause the most push back within Google.

To solve this GSON could provide provide a binary compatible release for clients that need to be able to read the old date format (If the clients absolutely needs it). Just drop in the new version and now the client can read not only the old date format (By default), but also new ones. HikariCP uses a somewhat similar approach in dealing with Java 6 and 7 vs. Java 8. It provides one release for Java 6 and 7 runtimes, and another for Java 8.

[jrn]

See http://crbug.com/gerrit/5489 for an example where this came up.