JodaModule creates org.joda.time.DateTime that fails .equals when compared with original

[robertkuhar]

Originally this was https://github.com/FasterXML/jackson-databind/issues/617

I was surprised to find that a org.joda.time.DateTime instance was not .equals to the org.joda.time.DateTime that produced the JSON in the first place.

/**
 * Configure Jackson to do ISO-8601 conversions on DateTimes.
 */
public class JodaThroughJacksonTest {
  static class Beanie {
    public final DateTime jodaDateTime;
    public final Date javaUtilDate;
    @JsonCreator
    public Beanie(@JsonProperty("jodaDateTime") DateTime jodaDateTime,
        @JsonProperty("javaUtilDate") Date javaUtilDate) {
      this.jodaDateTime = jodaDateTime;
      this.javaUtilDate = javaUtilDate;
    }
  }

  @Test
  public void iso8601ThroughJoda() throws ParseException, IOException {
    ObjectMapper mapper = new ObjectMapper();
    mapper.configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false);
    mapper.registerModule(new JodaModule());
    mapper.configure(SerializationFeature.WRITE_DATES_AS_TIMESTAMPS, false);

    DateFormat df = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss zzz");
    Date targetDate = df.parse("2000-01-01 00:00:00 UTC");
    Beanie expectedBean = new Beanie(new DateTime(targetDate.getTime()), targetDate);

    String expectedJSON =
        "{\"jodaDateTime\":\"2000-01-01T00:00:00.000Z\","
            + "\"javaUtilDate\":\"2000-01-01T00:00:00.000+0000\"}";
    String actualJSON = mapper.writeValueAsString(expectedBean);
    assertEquals(actualJSON, expectedJSON);

    Beanie actualBean = mapper.readValue(actualJSON, Beanie.class);
    assertEquals(actualBean.javaUtilDate, expectedBean.javaUtilDate);
    assertEquals(actualBean.jodaDateTime.getMillis(), expectedBean.jodaDateTime.getMillis());

    /*
     * Why doesn't the JodaModule deserialize a DateTime that is equal to the one it serialized in
     * the first place? I expected these two to be equal but they aren't.
     */
    assertEquals(actualBean.jodaDateTime, expectedBean.jodaDateTime);
  }
}

There is a git repository supported by a gradle build that demonstrates the issue at https://github.com/robertkuhar/jacksonjodawtf

[cowtowncoder]

After looking into this, I don't think this is a bug but expected behavior. The underlying timestamp is equal, but the timezone differs. This because test is constructing DateTime using the local timezone; but Joda serialization uses GMT. As such timezone offset used is different, and deserializer uses that timezone.

If identical representation is desired, one way to do this is to use @JsonFormat annotation on property, to hardcode timezone to use. Another possibility is to use ObjectMapper.setTimeZone() from default of GMT to local timezone, although it is not clear whether that setting is currently handler for serialization (it does seem supported for deserialization).

[tommack]

I just ran into this. With all of the work to get WRITE_DATES_WITH_ZONE_ID working, I was surprised to see this basic thing so broken. The whole point of a DateTime is that it is both a moment in time and a timezone. If WRITE_DATES_AS_TIMESTAMPS is off, this makes no difference. But when WRITE_DATES_AS_TIMESTAMPS is on, changing the timezone in the ISO 8601 string is just wrong.

This is particularly apparent when using WRITE_DATES_WITH_ZONE_ID. Now we have Jackson outputting strings like "2015-09-12T01:40:09.707Z[America/Chicago]" which makes no sense at all. The jackson-datatype-jsr310 library gets this right and will output "2015-09-11T20:42:52.295-05:00[America/Chicago]" for a ZonedDateTime (rough equivalent to Joda's DateTime).

[cowtowncoder]

As usual, pull requests welcome. Even if just for failing test case. I am going over accumulated bugs remaining in 2 .6.1 so this could be fixed for 2.6.2.

[wattazoum]

Hi, I also ran into this issue when migrating from Jackson 1.x to Jackson 2.x. I have some unit testing failure because jackson-joda outputs in UTC (if you want to keep the right value of the DateTime). When activating WRITE_DATES_WITH_ZONE_ID then the value is not at all the same.

[wattazoum]

I just noticed that your test is wrong. The problem resides in the creation of the DateTime you are using for the test. See bellow the right test.

    @Test
    public void iso8601ThroughJoda() throws IOException, ParseException {
        String source = "2000-01-01 00:00:00 UTC";

        ObjectMapper mapper = getObjectMapper();

        DateFormat df = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss zzz");
        Date targetDate = df.parse(source);
        Beanie expectedBean = new Beanie(new DateTime(targetDate.getTime(), DateTimeZone.UTC), targetDate);

        String expectedJSON =
                "{\"jodaDateTime\":\"2000-01-01T00:00:00.000Z\","
                        + "\"javaUtilDate\":\"2000-01-01T00:00:00.000+0000\"}";

        Beanie actualBean = mapper.readValue(expectedJSON, Beanie.class);
        assertEquals(expectedBean.javaUtilDate, actualBean.javaUtilDate);
        assertEquals(expectedBean.jodaDateTime.getMillis(), actualBean.jodaDateTime.getMillis());

        assertEquals(expectedBean.jodaDateTime, actualBean.jodaDateTime);
    }

However, this works only because your test data is in UTC. Here is a failing one that illustrates your problem more properly :

    @Test
    public void iso8601ThroughJodaCEST() throws IOException, ParseException {
        String source = "2000-01-01 01:00:00 CET";

        ObjectMapper mapper = getObjectMapper();

        DateFormat df = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss zzz");
        Date targetDate = df.parse(source);
        Beanie expectedBean = new Beanie(new DateTime(targetDate.getTime(), DateTimeZone.forID("CET")), targetDate);

        String expectedJSON =
                "{\"jodaDateTime\":\"2000-01-01T01:00:00.000+01:00\","
                        + "\"javaUtilDate\":\"2000-01-01T00:00:00.000+0000\"}";

        Beanie actualBean = mapper.readValue(expectedJSON, Beanie.class);
        assertEquals(expectedBean.javaUtilDate, actualBean.javaUtilDate);
        assertEquals(expectedBean.jodaDateTime.getMillis(), actualBean.jodaDateTime.getMillis());

    /*
     * Why doesn't the JodaModule deserialize a DateTime that is equal to the one it serialized in
     * the first place? I expected these two to be equal but they aren't.
     */
        assertEquals(expectedBean.jodaDateTime, actualBean.jodaDateTime);
    }

What I noticed is that the issue is in the Deserialisation in this case. Jackson set the created DateTime in UTC (there is no reason not to set the right TZ in the DateTime, the question seemed to be an issue regarding a comment in the code of DateTimeDeserializer

Not sure if it should use timezone or not... 15-Sep-2015, tatu: impl of 'createParser()' SHOULD handle all timezone/locale setup

I don't agree with Tatu, one will expect Jackson to read the full value of the information fed to it. TZ is an important information that we want to keep.

44 and #43 will be interested in this.

[wattazoum]

I didn't fix the unit tests because that means changing the default behavior of that plugin. Hope we can discuss it. Please have a look at my branch and see the failing tests. So the workaround for usage in my project will be the following :

        ObjectMapper mapper = new ObjectMapper();
        mapper.configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false);
        mapper.registerModule(new JodaModule()
                .addSerializer(DateTime.class, new MyDateTimeSerializer())
                .addDeserializer(DateTime.class, MyDateTimeDeserializer.forType(DateTime.class) ));
        mapper.configure(SerializationFeature.WRITE_DATES_AS_TIMESTAMPS, false);

where MyDateTimeXXXX are subclasses of the original ones and override the (de)serialize method with my fix.

[cowtowncoder]

@wattazoum Please bring these up on mailing lists. I am not an expert on date/time handling, so you better convince those members who are, like @tommack @beamerblvd.

As to keeping TZ, handling, there have been extensive discussions back and forth, including cross-module behavior (core databind, Java 8 date/time, Joda) and settings. There are no easy fixes, and changes that affect backwards compatibility are tricky. At this point there is not much value in Jackson 1.x -> 2.x compatibility, however, compared to compatibility between 2.x minor versions.

[wattazoum]

I didn't find the mailing list but I used the forum to start the discussion

[cowtowncoder]

@wattazoum Appreciate your effort, but unfortunately I do not think many of developers participate in that forum (I do try to follow up myself). jackson-dev google group is the main forum.

Mailing group is at:

https://groups.google.com/forum/#!forum/jackson-dev

[cowtowncoder]

@wattazoum thank you for sending the question on the dev mailing list. This would be good time to get things started for changes (if any) for 2.8. With 2.7 patch releases we can fix bugs, but need to be cautious on otherwise changing behavior (definition of bug/flaw, and other sub-optimal behaviors may be arbitrary of course but still).

[j-renkel]

We've run into a similar, if not the same, problem when attempting to use the latest version (2.8.6) with our application.

Our application makes extensive use of new DateTime() (which produces a DateTime instance in the local timezone) and needs these to be equal, using DateTime.equals(), to a DateTime produced by JodaModule.

Unfortunately, JodaModule by default produces a DateTime with the UTC timezone.

Our attempt to fix this is:

        JodaModule jodaModule = new JodaModule();
        JacksonJodaDateFormat jacksonJodaDateFormat = new JacksonJodaDateFormat(FormatConfig.DEFAULT_DATETIME_PARSER, TimeZone.getDefault());
        DateTimeDeserializer deserializer = new DateTimeDeserializer(DateTime.class, jacksonJodaDateFormat);
        jodaModule.addDeserializer(DateTime.class, deserializer);
        ObjectMapper objectMapper = new ObjectMapper();
        objectMapper.registerModule(jodaModule);

Unfortunately, the jodaModule.addDeserializer() call gets the following compiler error:

The method addDeserializer(Class<T>, JsonDeserializer<? extends T>) in the type SimpleModule is not applicable for the arguments (Class<DateTime>, DateTimeDeserializer)

This is because the declaration of DateTimeDeserializer is:

public class DateTimeDeserializer extends JodaDateDeserializerBase<ReadableInstant>

Shouldn't this be:

public class DateTimeDeserializer extends JodaDateDeserializerBase<DateTime>

We made this change in a private copy of DateTimeDeserializer and this change fixes the compilation error and seems to work (Several other small changes were necessary because of the change in the type parameter.).

Is this safe / correct, or are we missing something?

Thanks in advance for your attention to this.

Jim Renkel AMI Entertainment Network, Inc. LLC

[wattazoum]

This is because the declaration of DateTimeDeserializer is: public class DateTimeDeserializer extends JodaDateDeserializerBase<ReadableInstant> Shouldn't this be: public class DateTimeDeserializer extends JodaDateDeserializerBase<DateTime>

I think you are right. Looks like a bug.

[j-renkel]

Will this be fixed? Or should we submit our fix?

We use a lot of open source code, but I don't think we've yet contributed to any.

This may be a good opportunity for us to start doing that, although we will prolly need some coaching.

Do you know of a good, step-by-step tutorial on how to do this?

Thanks,

Jim

[cowtowncoder]

No changes should be needed; although type assignment may not be optimal there is a better way to instantiate deserializer, used by JodaModule itself:

addDeserializer(DateTime.class,
    DateTimeDeserializer.forType(DateTime.class));
addDeserializer(ReadableDateTime.class, 
    DateTimeDeserializer.forType(ReadableDateTime.class));
addDeserializer(ReadableInstant.class,
    DateTimeDeserializer.forType(ReadableInstant.class));