microseconds in date

[crotwell]

Not sure if this is a problem, but I think dates are millisecond precision in javascript, and so saying that the fdsn-date-time is format: date-time and that the format is YYYY-MM-DDThh:mm:ss[.SSSSSS] might be a problem as a default parser in javascript and likely java and other languages will only be able to handle YYYY-MM-DDThh:mm:ss[.SSS]

It might be better to define the fdsn-date-time in a way that uses a millisecond ISO 8601 format date plus an extarnally defined 0-999 microseconds.

[CTrabant]

The JSON schema says it needs to be a "profile" of ISO 8601 defined here: https://tools.ietf.org/html/draft-fge-json-schema-validation-00#section-7.3.1. According to Wikipedia (ISO 8601/RFC3339) there is no limit on the number of fractional decimals. I did run into a problem with one of the online validators but I think that's a validator problem.

I do not think we should alter the schema to try and accommodate JavaScript or any other lowest common denominator of limitations. I'm guessing most will work fine anyway, willing to reconsider with if some testing indicates a problem would be widespread. Splitting the time up should be a last-ditch option in my opinion, that's just forcing terribness on all future parsing.

[crotwell]

I think I pretty much agree, it can be a bit of a pain, but it is just a string so languages that only do milliseconds can pull off the microseconds if needed and parse it. I did some looking and python does microseconds now, javascript 5 will do microseconds. Java SimpleDateFormatt is just unfortunately milliseconds, so a mixed bag.

One further clarification, are dates required to pad microseconds with zeros? In other words always have 0 or 6 digits after the decimal, 2001-07-04T12:08:56.235 vs 2001-07-04T12:08:56.235000?

[CTrabant]

Does the Java SimpleDateFormat fail in any way or are the microseconds simply left behind? If the latter then I don't think that's a problem for the schema to solve.

I do not think fractional seconds need to be padded out to microseconds. I know of no parsing systems that need that and it's usually a non-problem.

[crotwell]

So I was wrong, for Java it can handle the micros (they are lost can only hold millis), but needs the padding. Weird.

    String isoMilli = "2012-10-21T23:59:50.123Z";
    String isoMicro = "2012-10-21T23:59:50.123456Z";
    String isoTenthMilli = "2012-10-21T23:59:50.1234Z";
    SimpleDateFormat fMilli = new SimpleDateFormat("yyyy-MM-dd'T'HH:mm:ss.SSSX");
    SimpleDateFormat fMicro = new SimpleDateFormat("yyyy-MM-dd'T'HH:mm:ss.SSSSSSX");
    // this works
    Date dateMilli = fMilli.parse(isoMilli);

    // this fails with java.text.ParseException: Unparseable date: 
    Date dateMicro = fMilli.parse(isoMicro);

    // this works
    Date dateMicroB = fMicro.parse(isoMicro);

    // this fails with java.text.ParseException: Unparseable date: 
    Date dateMilliB = fMicro.parse(isoMilli);

    // this fails with java.text.ParseException: Unparseable date: 
    Date dateTenthMilliB = fMicro.parse(isoTenthMilli);

[crotwell]

And this fails to parse as well:

    String isoTenth = "2012-10-21T23:59:50.1Z";
    SimpleDateFormat fMilli = new SimpleDateFormat("yyyy-MM-dd'T'HH:mm:ss.SSSX");
    // this fails with java.text.ParseException: Unparseable date: 
    Date dateTenth = fMilli.parse(isoTenth);

[CTrabant]

Wow, SimpleDateFormat parsing sucks in this regard. You need to explicitly know how many decimals of fractional seconds a priori? No wonder folks moved to Joda-Time and are excited about Java 8's improved time handling, it must be better than this.

[crotwell]

Yep, java time handling is, well, an interesting example of software design...

On Tue, Aug 4, 2015 at 5:17 PM, CTrabant notifications@github.com wrote:

Wow, SimpleDateFormat parsing sucks in this regard. You need to explicitly know how many decimals of fractional seconds a priori? No wonder folks moved to Joda-Time and are excited about Java 8's improved time handling, it must be better than this.

— Reply to this email directly or view it on GitHub https://github.com/CTrabant/StationJSON/issues/5#issuecomment-127764584.

[crotwell]

Just for future reference, here is a java class I wrote way back before the invention of the wheel to deal with java's data parsing, um, oddities. Note the cleanDate method in particular. It is more complicated due to the multiple iso formats supported, but does have the string manipulation steps to get the fractional seconds of the right length so SimpleDateFormat can do the parsing.

https://github.com/crotwell/fissuresImpl/blob/master/src/main/java/edu/iris/Fissures/model/ISOTime.java