cognitect-labs / aws-api

AWS, data driven
Apache License 2.0
724 stars 100 forks source link

Failures faithfully parsing ISO8601 timestamps with microsecond precision #244

Open grzm opened 8 months ago

grzm commented 8 months ago

Dependencies

% git log -n1
commit 5900e359365041ed9380947d010b0f0a853e793c (HEAD -> main, origin/main)
Author: David Chelimsky <dchelimsky@gmail.com>
Date:   Wed Jul 12 10:04:32 2023

    pass around shapes instead of entire service (#43)

Description with failing test case

In cognitect.client.impl-test/test-invoke-happy-path there's a discrepancy in the CreationDate timestamps provided by list-buckets-http-response and list-buckets-aws-client-response

CreationDate value
list-buckets-http-response "2023-01-23T11:59:03.575496Z"
list-buckets-aws-client-response "2023-01-23T12:08:38.496-00:00"

https://github.com/cognitect-labs/aws-api/blob/main/test/src/cognitect/client/impl_test.clj#L56-L100

% clj
Clojure 1.11.1
user=> (require '[cognitect.aws.util :as u])
nil
user=> (u/parse-date u/iso8601-msecs-date-format "2023-01-23T11:59:03.575496Z")
#inst "2023-01-23T12:08:38.496-00:00"

This looks like it's due to precision limitations in java.text.SimpleDateFormat.

While I haven't found it in official documentation, java.text.SimpleDateFormat can only support millisecond precision, and it looks like it fails somewhat spectacularly with microsecond precision. It looks like what's happening is that it's considering the value to be #inst "2023-01-23T11:59:03" + 575496 millis:

% clj -M -e '(.plus (.toInstant #inst "2023-01-23T11:59:03Z") (java.time.Duration/ofMillis 575496))'
#object[java.time.Instant 0x11ee02f8 "2023-01-23T12:08:38.496Z"]

https://stackoverflow.com/questions/30135025/java-date-parsing-with-microsecond-or-nanosecond-accuracy

Given this is an explicit example in the tests, it wouldn't surprise me if this test is there to explicitly document this behavior. That said, I don't see a comment in the source or the commit when this test was added that indicates this is in fact the case.

Is this known, accepted (if infelicitous) behavior?

While java.time.format.DateTimeFormatter can handle this, I believe aws-api is intended to still support pre-JDK8 platforms, so this might not be available. (awyeah-api doesn't have this particular platform limitation and I worked around this issue here: https://github.com/grzm/awyeah-api/commit/2ac9da454e1fa482c3f76cac28b557eb831e2075)