dacort / metabase-athena-driver

An Amazon Athena driver for Metabase 0.32 and later
Apache License 2.0
225 stars 32 forks source link

Metabase v0.34 is not displaying date/datetime fields properly #27

Closed dacort closed 4 years ago

dacort commented 4 years ago

As noted in metabase/metabase#11607.

Will try to take a look at both this and #26 within the next few days.

dacort commented 4 years ago

Looking at Presto date/time types, we can see that there are a two that fail.

SELECT 
    DATE '2001-08-22' as type_date,
    TIME '01:02:03.456' as type_time,
    TIME '01:02:03.456 America/Los_Angeles' AS type_time_with_timezone,
    TIMESTAMP '2001-08-22 03:04:05.321' AS type_timestamp,
    TIMESTAMP '2001-08-22 03:04:05.321 America/Los_Angeles' AS type_timestamp_with_timezone,
    INTERVAL '3' MONTH AS type_interval_year_to_month,
    INTERVAL '2' DAY AS type_interval_day_to_second

type_date and type_timestamp return empty values.

image

In the console logs, we see a couple corresponding error messages:

For column 1 (DATE data type):

01-03 22:41:20 ERROR sql-jdbc.execute :: Error reading :athena column 1 type_date DATE
java.sql.SQLNonTransientException: [Simba][JDBC](11040) Invalid type for data - parameter: 1, type: java.time.LocalDate.
    at com.simba.athena.exceptions.ExceptionConverter.toSQLException(Unknown Source)
    at com.simba.athena.jdbc.jdbc41.utilities.ResultSetUtilities.getObjectByType(Unknown Source)
    at com.simba.athena.jdbc.jdbc41.S41ForwardResultSet.getObject(Unknown Source)
    at com.mchange.v2.c3p0.impl.NewProxyResultSet.getObject(NewProxyResultSet.java:217)
    at metabase.driver.sql_jdbc.execute$get_object_of_class.invokeStatic(execute.clj:66)

For column 4 (TIMESTAMP data type):

01-03 22:41:20 ERROR sql-jdbc.execute :: Error reading :athena column 4 type_timestamp TIMESTAMP
java.sql.SQLNonTransientException: [Simba][JDBC](11040) Invalid type for data - parameter: 4, type: java.time.LocalDateTime.
    at com.simba.athena.exceptions.ExceptionConverter.toSQLException(Unknown Source)
    at com.simba.athena.jdbc.jdbc41.utilities.ResultSetUtilities.getObjectByType(Unknown Source)
    at com.simba.athena.jdbc.jdbc41.S41ForwardResultSet.getObject(Unknown Source)
    at com.mchange.v2.c3p0.impl.NewProxyResultSet.getObject(NewProxyResultSet.java:217)
    at metabase.driver.sql_jdbc.execute$get_object_of_class.invokeStatic(execute.clj:66)
dacort commented 4 years ago

It seems like the Athena JDBC driver doesn't support the java.time.LocalDate or java.time.LocalDateTime types.

It looks like we can switch back to the legacy implementation, but I want to confirm what the differences are.

This might be a good opportunity to write some basic tests that run directly against Athena. I will spend some time on #12.

dacort commented 4 years ago

Previous version: image

New version with "legacy" fix: image

Looks like we get the same results. I don't like how the "date" format includes the time, but maybe that's to be expected.

flamber commented 4 years ago

I don't like how the "date" format includes the time, but maybe that's to be expected.

That's an oldie Metabase issue - see https://github.com/metabase/metabase/issues/5859 or https://github.com/metabase/metabase/issues/7494, or my catch-all issue that goes through the various date formatting issues and workarounds https://github.com/metabase/metabase/issues/11394

camsaul commented 4 years ago

There's a shared implementation you can use for drivers that don't support the new Java 8 types. Just add to your call to driver/register! – see the Redshift driver for example.

(driver/register! :presto, :parent #{:sql-jdbc ::legacy/use-legacy-classes-for-read-and-set})

Implementation is in this namespace.