search

IBM / trino-db2

Db2 JDBC connector for Trino
Apache License 2.0
18 stars 30 forks source link

java.lang.IllegalArgumentException: Precision is out of range: 10 #70

Closed shawnzhu closed 3 years ago

shawnzhu commented 3 years ago

When querying a Db2 table with column contains type timestamp(10), I got:

Query 20210413_003738_00007_j43yx failed: com.google.common.util.concurrent.UncheckedExecutionException: java.lang.IllegalArgumentException: Precision is out of range: 10

The stack trace from server log:

Apr 12 20:32:50 prestosql-coordinator-bb478d9ff-5hxj5 presto-coordinator ERROR remote-task-callback-137 io.prestosql.execution.StageStateMachine    Stage 20210413_003248_00006_j43yx.2 failed
com.google.common.util.concurrent.UncheckedExecutionException: com.google.common.util.concurrent.UncheckedExecutionException: java.lang.IllegalArgumentException: Precision is out of range: 10
    at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2051)
    at com.google.common.cache.LocalCache.get(LocalCache.java:3951)
    at com.google.common.cache.LocalCache$LocalManualCache.get(LocalCache.java:4871)
    at io.prestosql.plugin.jdbc.CachingJdbcClient.get(CachingJdbcClient.java:535)
    at io.prestosql.plugin.jdbc.CachingJdbcClient.getColumns(CachingJdbcClient.java:134)
    at io.prestosql.plugin.jdbc.JdbcMetadata.getTableMetadata(JdbcMetadata.java:341)
    at io.prestosql.plugin.jdbc.JdbcMetadata.lambda$listTableColumns$5(JdbcMetadata.java:370)
    at java.base/java.util.Optional.ifPresent(Optional.java:183)
    at io.prestosql.plugin.jdbc.JdbcMetadata.listTableColumns(JdbcMetadata.java:370)
    at io.prestosql.metadata.MetadataManager.listTableColumns(MetadataManager.java:600)
    at io.prestosql.metadata.MetadataListing.listTableColumns(MetadataListing.java:135)
    at io.prestosql.connector.informationschema.InformationSchemaPageSource.addColumnsRecords(InformationSchemaPageSource.java:251)
    at io.prestosql.connector.informationschema.InformationSchemaPageSource.buildPages(InformationSchemaPageSource.java:216)
    at io.prestosql.connector.informationschema.InformationSchemaPageSource.getNextPage(InformationSchemaPageSource.java:183)
    at io.prestosql.operator.ScanFilterAndProjectOperator$ConnectorPageSourceToPages.process(ScanFilterAndProjectOperator.java:376)
    at io.prestosql.operator.WorkProcessorUtils$ProcessWorkProcessor.process(WorkProcessorUtils.java:372)
    at io.prestosql.operator.WorkProcessorUtils.getNextState(WorkProcessorUtils.java:221)
    at io.prestosql.operator.WorkProcessorUtils$YieldingProcess.process(WorkProcessorUtils.java:181)
    at io.prestosql.operator.WorkProcessorUtils$ProcessWorkProcessor.process(WorkProcessorUtils.java:372)
    at io.prestosql.operator.WorkProcessorUtils$3.process(WorkProcessorUtils.java:306)
    at io.prestosql.operator.WorkProcessorUtils$ProcessWorkProcessor.process(WorkProcessorUtils.java:372)
    at io.prestosql.operator.WorkProcessorUtils$3.process(WorkProcessorUtils.java:306)
    at io.prestosql.operator.WorkProcessorUtils$ProcessWorkProcessor.process(WorkProcessorUtils.java:372)
    at io.prestosql.operator.WorkProcessorUtils$3.process(WorkProcessorUtils.java:306)
    at io.prestosql.operator.WorkProcessorUtils$ProcessWorkProcessor.process(WorkProcessorUtils.java:372)
    at io.prestosql.operator.WorkProcessorUtils.getNextState(WorkProcessorUtils.java:221)
    at io.prestosql.operator.WorkProcessorUtils.lambda$processStateMonitor$2(WorkProcessorUtils.java:200)
    at io.prestosql.operator.WorkProcessorUtils$ProcessWorkProcessor.process(WorkProcessorUtils.java:372)
    at io.prestosql.operator.WorkProcessorUtils.lambda$flatten$6(WorkProcessorUtils.java:277)
    at io.prestosql.operator.WorkProcessorUtils$3.process(WorkProcessorUtils.java:319)
    at io.prestosql.operator.WorkProcessorUtils$ProcessWorkProcessor.process(WorkProcessorUtils.java:372)
    at io.prestosql.operator.WorkProcessorUtils$3.process(WorkProcessorUtils.java:306)
    at io.prestosql.operator.WorkProcessorUtils$ProcessWorkProcessor.process(WorkProcessorUtils.java:372)
    at io.prestosql.operator.WorkProcessorUtils.getNextState(WorkProcessorUtils.java:221)
    at io.prestosql.operator.WorkProcessorUtils.lambda$processStateMonitor$2(WorkProcessorUtils.java:200)
    at io.prestosql.operator.WorkProcessorUtils$ProcessWorkProcessor.process(WorkProcessorUtils.java:372)
    at io.prestosql.operator.WorkProcessorUtils.getNextState(WorkProcessorUtils.java:221)
    at io.prestosql.operator.WorkProcessorUtils.lambda$finishWhen$3(WorkProcessorUtils.java:215)
    at io.prestosql.operator.WorkProcessorUtils$ProcessWorkProcessor.process(WorkProcessorUtils.java:372)
    at io.prestosql.operator.WorkProcessorSourceOperatorAdapter.getOutput(WorkProcessorSourceOperatorAdapter.java:149)
    at io.prestosql.operator.Driver.processInternal(Driver.java:379)
    at io.prestosql.operator.Driver.lambda$processFor$8(Driver.java:283)
    at io.prestosql.operator.Driver.tryWithLock(Driver.java:675)
    at io.prestosql.operator.Driver.processFor(Driver.java:276)
    at io.prestosql.execution.SqlTaskExecution$DriverSplitRunner.processFor(SqlTaskExecution.java:1076)
    at io.prestosql.execution.executor.PrioritizedSplitRunner.process(PrioritizedSplitRunner.java:163)
    at io.prestosql.execution.executor.TaskExecutor$TaskRunner.run(TaskExecutor.java:484)
    at io.prestosql.$gen.Presto_348____20210412_143813_2.run(Unknown Source)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
    at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: com.google.common.util.concurrent.UncheckedExecutionException: java.lang.IllegalArgumentException: Precision is out of range: 10
    at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2051)
    at com.google.common.cache.LocalCache.get(LocalCache.java:3951)
    at com.google.common.cache.LocalCache$LocalManualCache.get(LocalCache.java:4871)
    at io.prestosql.plugin.jdbc.CachingJdbcClient.get(CachingJdbcClient.java:535)
    at io.prestosql.plugin.jdbc.CachingJdbcClient.getColumns(CachingJdbcClient.java:134)
    at io.prestosql.plugin.jdbc.CachingJdbcClient.lambda$getColumns$3(CachingJdbcClient.java:134)
    at com.google.common.cache.LocalCache$LocalManualCache$1.load(LocalCache.java:4876)
    at com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3529)
    at com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2278)
    at com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2155)
    at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2045)
    ... 50 more
Caused by: java.lang.IllegalArgumentException: Precision is out of range: 10
    at com.google.common.base.Preconditions.checkArgument(Preconditions.java:190)
    at io.prestosql.plugin.jdbc.StandardColumnMappings.timestampColumnMappingUsingSqlTimestamp(StandardColumnMappings.java:442)
    at io.prestosql.plugin.db2.DB2Client.toPrestoType(DB2Client.java:98)
    at io.prestosql.plugin.jdbc.BaseJdbcClient.getColumns(BaseJdbcClient.java:290)
    at io.prestosql.plugin.jdbc.ForwardingJdbcClient.getColumns(ForwardingJdbcClient.java:85)
    at io.prestosql.plugin.jdbc.jmx.StatisticsAwareJdbcClient.lambda$getColumns$4(StatisticsAwareJdbcClient.java:101)
    at io.prestosql.plugin.jdbc.jmx.JdbcApiStats.wrap(JdbcApiStats.java:35)
    at io.prestosql.plugin.jdbc.jmx.StatisticsAwareJdbcClient.getColumns(StatisticsAwareJdbcClient.java:101)
    at io.prestosql.plugin.jdbc.CachingJdbcClient.lambda$getColumns$3(CachingJdbcClient.java:134)
    at com.google.common.cache.LocalCache$LocalManualCache$1.load(LocalCache.java:4876)
    at com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3529)
    at com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2278)
    at com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2155)
    at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2045)
    ... 60 more

Which points to this line of code:

https://github.com/IBM/trino-db2/blob/a4953ceddb58c019b19f4dcb71a3ff36131a0882/src/main/java/io/trino/plugin/db2/DB2Client.java#L98

shawnzhu commented 3 years ago

I can reproduce it locally by using a column with type TIMESTAMP(10).

the very problem is this line of code:

https://github.com/trinodb/trino/blob/b94026852e7aa253a34a354b42cf958d4bd0794c/plugin/trino-base-jdbc/src/main/java/io/trino/plugin/jdbc/StandardColumnMappings.java#L489

where it will throw the :point_up: IllegalArgumentException like Precision is out of range: 10 if the precision is greater than 9.

Summary

it only supports nanosecond precision (a.k.a., TIMESTAMP(9)) out of #54.

When source table contains column with type TIMESTAMP(12) which means picoseconds, this issue will occur.

Proposed fix

It should suggest using TIMESTAMP(9) or with lower precision since nanosecond is the max precision date time function can achieve with JDK 11. see java.time.LocalDateTime So as the max precision that Trino can support as TIMESTAMP types.

shawnzhu commented 3 years ago

Followed up here to clarify timestamp precision support: https://trinodb.slack.com/archives/CP1MUNEUX/p1618366896274800

shawnzhu commented 3 years ago

PR #71 will fix the code of this connector, the actual fix will depend on https://github.com/trinodb/trino/issues/7590

Results

With #71, it will support precision up to 8 with https://github.com/trinodb/trino/issues/7591 it will support precision up to 9