search

chequer-io / JDBC.NET

It is a wrapper that allows you to use JDBC drivers in ADO.NET
MIT License
33 stars 11 forks source link

Azure Databricks JDBC + Driver #41

Open Nerus87 opened 3 months ago

Nerus87 commented 3 months ago

Azure cloud Driver: DatabricksJDBC42.jar Download: https://www.databricks.com/spark/jdbc-drivers-download Guide: https://docs.databricks.com/en/_extras/documents/Databricks-JDBC-Driver-Install-and-Configuration-Guide.pdf

Hello, I have a problem fetching data, DB can be changed and table columns are shown but all the data are missing. I've tried even with Dapper but it is not working.

Nerus87 commented 3 months ago

Additional info, logs from connection:

Mar 21 10:07:43.349 WARN  31 com.databricks.client.hivecommon.dataengine.BackgroundFetcher.run: Caught Throwable in BackgroundFetcher.run() while the thread is running: null
Mar 21 10:07:43.354 ERROR 27 com.databricks.client.exceptions.ExceptionConverter.toSQLException: [Databricks][JDBCDriver](500540) Error caught in BackgroundFetcher. Foreground thread ID: 27. Background thread ID: 31. Error caught: null.
java.sql.SQLException: [Databricks][JDBCDriver](500540) Error caught in BackgroundFetcher. Foreground thread ID: 27. Background thread ID: 31. Error caught: null.
    at com.databricks.client.hivecommon.dataengine.BackgroundFetcher.run(Unknown Source)
    at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
    at java.base/java.util.concurrent.FutureTask.run(Unknown Source)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
Caused by: com.databricks.client.support.exceptions.GeneralException: [Databricks][JDBCDriver](500540) Error caught in BackgroundFetcher. Foreground thread ID: 27. Background thread ID: 31. Error caught: null.
    ... 5 more
Caused by: java.lang.ExceptionInInitializerError
    at com.databricks.client.jdbc42.internal.apache.arrow.memory.ArrowBuf.getDirectBuffer(ArrowBuf.java:229)
    at com.databricks.client.jdbc42.internal.apache.arrow.memory.ArrowBuf.nioBuffer(ArrowBuf.java:224)
    at com.databricks.client.jdbc42.internal.apache.arrow.vector.ipc.ReadChannel.readFully(ReadChannel.java:87)
    at com.databricks.client.jdbc42.internal.apache.arrow.vector.ipc.message.MessageSerializer.readMessageBody(MessageSerializer.java:728)
    at com.databricks.client.jdbc42.internal.apache.arrow.vector.ipc.message.MessageChannelReader.readNext(MessageChannelReader.java:67)
    at com.databricks.client.jdbc42.internal.apache.arrow.vector.ipc.ArrowStreamReader.loadNextBatch(ArrowStreamReader.java:145)
    at com.databricks.client.spark.arrow.ArrowBuffer.handleInitializeBuffer(Unknown Source)
    at com.databricks.client.hivecommon.api.HiveServer2BaseBuffer.initializeBuffer(Unknown Source)
    at com.databricks.client.spark.jdbc.ResultFileDownloadManager.getNextDownloadedFile(Unknown Source)
    at com.databricks.client.spark.jdbc.DownloadableFetchClient.fetchNRows(Unknown Source)
    at com.databricks.client.hivecommon.api.HS2Client.fetchRows(Unknown Source)
    at com.databricks.client.hivecommon.dataengine.BackgroundFetcher.run(Unknown Source)
    at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
    at java.base/java.util.concurrent.FutureTask.run(Unknown Source)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
    at java.base/java.lang.Thread.run(Unknown Source)
Caused by: java.lang.RuntimeException: Failed to initialize MemoryUtil.
    at com.databricks.client.jdbc42.internal.apache.arrow.memory.util.MemoryUtil.<clinit>(MemoryUtil.java:136)
    ... 17 more
Caused by: java.lang.reflect.InaccessibleObjectException: Unable to make field long java.nio.Buffer.address accessible: module java.base does not "opens java.nio" to unnamed module @1c655221
    at java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(Unknown Source)
    at java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(Unknown Source)
    at java.base/java.lang.reflect.Field.checkCanSetAccessible(Unknown Source)
    at java.base/java.lang.reflect.Field.setAccessible(Unknown Source)
    at com.databricks.client.jdbc42.internal.apache.arrow.memory.util.MemoryUtil.<clinit>(MemoryUtil.java:84)
    ... 17 more

Mar 21 10:07:43.392 WARN  27 com.databricks.client.hivecommon.utils.HeartBeatThread.removeOperationHandle: THandleIdentifier(guid:57 FB 64 B6 55 22 48 92 BF 3B 49 A6 7A D7 70 58, secret:20 7C 01 CB 74 64 42 24 99 E4 C5 1E 00 60 30 E6) is not in the heartbeat opeartion handle list
Mar 21 10:07:43.392 WARN  27 com.databricks.client.hivecommon.utils.HeartBeatThread.removeOperationHandle: THandleIdentifier(guid:57 FB 64 B6 55 22 48 92 BF 3B 49 A6 7A D7 70 58, secret:20 7C 01 CB 74 64 42 24 99 E4 C5 1E 00 60 30 E6) is not in the heartbeat opeartion handle list
idunnoaiii commented 1 month ago

I'm try clone the source and add this --add-opens java.base/java.nio=ALL_UNNAMED in the JdbcBridge before execute command java and it dose working. Waiting the author add this config to the java package.

image