search

ClickHouse / clickhouse-jdbc-bridge

A JDBC proxy from ClickHouse to external databases
Apache License 2.0
168 stars 60 forks source link

How to use JAAS config with jdbc bridge #126

Closed gfunc closed 2 years ago

gfunc commented 2 years ago

background:

I am trying to configure ClickHouse to query Impala via jdbc-bridge our Impala implementation was authenticated via Kerberos and thus require a keytab for connection

this is what I did:

  1. create a JAAS configuration file as per document page 16 
    Client {
com.sun.security.auth.module.Krb5LoginModule required
useKeyTab=true
keyTab="/var/opt/clickhouse/clickhouse.keytab"
principal="clickhouse/SOME_HOST@SOME_DOMAIN"
doNotPrompt=true;
};

  2. change set-env.sh to this:

    #!/bin/bash

JVM_ARGS="-Xms5120m -Xmx10240m -Djava.security.auth.login.config=/etc/clickhouse-jdbc-bridge/krb5Config.ini - 
Djava.security.krb5.conf=/etc/krb5.conf"
  3. configure impala.json file under /etc/clickhouse-jdbc-bridge/config/datasources: 
    {
"$schema": "../datasource.jschema",
"impala": {
    "driverUrls": [
        "/path/to/jars/ClouderaImpalaJDBC-2.6.24.1029/ImpalaJDBC42.jar"
    ],
    "driverClassName": "com.cloudera.impala.jdbc.Driver",
    "jdbcUrl": "jdbc:impala://IMPALA_HOST:21050;AuthMech=1;AutoReconnect=1;KrbRealm=SOME_DOMAIN;KrbHostFQDN=IMPALA_HOST;Host=IMPALA_HOST;KrbServiceName=impala;QueryRetryInterval=5s",
    "timezone": "UTC",
    "initializationFailTimeout": 0,
    "minimumIdle": 0,
    "maximumPoolSize": 10
    }
}
  4. select * from jdbc('','show datasources') is showing impala without a problem

error:

the connection is not working and showing below error,

[vert.x-eventloop-thread-0] [ERROR] {JdbcBridgeVerticle:318} - Failed to respond
java.lang.IllegalStateException: Failed to infer schema from [impala] due to: Failed to access [impala] due to: impala - Connection is not available, request timed out after 30000ms.
        at ru.yandex.clickhouse.jdbcbridge.core.NamedDataSource.getResultColumns(NamedDataSource.java:456)
        at ru.yandex.clickhouse.jdbcbridge.JdbcBridgeVerticle.handleColumnsInfo(JdbcBridgeVerticle.java:390)
        at ru.yandex.clickhouse.jdbcbridge.internal.vertx.ext.web.impl.RouteState.handleContext(RouteState.java:1038)
        at ru.yandex.clickhouse.jdbcbridge.internal.vertx.ext.web.impl.RoutingContextImplBase.iterateNext(RoutingContextImplBase.java:101)
        at ru.yandex.clickhouse.jdbcbridge.internal.vertx.ext.web.impl.RoutingContextImpl.next(RoutingContextImpl.java:132)
        at ru.yandex.clickhouse.jdbcbridge.internal.vertx.ext.web.handler.impl.TimeoutHandlerImpl.handle(TimeoutHandlerImpl.java:45)
        at ru.yandex.clickhouse.jdbcbridge.internal.vertx.ext.web.handler.impl.TimeoutHandlerImpl.handle(TimeoutHandlerImpl.java:27)
        at ru.yandex.clickhouse.jdbcbridge.internal.vertx.ext.web.impl.RouteState.handleContext(RouteState.java:1038)
        at ru.yandex.clickhouse.jdbcbridge.internal.vertx.ext.web.impl.RoutingContextImplBase.iterateNext(RoutingContextImplBase.java:137)
        at ru.yandex.clickhouse.jdbcbridge.internal.vertx.ext.web.impl.RoutingContextImpl.next(RoutingContextImpl.java:132)
        at ru.yandex.clickhouse.jdbcbridge.internal.vertx.ext.web.handler.impl.ResponseContentTypeHandlerImpl.handle(ResponseContentTypeHandlerImpl.java:54)
        at ru.yandex.clickhouse.jdbcbridge.internal.vertx.ext.web.handler.impl.ResponseContentTypeHandlerImpl.handle(ResponseContentTypeHandlerImpl.java:28)
        at ru.yandex.clickhouse.jdbcbridge.internal.vertx.ext.web.impl.RouteState.handleContext(RouteState.java:1038)
        at ru.yandex.clickhouse.jdbcbridge.internal.vertx.ext.web.impl.RoutingContextImplBase.iterateNext(RoutingContextImplBase.java:101)
        at ru.yandex.clickhouse.jdbcbridge.internal.vertx.ext.web.impl.RoutingContextImpl.next(RoutingContextImpl.java:132)
        at ru.yandex.clickhouse.jdbcbridge.JdbcBridgeVerticle.responseHandlers(JdbcBridgeVerticle.java:314)
        at ru.yandex.clickhouse.jdbcbridge.internal.vertx.ext.web.impl.RouteState.handleContext(RouteState.java:1038)
        at ru.yandex.clickhouse.jdbcbridge.internal.vertx.ext.web.impl.RoutingContextImplBase.iterateNext(RoutingContextImplBase.java:101)
        at ru.yandex.clickhouse.jdbcbridge.internal.vertx.ext.web.impl.RoutingContextImpl.next(RoutingContextImpl.java:132)
        at ru.yandex.clickhouse.jdbcbridge.internal.vertx.ext.web.handler.impl.BodyHandlerImpl$BHandler.doEnd(BodyHandlerImpl.java:296)
        at ru.yandex.clickhouse.jdbcbridge.internal.vertx.ext.web.handler.impl.BodyHandlerImpl$BHandler.end(BodyHandlerImpl.java:276)
        at ru.yandex.clickhouse.jdbcbridge.internal.vertx.ext.web.handler.impl.BodyHandlerImpl.lambda$handle$0(BodyHandlerImpl.java:87)
        at ru.yandex.clickhouse.jdbcbridge.internal.vertx.core.http.impl.HttpServerRequestImpl.onEnd(HttpServerRequestImpl.java:523)
        at ru.yandex.clickhouse.jdbcbridge.internal.vertx.core.http.impl.HttpServerRequestImpl.handleEnd(HttpServerRequestImpl.java:509)
        at ru.yandex.clickhouse.jdbcbridge.internal.vertx.core.http.impl.Http1xServerConnection.handleEnd(Http1xServerConnection.java:176)
        at ru.yandex.clickhouse.jdbcbridge.internal.vertx.core.http.impl.Http1xServerConnection.handleMessage(Http1xServerConnection.java:138)
        at ru.yandex.clickhouse.jdbcbridge.internal.vertx.core.impl.ContextImpl.executeTask(ContextImpl.java:366)
        at ru.yandex.clickhouse.jdbcbridge.internal.vertx.core.impl.EventLoopContext.execute(EventLoopContext.java:43)
        at ru.yandex.clickhouse.jdbcbridge.internal.vertx.core.impl.ContextImpl.executeFromIO(ContextImpl.java:229)
        at ru.yandex.clickhouse.jdbcbridge.internal.vertx.core.net.impl.VertxHandler.channelRead(VertxHandler.java:163)
        at ru.yandex.clickhouse.jdbcbridge.internal.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
        at ru.yandex.clickhouse.jdbcbridge.internal.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
        at ru.yandex.clickhouse.jdbcbridge.internal.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
        at ru.yandex.clickhouse.jdbcbridge.internal.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93)
        at ru.yandex.clickhouse.jdbcbridge.internal.netty.handler.codec.http.websocketx.extensions.WebSocketServerExtensionHandler.channelRead(WebSocketServerExtensionHandler.java:101)
        at ru.yandex.clickhouse.jdbcbridge.internal.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
        at ru.yandex.clickhouse.jdbcbridge.internal.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
        at ru.yandex.clickhouse.jdbcbridge.internal.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
        at ru.yandex.clickhouse.jdbcbridge.internal.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324)
        at ru.yandex.clickhouse.jdbcbridge.internal.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296)
        at ru.yandex.clickhouse.jdbcbridge.internal.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
        at ru.yandex.clickhouse.jdbcbridge.internal.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
        at ru.yandex.clickhouse.jdbcbridge.internal.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
        at ru.yandex.clickhouse.jdbcbridge.internal.vertx.core.http.impl.Http1xOrH2CHandler.end(Http1xOrH2CHandler.java:61)
        at ru.yandex.clickhouse.jdbcbridge.internal.vertx.core.http.impl.Http1xOrH2CHandler.channelRead(Http1xOrH2CHandler.java:38)
        at ru.yandex.clickhouse.jdbcbridge.internal.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
        at ru.yandex.clickhouse.jdbcbridge.internal.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
        at ru.yandex.clickhouse.jdbcbridge.internal.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
        at ru.yandex.clickhouse.jdbcbridge.internal.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)
        at ru.yandex.clickhouse.jdbcbridge.internal.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
        at ru.yandex.clickhouse.jdbcbridge.internal.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
        at ru.yandex.clickhouse.jdbcbridge.internal.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)
        at ru.yandex.clickhouse.jdbcbridge.internal.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163)
        at ru.yandex.clickhouse.jdbcbridge.internal.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714)
        at ru.yandex.clickhouse.jdbcbridge.internal.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:650)
        at ru.yandex.clickhouse.jdbcbridge.internal.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:576)
        at ru.yandex.clickhouse.jdbcbridge.internal.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493)
        at ru.yandex.clickhouse.jdbcbridge.internal.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
        at ru.yandex.clickhouse.jdbcbridge.internal.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
        at ru.yandex.clickhouse.jdbcbridge.internal.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
        at java.lang.Thread.run(Thread.java:748)
Caused by: ru.yandex.clickhouse.jdbcbridge.core.DataAccessException: Failed to access [impala] due to: impala - Connection is not available, request timed out after 30000ms.
        at ru.yandex.clickhouse.jdbcbridge.impl.JdbcDataSource.inferTypes(JdbcDataSource.java:660)
        at ru.yandex.clickhouse.jdbcbridge.core.NamedDataSource.lambda$getResultColumns$0(NamedDataSource.java:452)
        at ru.yandex.clickhouse.jdbcbridge.internal.github.benmanes.caffeine.cache.LocalCache.lambda$statsAware$0(LocalCache.java:139)
        at ru.yandex.clickhouse.jdbcbridge.internal.github.benmanes.caffeine.cache.BoundedLocalCache.lambda$doComputeIfAbsent$14(BoundedLocalCache.java:2343)
        at java.util.concurrent.ConcurrentHashMap.compute(ConcurrentHashMap.java:1853)
        at ru.yandex.clickhouse.jdbcbridge.internal.github.benmanes.caffeine.cache.BoundedLocalCache.doComputeIfAbsent(BoundedLocalCache.java:2341)
        at ru.yandex.clickhouse.jdbcbridge.internal.github.benmanes.caffeine.cache.BoundedLocalCache.computeIfAbsent(BoundedLocalCache.java:2324)
        at ru.yandex.clickhouse.jdbcbridge.internal.github.benmanes.caffeine.cache.LocalCache.computeIfAbsent(LocalCache.java:108)
        at ru.yandex.clickhouse.jdbcbridge.internal.github.benmanes.caffeine.cache.LocalManualCache.get(LocalManualCache.java:62)
        at ru.yandex.clickhouse.jdbcbridge.core.NamedDataSource.getResultColumns(NamedDataSource.java:451)
        ... 60 more
Caused by: java.sql.SQLTransientConnectionException: impala - Connection is not available, request timed out after 30000ms.
        at ru.yandex.clickhouse.jdbcbridge.internal.zaxxer.hikari.pool.HikariPool.createTimeoutException(HikariPool.java:695)
        at ru.yandex.clickhouse.jdbcbridge.internal.zaxxer.hikari.pool.HikariPool.getConnection(HikariPool.java:197)
        at ru.yandex.clickhouse.jdbcbridge.internal.zaxxer.hikari.pool.HikariPool.getConnection(HikariPool.java:162)
        at ru.yandex.clickhouse.jdbcbridge.internal.zaxxer.hikari.HikariDataSource.getConnection(HikariDataSource.java:100)
        at ru.yandex.clickhouse.jdbcbridge.impl.JdbcDataSource.getConnection(JdbcDataSource.java:458)
        at ru.yandex.clickhouse.jdbcbridge.impl.JdbcDataSource.inferTypes(JdbcDataSource.java:628)
        ... 69 more
Caused by: java.sql.SQLException: [Cloudera][ImpalaJDBCDriver](500164) Error initialized or created transport for authentication: Empty nameString not allowed.
        at com.cloudera.impala.hivecommon.api.HiveServer2ClientFactory.createTransport(Unknown Source)
        at com.cloudera.impala.hivecommon.api.HiveServer2ClientFactory.createClient(Unknown Source)
        at com.cloudera.impala.hivecommon.core.HiveJDBCCommonConnection.establishConnection(Unknown Source)
        at com.cloudera.impala.impala.core.ImpalaJDBCDSIConnection.establishConnection(Unknown Source)
        at com.cloudera.impala.jdbc.core.LoginTimeoutConnection$1.call(Unknown Source)
        at com.cloudera.impala.jdbc.core.LoginTimeoutConnection$1.call(Unknown Source)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
Caused by: com.cloudera.impala.support.exceptions.GeneralException: [Cloudera][ImpalaJDBCDriver](500164) Error initialized or created transport for authentication: Empty nameString not allowed.
        ... 9 more
Caused by: java.lang.IllegalArgumentException: Empty nameString not allowed
        at sun.security.krb5.PrincipalName.validateNameStrings(PrincipalName.java:179)
        at sun.security.krb5.PrincipalName.<init>(PrincipalName.java:402)
        at sun.security.krb5.PrincipalName.<init>(PrincipalName.java:477)
        at sun.security.jgss.krb5.Krb5NameElement.getInstance(Krb5NameElement.java:123)
        at sun.security.jgss.krb5.Krb5MechFactory.getNameElement(Krb5MechFactory.java:95)
        at sun.security.jgss.GSSManagerImpl.getNameElement(GSSManagerImpl.java:203)
        at sun.security.jgss.GSSNameImpl.getElement(GSSNameImpl.java:477)
        at sun.security.jgss.GSSNameImpl.init(GSSNameImpl.java:201)
        at sun.security.jgss.GSSNameImpl.<init>(GSSNameImpl.java:170)
        at sun.security.jgss.GSSManagerImpl.createName(GSSManagerImpl.java:138)
        at com.sun.security.sasl.gsskerb.GssKrb5Client.<init>(GssKrb5Client.java:107)
        at com.sun.security.sasl.gsskerb.FactoryImpl.createSaslClient(FactoryImpl.java:63)
        at javax.security.sasl.Sasl.createSaslClient(Sasl.java:420)
        at com.cloudera.impala.jdbc42.internal.apache.thrift.transport.TSaslClientTransport.<init>(TSaslClientTransport.java:73)
        at com.cloudera.impala.hivecommon.api.HiveServerPrivilegedAction.run(Unknown Source)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:360)
        at com.cloudera.impala.hivecommon.api.HiveServer2ClientFactory.createTransport(Unknown Source)
        at com.cloudera.impala.hivecommon.api.HiveServer2ClientFactory.createClient(Unknown Source)
        at com.cloudera.impala.hivecommon.core.HiveJDBCCommonConnection.establishConnection(Unknown Source)
        at com.cloudera.impala.impala.core.ImpalaJDBCDSIConnection.establishConnection(Unknown Source)
        at com.cloudera.impala.jdbc.core.LoginTimeoutConnection$1.call(Unknown Source)
        at com.cloudera.impala.jdbc.core.LoginTimeoutConnection$1.call(Unknown Source)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

judging by the error messge, it seems the JAAS configuration file is not properly read by jdbc-brdige, since it is complaining about empty principal.
I am not familiar with Java nor JVM enough to debug this error.
could anyone help? Thanks in advance!

zhicwu commented 2 years ago

Sorry for the late response and thanks for the detailed information.

JDBC bridge uses a custom class loader by default in order to support loading driver from URL and versioning(different versions of same driver). It's probably why you ran into the issue. If you're using docker, you may disable it by setting environment variable CUSTOM_DRIVER_LOADER to false. Alternatively, you may refer to below code snippet to start JDBC bridge - just make sure every jar you need is specified in classpath.

https://github.com/ClickHouse/clickhouse-jdbc-bridge/blob/70e1f25a25f0f0dc4d453f63ead3542263d85d80/docker/docker-entrypoint.sh#L23-L29

gfunc commented 2 years ago

@zhicwu Thanks for your reply!
I am not using docker, but I will try to study this script.

gfunc commented 2 years ago

This problem is solved by using docker with below command.

docker run -d -p 9019:9019 \
  --name clickhouse-jdbc-bridge \
  -v ${JDBC_DRIVER_DIR}:/app/drivers \
  -v ${JDBC_DATASOURCE_DIR}:/app/config/datasources \
  -v ${PWD}/clickhouse.keytab:/app/clickhouse.keytab \
  -v ${PWD}/krb5.conf:/etc/krb5.conf \
  -v ${PWD}/krb5Config.ini:/app/config/krb5Config.ini \
  -e CUSTOM_DRIVER_LOADER=false \
  -e JDBC_BRIDGE_JVM_OPTS='-Xms1024m -Xmx10240m -Djava.security.auth.login.config=/app/config/krb5Config.ini -Djava.security.krb5.conf=/etc/krb5.conf' \
  clickhouse/jdbc-bridge:2.0.6

Thanks again for your help @zhicwu !