search

ClickHouse / clickhouse-jdbc-bridge

A JDBC proxy from ClickHouse to external databases
Apache License 2.0
168 stars 60 forks source link

Chinese table name and column name #140

Closed delubee closed 2 years ago

delubee commented 2 years ago

when the column name is Chinese I got this error:

二月 18, 2022 7:18:46 下午 ru.yandex.clickhouse.jdbcbridge.core.NamedDataSource executeQuery
信息: Executing query(schema=[]):
select * from t2
二月 18, 2022 7:18:46 下午 ru.yandex.clickhouse.jdbcbridge.JdbcBridgeVerticle errorHandler
严重: Failed to respond
java.lang.IllegalArgumentException: Unknown column [å­æ®µ1]! Available columns are: [id],[字段1]
    at ru.yandex.clickhouse.jdbcbridge.core.DataTableReader.process(DataTableReader.java:128)
    at ru.yandex.clickhouse.jdbcbridge.impl.JdbcDataSource.writeQueryResult(JdbcDataSource.java:721)
    at ru.yandex.clickhouse.jdbcbridge.core.NamedDataSource.executeQuery(NamedDataSource.java:552)
    at ru.yandex.clickhouse.jdbcbridge.JdbcBridgeVerticle.handleQuery(JdbcBridgeVerticle.java:487)
    at ru.yandex.clickhouse.jdbcbridge.internal.vertx.ext.web.impl.BlockingHandlerDecorator.lambda$handle$0(BlockingHandlerDecorator.java:48)
    at ru.yandex.clickhouse.jdbcbridge.internal.vertx.core.impl.ContextImpl.lambda$executeBlocking$2(ContextImpl.java:313)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at ru.yandex.clickhouse.jdbcbridge.internal.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
    at java.lang.Thread.run(Thread.java:748)

table t2 DDL : create table t2 ( id int not null, 字段1 varchar(255) )

and when the table name is Chinese I got this error:

二月 18, 2022 7:31:19 下午 ru.yandex.clickhouse.jdbcbridge.core.NamedDataSource executeQuery
信息: Executing query(schema=[]):
select * from 表1
二月 18, 2022 7:31:19 下午 ru.yandex.clickhouse.jdbcbridge.JdbcBridgeVerticle errorHandler
严重: Failed to respond
ru.yandex.clickhouse.jdbcbridge.core.DataAccessException: Failed to access [mssql] due to: SQLState(S0001) VendorCode(102) “¡”附近有语法错误。
    at ru.yandex.clickhouse.jdbcbridge.impl.JdbcDataSource.writeQueryResult(JdbcDataSource.java:734)
    at ru.yandex.clickhouse.jdbcbridge.core.NamedDataSource.executeQuery(NamedDataSource.java:552)
    at ru.yandex.clickhouse.jdbcbridge.JdbcBridgeVerticle.handleQuery(JdbcBridgeVerticle.java:487)
    at ru.yandex.clickhouse.jdbcbridge.internal.vertx.ext.web.impl.BlockingHandlerDecorator.lambda$handle$0(BlockingHandlerDecorator.java:48)
    at ru.yandex.clickhouse.jdbcbridge.internal.vertx.core.impl.ContextImpl.lambda$executeBlocking$2(ContextImpl.java:313)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at ru.yandex.clickhouse.jdbcbridge.internal.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
    at java.lang.Thread.run(Thread.java:748)
Caused by: com.microsoft.sqlserver.jdbc.SQLServerException: “¡”附近有语法错误。
    at com.microsoft.sqlserver.jdbc.SQLServerException.makeFromDatabaseError(SQLServerException.java:262)
    at com.microsoft.sqlserver.jdbc.SQLServerStatement.getNextResult(SQLServerStatement.java:1624)
    at com.microsoft.sqlserver.jdbc.SQLServerStatement.doExecuteStatement(SQLServerStatement.java:868)
    at com.microsoft.sqlserver.jdbc.SQLServerStatement$StmtExecCmd.doExecute(SQLServerStatement.java:768)
    at com.microsoft.sqlserver.jdbc.TDSCommand.execute(IOBuffer.java:7194)
    at com.microsoft.sqlserver.jdbc.SQLServerConnection.executeCommand(SQLServerConnection.java:2979)
    at com.microsoft.sqlserver.jdbc.SQLServerStatement.executeCommand(SQLServerStatement.java:248)
    at com.microsoft.sqlserver.jdbc.SQLServerStatement.executeStatement(SQLServerStatement.java:223)
    at com.microsoft.sqlserver.jdbc.SQLServerStatement.execute(SQLServerStatement.java:744)
    at ru.yandex.clickhouse.jdbcbridge.internal.zaxxer.hikari.pool.ProxyStatement.execute(ProxyStatement.java:95)
    at ru.yandex.clickhouse.jdbcbridge.internal.zaxxer.hikari.pool.HikariProxyStatement.execute(HikariProxyStatement.java)
    at ru.yandex.clickhouse.jdbcbridge.impl.JdbcDataSource.writeQueryResult(JdbcDataSource.java:718)
    ... 8 more

table 表1 DDL: create table 表1 ( id int not null, f1 varchar(255) )

and when column and table names is ASCII ， it’s fine.

yingyingqiqi commented 2 years ago

I have just started to try JDBC bridge, and we also have an abnormal problem in Chinese fields. Do you have a repair plan?

zhicwu commented 2 years ago

I have just started to try JDBC bridge, and we also have an abnormal problem in Chinese fields. Do you have a repair plan?

I'll take a look tonight to see if this can be quickly fixed.

yingyingqiqi commented 2 years ago

The question arises here:： https://github.com/ClickHouse/clickhouse-jdbc-bridge/blob/f02b015093a3ef6b159c8ebd5fec1ceca98e0e87/src/main/java/ru/yandex/clickhouse/jdbcbridge/core/DataTableReader.java#L105-L116 due to： ColumnDefinition[] requestColumn != ColumnDefinition[] resultColumns Because the following code cannot parse Chinese: https://github.com/ClickHouse/clickhouse-jdbc-bridge/blob/f02b015093a3ef6b159c8ebd5fec1ceca98e0e87/src/main/java/ru/yandex/clickhouse/jdbcbridge/core/QueryParser.java#L317-L331 I think it can be changed to: URLDecoder.decode(str, "utf-8");

zhicwu commented 2 years ago

Thanks @yingyingqiqi for investigating the issue. Actually above code has nothing to do with charset encoding because it's based on characters. However, there is issue extracting query parameters but I haven't checked if it's caused by Vertx or ClickHouse.

delubee commented 2 years ago

The question arises here:：

https://github.com/ClickHouse/clickhouse-jdbc-bridge/blob/f02b015093a3ef6b159c8ebd5fec1ceca98e0e87/src/main/java/ru/yandex/clickhouse/jdbcbridge/core/DataTableReader.java#L105-L116

due to： ColumnDefinition[] requestColumn != ColumnDefinition[] resultColumns Because the following code cannot parse Chinese: https://github.com/ClickHouse/clickhouse-jdbc-bridge/blob/f02b015093a3ef6b159c8ebd5fec1ceca98e0e87/src/main/java/ru/yandex/clickhouse/jdbcbridge/core/QueryParser.java#L317-L331

I think it can be changed to: URLDecoder.decode(str, "utf-8");

it works :

// columns = unescape(table.substring(len, index)); // table = unescape(table.substring(index + 1)); try { columns = URLDecoder.decode(table.substring(len, index), "utf-8"); table = URLDecoder.decode(table.substring(index + 1), "utf-8"); } catch (UnsupportedEncodingException e) { e.printStackTrace(); }

zhicwu commented 2 years ago

it works

Good work. Mind to submit a pull request?