Exception thrown after insertion

[sliontc]

How to get this fixed?

[zhicwu]

Can you increase max execution time in config/vertx.json?

[sliontc]

Can you increase max execution time in config/vertx.json?

I didn't use that configuration. My usecase is inserting data into jdbc table engine.

[zhicwu]

Are you using docker? Regardless, you can take this as an example, put config/vertx.json under your work directory(e.g. /app/config/vertx.json in docker container).

[sliontc]

Are you using docker? Regardless, you can take this as an example, put config/vertx.json under your work directory(e.g. /app/config/vertx.json in docker container).

No, I use Java CLI after building your git repo. Here is my datasource config. Seems vertx.json is not used.

[sliontc]

The insert is almost done.

[zhicwu]

Seems vertx.json is not used.

You can create the file if it does not exist.

[sliontc]

Seems vertx.json is not used.

You can create the file if it does not exist.

I created the file vertx.json:

{ "maxEventLoopExecuteTime": 15, "maxEventLoopExecuteTimeUnit": "SECONDS", "maxWorkerExecuteTime": 3000, "maxWorkerExecuteTimeUnit": "SECONDS", "workerPoolSize": 20 }

But it failed with different error:

[sliontc]

If I insert select top 100 records, it run successfully.

[zhicwu]

What's the log level in your log4j.properties? You may hit the error when trace level is enabled. Can you edit the file in your clickhouse-jdbc-bridge jar file by changing log level to WARN or ERROR?

Alternatively, you can use this new build, which turned off debug log causing the problem.

[sliontc]

What's the log level in your log4j.properties? You may hit the error when trace level is enabled. Can you edit the file in your clickhouse-jdbc-bridge jar file by changing log level to WARN or ERROR?

Alternatively, you can use this new build, which turned off debug log causing the problem.

Your log4j.properties is same with mine like below:

log4j.rootLogger=DEBUG, STDOUT log4j.category.ru.yandex.clickhouse=WARN log4j.category.ru.yandex.clickhouse.jdbcbridge=DEBUG log4j.category.ru.yandex.clickhouse.jdbcbridge.internal=WARN log4j.appender.STDOUT=ru.yandex.clickhouse.jdbcbridge.internal.log4j.ConsoleAppender log4j.appender.STDOUT.layout=ru.yandex.clickhouse.jdbcbridge.internal.log4j.PatternLayout log4j.appender.STDOUT.layout.ConversionPattern=%d{yyyy-MM-dd HH:mm:ss.sss} [%t] [%-5p] {%c{1}:%L} - %m%n

[zhicwu]

I see. Did you try the alternative approach? Changing log level to warn/error works too.

[sliontc]

I see. Did you try the alternative approach? Changing log level to warn/error works too. I use your build and following error:

[sliontc]

And this is error from CK log:

[zhicwu]

Thank you @sliontc for reporting the issues. From your screenshots, it seems the 2M rows were all inserted into SQL server, but in the end JDBC bridge failed to telling ClickHouse the mutation has been completed. Let me try to fix the issue(and the incorrect effectedRows in log) later today. Meanwhile, you may want to verify if all 2,515,081 rows were inserted into SQL server successfully.

select * from jdbc('<your datasource>', 'select count(1) from t_UnionResource')

Moreover, I noticed that the insertion took 470 seconds, so maybe batch_size(defaults to 100) can be increased for further improvement.

[sliontc]

Thank you @sliontc for reporting the issues. From your screenshots, it seems the 2M rows were all inserted into SQL server, but in the end JDBC bridge failed to telling ClickHouse the mutation has been completed. Let me try to fix the issue(and the incorrect effectedRows in log) later today. Meanwhile, you may want to verify if all 2,515,081 rows were inserted into SQL server successfully.

select * from jdbc('<your datasource>', 'select count(1) from t_UnionResource')

Moreover, I noticed that the insertion took 470 seconds, so maybe batch_size(defaults to 100) can be increased for further improvement.

yes, the data were all inserted.

[sliontc]

So the bug is fixed?

[zhicwu]

Yes, it's been fixed. However, I'm currently having trouble releasing 2.0.0 because of slowness issue on sonatype. I'll try again tomorrow morning and then merge released code into master branch.

Meantime, if you don't want to wait, please follow instructions below:

• build latest code on develop branch

  git clone -b develop https://github.com/ClickHouse/clickhouse-jdbc-bridge.git
  cd clickhouse-jdbc-bridge
  mvn -DskipTests -Drevision=2.0.0 clean package

• use target/clickhouse-jdbc-bridge-2.0.0-shaded.jar to replace yours
• create server.json based on this example in your work directory - you probably should increase queryTimeout to 600000 milliseconds or so
• restart jdbc bridge, truncate your table and start over

  select * from jdbc('<your datasource id>?mutation', 'truncate table dbo.t_UnionResource');
  insert into ...

[sliontc]

Whoo, that's great. You are so awesome.