search

ClickHouse / clickhouse-java

ClickHouse Java Clients & JDBC Driver
https://clickhouse.com
Apache License 2.0
1.45k stars 535 forks source link

How to use lz4/zstd compress data when insert data into clickhouse? I use clickhouse jdbc client #1813

Closed zalj closed 2 months ago

zalj commented 2 months ago

I am using the ClickHouse-JDBC client to write a considerable amount of data (hundreds of TiB) into ClickHouse in batches. I want to reduce the total traffic (save bandwidth).

I have learned that ClickHouse supports data compress. However, I am currently unsure how to enable the compression feature in the ClickHouse JDBC client.

The work I have done so far includes: (1) Adding the 'decompress' parameter in the jdbc-url. The jdbc-url I am using is similar to the following: jdbc:ch://localhost:8123?decompress=true (2) Including the LZ4 dependency

        <dependency>
            <groupId>org.lz4</groupId>
            <artifactId>lz4-java</artifactId>
            <version>1.8.0</version>
        </dependency>

I would like to know is these enough, and if not, what else needs to be done?

Also, how can I confirm whether the compress has been enabled?