After StarRocks supports lz4 compression for stream load json format in https://github.com/StarRocks/starrocks/pull/43732, the connector can compress the json data before sending to StarRocks which will reduce the network traffic significantly. In the test to load clickbench data to starrocks, the compression ratio can be ~8, and the load performance has a 3.64% degradation which is acceptable. This RP depends on stream-load-sdk.
You can enable it with the following configuration
After StarRocks supports lz4 compression for stream load json format in https://github.com/StarRocks/starrocks/pull/43732, the connector can compress the json data before sending to StarRocks which will reduce the network traffic significantly. In the test to load clickbench data to starrocks, the compression ratio can be ~8, and the load performance has a 3.64% degradation which is acceptable. This RP depends on stream-load-sdk. You can enable it with the following configuration