housepower / clickhouse_sinker

Easily load data from kafka to ClickHouse
https://housepower.github.io/clickhouse_sinker
Apache License 2.0
514 stars 118 forks source link

[help] 每次处理的message 批次大小不一? #178

Closed LeeMo2K10 closed 1 year ago

LeeMo2K10 commented 1 year ago

"flushInterval": 60, "bufferSize": 131072, 任务设置的是60s, 131072, 看日志并没有每次都拉取到131072, image

注意:第6行 2023-06-06T07:05:48.428Z 没到60s

image

yuzhichang commented 1 year ago

正常,批次大小只是一个大致规格。建议使用最新v3.0.6,性能和内存都有改进。