search

go-graphite / carbon-clickhouse

Graphite metrics receiver with ClickHouse as storage
MIT License
186 stars 47 forks source link

Uploader max chunk size #81

Closed msaf1980 closed 3 years ago

msaf1980 commented 3 years ago

Allow rotate chunk with max size (additional parameter to interval). In minute range we have different metric rate (for first 10 sec heavy rate, and lower rate in other time). So first chunk is huge and smaller after (it's negative for insert perfomance, many parts in clickhouse generated). With this parameter I can switch chunk with optimal chunk size (with failback interval, if small metric count was received). Like this

Before

[data]
..
chunk-interval = "10s"

After

[data]
..
chunk-switch-size = MAX_SIZE
chunk-interval = "20s"
Felixoid commented 3 years ago

@msaf1980 you can consider https://github.com/Felixoid/carbon-clickhouse/commit/ef23585 as a fix :+1: