search

jaegertracing / jaeger-clickhouse

Jaeger ClickHouse storage plugin implementation
Apache License 2.0
236 stars 50 forks source link

Change batch writing policy #58

Closed EinKrebs closed 2 years ago

EinKrebs commented 2 years ago

Currently, we insert batch by one span. But for ClickHouse, it's better to insert big amounts of data.

EinKrebs commented 2 years ago

Also, if we write huge amounts of data, it's impossible not to repeat failed transactions. So, we need to figure something out.

EinKrebs commented 2 years ago

Ok, I take my words back, It's clickhouse-go driver feature, and actually data's written by big amounts.

pavolloffay commented 2 years ago

one question, could we remove the internal batching (time and size) and depend on the driver?

EinKrebs commented 2 years ago

No. Driver makes it possible to prepare statement, then execute it many times in one transaction, and it will be written in one bulk.