SigNoz / signoz-otel-collector

SigNoz distro for OpenTelemetry Collector
44 stars 39 forks source link

Do not write to time_series_v4 table if the same fingerprint is already seen in hour window #325

Closed srikanthccv closed 2 months ago

srikanthccv commented 3 months ago

While the time series change over time, the rate at which they change is not often. The time_series_v4 table has the 1-hour granularity. Today, we continue to write to ClickHouse and let it do the deduplication using the ReplacingMergeTree. We can skip writing the same fingerprint for the one-hour window so that resources spent by ClickHouse are allocated elsewhere.