search

thanos-io / thanos

Highly available Prometheus setup with long term storage capabilities. A CNCF Incubating project.
https://thanos.io
Apache License 2.0
13.12k stars 2.1k forks source link

Version 2 of string interning #7067

Open GiedriusS opened 10 months ago

GiedriusS commented 10 months ago

Is your proposal related to a problem?

We currently have interning in Receive under a feature flag. It works but it could be improved a lot. We still allocate memory for each duplicate and then deduplicate later on.

Remote write requests and series responses will allocate new memory for each duplicated string. The most obvious pathological case is when querying histograms. The label le can be easily allocated thousands of times. This is even worse because of incoming duplicated data due to HA setup.

I think this leads to huge inefficiencies.

Describe the solution you'd like

We should avoid allocating memory for duplicated strings. This could be a per-request map of hash to []byte/string.

We should be able to use this same deduplication technique in remote write, series responses, and any other place.

bwplotka commented 10 months ago

BTW this problem will be totally fixed by Remote Write 2.0 https://docs.google.com/document/d/1PljkX3YLLT-4f7MqrLt7XCVPG3IsjRREzYrUzBxCPV0/edit#heading=h.3p42p5s8n0ui

GiedriusS commented 10 months ago

That's true but I'd like to solve this problem at the RPC level once and for all so that all and future hot RPC paths would be protected from this issue automatically. Otherwise, we will have to implement this https://github.com/thanos-io/thanos/pull/5906, have a list of string refs like your proposal in the remote_write calls, etc.