search

parca-dev / parca

Continuous profiling for analysis of CPU and memory usage, down to the line number and throughout time. Saving infrastructure cost, improving performance, and increasing reliability.
https://parca.dev/
Apache License 2.0
4.16k stars 217 forks source link

Support for postponing data transfer between agent and server #4429

Open pavolzibrita opened 7 months ago

pavolzibrita commented 7 months ago

In some cases it is not easy to ensure or create connection between agent and server. It would be great if agent can log the data locally, the data could be later transferred were they can be processed (e.g. to s3), and then some other process could upload the data to the server.

So a data line like this: parca agent -> local disk -> move to s3 -> parca uploader (from s3 to server) -> parca server.

brancz commented 7 months ago

The parca server can handle out of order inserts just fine, so just creating a log from parca-agent would already be sufficient. I like this idea!