ncabatoff / process-exporter

Prometheus exporter that mines /proc to report on selected processes

MIT License

1.72k stars 270 forks source link

Storing the data for long period of time #221

Open vickyvikas7988 opened 2 years ago

vickyvikas7988 commented 2 years ago

As process exporter emits a lot of data metrics and we use prometheus to store the data. We need to store the data for a long time and we heard prometheus is single point failure causes issues.

We have 50 data centers and we would need to monitor 50 nodes. 50 * 50 = 250

and this can increase a lot.

Is there a solution that we can store this data for longer time other than prometheus? What is best/decent solution gathering metrics data from all around the world?

Your answer will help me and lot other who are not finding relevant answers over google.

Thanks a lot

richard001100 commented 2 years ago

A good solution if you have aws cloud access would be to store data in an s3 bucket. You can incorporate thanos sidecars which can run in line with your prometheus instances and push your WAL files up to s3

vickyvikas7988 commented 2 years ago

Hi @richard001100

thanks for the quick reply.

We are using OCI Cloud (Oracle Cloud) and we have servers across all regions.

leoberbert commented 2 years ago

Dear,

An excellent alternative would be to change the storage time of prometheus using:

--storage.tsdb.retention.time: When to remove old data. Defaults to 15d. overrides

Change for:

--storage.tsdb.retention.time=1y

or

--storage.tsdb.retention.time=365d

https://prometheus.io/docs/prometheus/latest/storage/

Regards,

flixr commented 2 years ago

This is not a question/issue for this exporter...

I would suggest to look into Thanos or Grafana Mimir for scalable long term storage.