search

johrstrom / jmeter-prometheus-plugin

A Prometheus Listener for Apache JMeter that exposes results in an http API
Apache License 2.0
164 stars 109 forks source link

How to increase data retention duration in Prometheus? #114

Open Sravanthi0614 opened 1 year ago

Sravanthi0614 commented 1 year ago

HI,

I am using Prometheus-2.32.1 version, in the prometheus.yml file how can I increase the duration for data retention.

Thanks, Sravanthi MS

johrstrom commented 1 year ago

It appears to be this flag --storage.tsdb.retention.time.

https://prometheus.io/docs/prometheus/latest/storage/

Sravanthi0614 commented 1 year ago

Using this tag in prometheus.yml file, it is throwing error. Kindly help me where and how I need to configure this tag.

johrstrom commented 1 year ago

I think it's a flag that you pass to Prometheus when you boot it up.

Sravanthi0614 commented 1 year ago

Can you please brief me on where the parameters are passed to Prometheus(eg:setup or bin or etc folders or any file - as i couldn't find them) during the boot up.

johrstrom commented 1 year ago

How does your Prometheus server start? Do you have a systemd unit file?

Sravanthi0614 commented 1 year ago

I am starting Prometheus as a service, by clicking on prometheus.exe there is no systemd unit file

Sravanthi0614 commented 1 year ago

FYI, i am running on windows machine.

johrstrom commented 1 year ago

Got it. So I'm unfamiliar with how Windows made that icon that you click. But if it's a text file that you see prometheus.exe in, that's where you'd edit.

Basically you want to issue this to use 2 weeks, instead of just issuing prometheus.exe.

prometheus.exe --storage.tsdb.retention.time=2w
Sravanthi0614 commented 1 year ago

I successfully started prometheus with the following argument in the windows command prompt. prometheus.exe --storage.tsdb.retention.time=1y

and the console no longer announces prometheus is using the default 15 days data retention. But it is not confirming its using 1 year data retention in the console (it doesn't report anything about data retention, this time), and I also saw in the console multiple ..."Deleting obsolete block"... lines. We dont have data that is older than 1 year but we do have data that goes back to 15 days old threshold, so it looks like it might still be behaving as if the data retention is still set for 15 days. To at least help figure out what is going on here, is there a way to get prometheus on windows to always announce in the console its data retention effective setting? Thanks in advance!

johrstrom commented 1 year ago

I also saw in the console multiple ..."Deleting obsolete block"... lines.

I believe this is normal in that the system is compacting data and/or restructuring the data internally. So it's not removing data really, it's just shifting it around for optimizations.

is there a way to get prometheus on windows to always announce in the console its data retention effective setting?

I believe there's a config page you can navigate to. This may tell you what you're looking for.