search

aws / amazon-managed-service-for-prometheus-roadmap

Amazon Managed Service for Prometheus Public Roadmap
Other
41 stars 3 forks source link

[Feature] Support for month-over-month and quarter-over-quarter queries #23

Open ampabhi-aws opened 2 years ago

ampabhi-aws commented 2 years ago

Why

In order to do solve capacity planning, SLI/SLO, or long term performance analysis, AMP needs to support long term trending queries beyond the 32 day limit that exists today. In order to make those queries efficient, AMP should support downsampling, and auto-reduce the resolution of data past 1 week (1 min resolution), 30 days (5 min resolution) 6 months (1 hour resolution) and provide the option for keeping or dropping raw data past the first tier.

rsheldon-ansira commented 1 year ago

Limiting queries to 32 days it very limiting when looking at trending data. e.g. how has my service performance changed over the last N months ( as much time as you have data in prometheus for). I don't just want to compare this month to a month N months ago - I want to see a graph showing how things have changed.

Obviously we don't need or want to use 10s data for this (huge), which is why this ties in with the ability to

  1. down-sample data from the collection interval (10s in our case), so lower intervals (minute, hour, day). Not sure if we could accomplish this with prometheus recording rules.
  2. have different data retention policies for different metrics - not all metrics have the same value, and for longer term trending you're likely using aggregated/down-sampled data, and we don't need to keep the fine-grained details.