search

robusta-dev / krr

Prometheus-based Kubernetes Resource Recommendations
MIT License
3k stars 157 forks source link

Provide recommendations for historical deployments #171

Open davidcurrie opened 11 months ago

davidcurrie commented 11 months ago

Is your feature request related to a problem? Please describe. I'd like to be able to use krr to make recommendations in a cluster used for executing test runs but, at least as far as I can tell, it will only make recommendations for currently running resources.

Describe the solution you'd like Provide a mode in which krr relies only on data from Prometheus so it can be used to provide recommendations for any resource that existed at some point within the history duration.

Describe alternatives you've considered As things stand, we'd have to modify every CI pipeline to run krr before cleaning up the resources.

Are you interested in contributing a PR for this? No - sorry - I'm not a Pythonista.

Additional context None.

aantn commented 11 months ago

Thanks for opening. It's an interesting use case.

We probably wont be able to prioritize this soon, but we're open to PRs as always. (I know you're not a Pythonista :()

aantn commented 5 months ago

Hey, just to update we're working on a mode where KRR can operate only based on Prometheus data. So this will be supported in the near future. (Feel free to help by testing the Prometheus-only mode!)

https://github.com/robusta-dev/krr/pull/266

davidcurrie commented 5 months ago

@aantn, thanks for the heads-up. We tried it, but it still doesn't seem to help for the scenario described in this issue. Does this PR only query Prometheus for currently active workloads? If so, that still doesn't help us, as the test workloads for which we're looking for recommendations are no longer running. The hope was that it would cover any workload that was active at some point during the history duration.

aantn commented 5 months ago

@davidcurrie no, it should fetch historical workloads too. What is the type of workload? If I provide you with a Prometheus query, can you check it manually and see if there is data for that query for the historical workloads?

JasonSinclair95 commented 5 months ago

Hi @aantn, i work with @davidcurrie and was the engineer behind testing this feature.

After installing a pod within a namespace containing a workload, I executed the 'krr' command, which detected the new pod. However, upon removing the pod from the namespace, subsequent executions of the command no longer displayed the pod. which tells me this feature operates exclusively in real-time, without having the ability to query historical data and just leverages Prometheus to retrieve its information. Am i wronge and if so can you guide me please

the commands i'm using: 

python krr.py simple --mode prometheus --prometheus-url http://localhost:9090/ --namespace test-namespace
aantn commented 5 months ago

Hi @JasonSinclair95, It is supposed to load the historical data, so that sounds like a bug on our side. What is the workload's type? I will try to reproduce this on my end.