Goldilocks can not recommend size for "Rollout" Kind

Yufeireal commented 1 year ago

What happened?

Hi team,

We used Rollout kind, ApiGroup "argoproj.io/v1alpha1" to control the replicas of the deployment. For some reason, I can see the recommandation values for "Deployment" type but can not see the recommendation for Rollout type.

What did you expect to happen?

Recommendation also appear for Rollout Type.

How can we reproduce this?

Deploy a deployment and rollout, use the rollout to control the deployment. Actually no vpa recommendation is generated for the Rollout

Version

v7.2.0

Search

[X] I did search for other open and closed issues before opening this.

Code of Conduct

[X] I agree to follow this project's Code of Conduct

Additional context

No response

sudermanjr commented 1 year ago

You may need to add the argo project rollout type to the RBAC for goldilocks.

Can you share the controller logs?

Yufeireal commented 1 year ago

Thanks for the quick response. I actually didn't see any specific ERROR logs for Rollout type. Found something like this, but it looks normal. I0926 17:38:44.007835 1 vpa.go:184] Reconciling Namespace/default for Rollout/vweb with VPA/goldilocks-vweb When I search ERROR. Get something like this E0926 18:13:29.924914 1 vpa.go:324] Error updating VPA/goldilocks-vsight in Namespace/default: Operation cannot be fulfilled on verticalpodautoscalers.autoscaling.k8s.io "goldilocks-vsight": the object has been modified; please apply your changes to the latest version and try again E0926 18:13:29.924944 1 pod.go:42] Error reconciling: Operation cannot be fulfilled on verticalpodautoscalers.autoscaling.k8s.io "goldilocks-vsight": the object has been modified; please apply your changes to the latest version and try again E0926 18:13:52.727675 1 vpa.go:324] Error updating VPA/goldilocks-vwatchdog-cronjob-staging-prod-livestreamsd in Namespace/default: Operation cannot be fulfilled on verticalpodautoscalers.autoscaling.k8s.io "goldilocks-vwatchdog-cronjob-staging-prod-livestreamsd": the object has been modified; please apply your changes to the latest version and try again E0926 18:13:52.727714 1 pod.go:42] Error reconciling: Operation cannot be fulfilled on verticalpodautoscalers.autoscaling.k8s.io "goldilocks-vwatchdog-cronjob-staging-prod-livestreamsd": the object has been modified; please apply your changes to the latest version and try again E0926 18:14:17.992868 1 vpa.go:324] Error updating VPA/goldilocks-valarm-kafka-consumer-hz-p1 in Namespace/default: Operation cannot be fulfilled on verticalpodautoscalers.autoscaling.k8s.io "goldilocks-valarm-kafka-consumer-hz-p1": the object has been modified; please apply your changes to the latest version and try again E0926 18:14:17.992900 1 pod.go:42] Error reconciling: Operation cannot be fulfilled on verticalpodautoscalers.autoscaling.k8s.io "goldilocks-valarm-kafka-consumer-hz-p1": the object has been modified; please apply your changes to the latest version and try again E0926 18:14:53.983836 1 vpa.go:324] Error updating VPA/goldilocks-vshort in Namespace/default: Operation cannot be fulfilled on verticalpodautoscalers.autoscaling.k8s.io "goldilocks-vshort": the object has been modified; please apply your changes to the latest version and try again E0926 18:14:53.983880 1 pod.go:42] Error reconciling: Operation cannot be fulfilled on verticalpodautoscalers.autoscaling.k8s.io "goldilocks-vshort": the object has been modified; please apply your changes to the latest version and try again Is there any other places which could show the logs. But df let me try add the rollout type to RBAC

sudermanjr commented 1 year ago

Is there a VPA object that gets created in that namespace? If so, can you share the yaml definition of that VPA object?

sudermanjr commented 1 year ago

I am wondering if this is similar to - https://github.com/FairwindsOps/goldilocks/issues/621

Yufeireal commented 1 year ago

Yeah, I do see the vpa is created, fyi, and it shows some data in status.

`apiVersion: autoscaling.k8s.io/v1 kind: VerticalPodAutoscaler metadata: creationTimestamp: "2023-09-19T20:56:57Z" generation: 5750 labels: creator: Fairwinds source: goldilocks name: goldilocks-vptz namespace: default resourceVersion: "7318817167" uid: f5060bc3-25ba-4491-82a8-20af305d4acb spec: targetRef: apiVersion: argoproj.io/v1alpha1 kind: Rollout name: vptz updatePolicy: updateMode: "Off" status: conditions:

lastTransitionTime: "2023-09-19T20:57:48Z" status: "True" type: RecommendationProvided recommendation: containerRecommendations:
- containerName: istio-proxy lowerBound: cpu: 10m memory: "203641596" target: cpu: 35m memory: "225384266" uncappedTarget: cpu: 35m memory: "225384266" upperBound: cpu: 55m memory: "257315135"
- containerName: vptz lowerBound: cpu: 10m memory: "52428800" target: cpu: 11m memory: "63544758" uncappedTarget: cpu: 11m memory: "63544758" upperBound: cpu: 12m memory: "72547335"`

Yufeireal commented 1 year ago

I am wondering if this is similar to - #621

Could be related I think. The Rollout is used to control Deployment(So basically we remove the replicas from deployment). Along with this, there's another HPA to control the Rollout.

sudermanjr commented 1 year ago

Interesting. Seeing that there's a VPA with recommendations, this may just be related to the goldilocks dashboard (the summary package really)

Yufeireal commented 8 months ago

Hi team, did we figure out why this is happening? Thanks!

Eslam-mohammed-anwar commented 3 months ago

https://github.com/FairwindsOps/goldilocks/pull/685/files

FairwindsOps / goldilocks