feat: add pod acknowledged time metric

njtran commented 1 week ago

Fixes #N/A

Description Keeps track of pods in cluster state depending on work we've made towards scheduling the pod.

Pod Ack: when do we first see the pending pod in the cache (first write wins)
Pod Scheduling Attempted: when did we first actually pop it off the scheduling queue and try to schedule it (first write wins)
Pod Scheduling Success: when did we first think that the pod could schedule to a node (theoretical or not) (first write wins)

This will emit metrics that detail:

Pod Scheduling Decision Latency = the time it takes for Karpenter to first decide if a pod is schedulable or not
Pod Provisioning Binding Latency = how long it took for the pod to bind after we first thought that we could schedule the pod
Pod Provisioning Startup Latency = how long it took for the pod to go running after we first thought that we could schedule the pod

This also changes a metric name (can change since it hasn't been released yet):

karpenter_pods_current_unbound_duration_seconds -> karpenter_pods_unbound_duration_seconds

How was this change tested? make presubmit

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

coveralls commented 1 week ago

Pull Request Test Coverage Report for Build 11850563098

Warning: This coverage report may be inaccurate.

This pull request's base commit is no longer the HEAD commit of its target branch. This means it includes changes from outside the original pull request, including, potentially, unrelated coverage changes.

For more information on this, see Tracking coverage changes with pull request builds.
To avoid this issue with future PRs, see these Recommended CI Configurations.
For a quick fix, rebase this PR at GitHub. Your next report should be accurate.

Details

132 of 133 (99.25%) changed or added relevant lines in 4 files are covered.
33 unchanged lines in 6 files lost coverage.
Overall coverage increased (+0.2%) to 81.05%

Changes Missing Coverage	Covered Lines	Changed/Added Lines	%
pkg/controllers/controllers.go	0	1	0.0%
<!--	Total:	132	133	99.25%	-->

Files with Coverage Reduction	New Missed Lines	%
pkg/utils/nodeclaim/nodeclaim.go	2	37.07%
pkg/controllers/controllers.go	3	0.0%
pkg/controllers/nodeclaim/lifecycle/launch.go	4	93.1%
pkg/controllers/provisioning/scheduling/scheduler.go	4	95.18%
pkg/controllers/nodeclaim/lifecycle/initialization.go	8	86.11%
pkg/controllers/nodeclaim/lifecycle/controller.go	12	71.07%
<!--	Total:	33	-->

Totals
Change from base Build 11749223840:	0.2%
Covered Lines:	8601
Relevant Lines:	10612

💛 - Coveralls

k8s-ci-robot commented 1 week ago

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: jonathan-innis, njtran

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files: - ~~[OWNERS](https://github.com/kubernetes-sigs/karpenter/blob/main/OWNERS)~~ [jonathan-innis,njtran] Approvers can indicate their approval by writing `/approve` in a comment Approvers can cancel approval by writing `/approve cancel` in a comment

kubernetes-sigs / karpenter