Refactor system component Metrics: Move away from Global Variables

[yongruilin]

What would you like to be added?

Refactor the current approach to metrics in Kubernetes system components by moving away from global (package-level) variable definitions and avoiding registration in a global store (registry). Instead, implement instance-based metric definitions and registrations to increase modularity and flexibility within components.

Context: Currently, most metrics for system components are defined as global variables within the respective packages. For example: https://github.com/kubernetes/kubernetes/blob/7c56aa5a58904ec9910b6bf29385a8b85fb115be/staging/src/k8s.io/apiserver/pkg/endpoints/metrics/metrics.go#L65-L297

And they also registered in the global registry(legacyregistry), for example https://github.com/kubernetes/kubernetes/blob/7c56aa5a58904ec9910b6bf29385a8b85fb115be/staging/src/k8s.io/apiserver/pkg/endpoints/metrics/metrics.go#L407 .

Why is this needed?

Though it is convenient to use global variables, it is challenging for testing. For example, in the integration test , all metrics are shared. They are initialized once then hard to reset between sub-tests.

Moving to instance-based metrics would enhance maintainability and improve alignment with best practices for modular design in large codebases, supporting Kubernetes’ scalability and reliability goals.

Ref: https://google.github.io/styleguide/go/best-practices.html#global-state

[yongruilin]

/sig instrumentation

[richabanker]

/triage accepted /assign @dgrisonnet