[cni-metrics-helper] feat: Expose container port when enabling prometheus metrics

toVersus commented 5 months ago

Issue

xref: https://github.com/aws/amazon-vpc-cni-k8s/pull/2603

To scrape Prometheus metrics using the Prometheus Operator's PodMonitor, container ports must be exposed via PodSpec:

the port must be specified with container port property.

apiVersion: monitoring.coreos.com/v1
kind: PodMonitor
metadata:
  name: cni-metrics-helper
  namespace: kube-system
spec:
  podMetricsEndpoints:
    - interval: 60s
      port: metrics # or 61681
      scrapeTimeout: 10s
  selector:
    matchLabels:
      k8s-app: cni-metrics-helper

Description of changes

If we set env.USE_PROMETHEUS to true, the metrics port is defined in the container's port within PodSpec. The port number of cni-metrics-helper cannot be modified, so we can hardcode the port number in Helm chart.

Checklist

[x] Added/modified documentation as required (such as the README.md for modified charts)
[ ] Incremented the chart version in Chart.yaml for the modified chart(s)
[x] Manually tested. Describe what testing was done in the testing section below
[x] Make sure the title of the PR is a good description that can go into the release notes

Testing

Setting env.USE_PROMETHEUS to true:

helm template cni-metrics-helper . --set-string env.USE_PROMETHEUS="true"

```yaml --- # Source: cni-metrics-helper/templates/serviceaccount.yaml apiVersion: v1 kind: ServiceAccount metadata: name: cni-metrics-helper namespace: default labels: helm.sh/chart: cni-metrics-helper-1.18.0 app.kubernetes.io/name: cni-metrics-helper app.kubernetes.io/instance: cni-metrics-helper app.kubernetes.io/version: "v1.18.0" app.kubernetes.io/managed-by: Helm --- # Source: cni-metrics-helper/templates/clusterrole.yaml apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRole metadata: name: cni-metrics-helper rules: - apiGroups: [""] resources: - pods - pods/proxy verbs: ["get", "watch", "list"] --- # Source: cni-metrics-helper/templates/clusterrolebinding.yaml apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRoleBinding metadata: name: cni-metrics-helper labels: helm.sh/chart: cni-metrics-helper-1.18.0 app.kubernetes.io/name: cni-metrics-helper app.kubernetes.io/instance: cni-metrics-helper app.kubernetes.io/version: "v1.18.0" app.kubernetes.io/managed-by: Helm roleRef: apiGroup: rbac.authorization.k8s.io kind: ClusterRole name: cni-metrics-helper subjects: - kind: ServiceAccount name: cni-metrics-helper namespace: default --- # Source: cni-metrics-helper/templates/deployment.yaml kind: Deployment apiVersion: apps/v1 metadata: name: cni-metrics-helper namespace: default labels: k8s-app: cni-metrics-helper spec: revisionHistoryLimit: 10 selector: matchLabels: k8s-app: cni-metrics-helper template: metadata: labels: k8s-app: cni-metrics-helper spec: containers: - env: - name: AWS_CLUSTER_ID value: "" - name: AWS_VPC_K8S_CNI_LOGLEVEL value: "INFO" - name: USE_CLOUDWATCH value: "true" - name: USE_PROMETHEUS value: "true" name: cni-metrics-helper image: "602401143452.dkr.ecr.us-west-2.amazonaws.com/cni-metrics-helper:v1.18.0" ports: - containerPort: 61681 name: metrics serviceAccountName: cni-metrics-helper ```

Setting env.USE_PROMETHEUS to false:

helm template cni-metrics-helper . --set-string env.USE_PROMETHEUS="false"

```yaml --- # Source: cni-metrics-helper/templates/serviceaccount.yaml apiVersion: v1 kind: ServiceAccount metadata: name: cni-metrics-helper namespace: default labels: helm.sh/chart: cni-metrics-helper-1.18.0 app.kubernetes.io/name: cni-metrics-helper app.kubernetes.io/instance: cni-metrics-helper app.kubernetes.io/version: "v1.18.0" app.kubernetes.io/managed-by: Helm --- # Source: cni-metrics-helper/templates/clusterrole.yaml apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRole metadata: name: cni-metrics-helper rules: - apiGroups: [""] resources: - pods - pods/proxy verbs: ["get", "watch", "list"] --- # Source: cni-metrics-helper/templates/clusterrolebinding.yaml apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRoleBinding metadata: name: cni-metrics-helper labels: helm.sh/chart: cni-metrics-helper-1.18.0 app.kubernetes.io/name: cni-metrics-helper app.kubernetes.io/instance: cni-metrics-helper app.kubernetes.io/version: "v1.18.0" app.kubernetes.io/managed-by: Helm roleRef: apiGroup: rbac.authorization.k8s.io kind: ClusterRole name: cni-metrics-helper subjects: - kind: ServiceAccount name: cni-metrics-helper namespace: default --- # Source: cni-metrics-helper/templates/deployment.yaml kind: Deployment apiVersion: apps/v1 metadata: name: cni-metrics-helper namespace: default labels: k8s-app: cni-metrics-helper spec: revisionHistoryLimit: 10 selector: matchLabels: k8s-app: cni-metrics-helper template: metadata: labels: k8s-app: cni-metrics-helper spec: containers: - env: - name: AWS_CLUSTER_ID value: "" - name: AWS_VPC_K8S_CNI_LOGLEVEL value: "INFO" - name: USE_CLOUDWATCH value: "true" - name: USE_PROMETHEUS value: "false" name: cni-metrics-helper image: "602401143452.dkr.ecr.us-west-2.amazonaws.com/cni-metrics-helper:v1.18.0" serviceAccountName: cni-metrics-helper ```

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

z0rc commented 3 months ago

@toVersus please resubmit the PR against https://github.com/aws/amazon-vpc-cni-k8s/tree/master/charts/cni-metrics-helper. This repo is just a downstream collection of charts from AWS projects, charts are maintained in their own repos.

toVersus commented 3 months ago

Thanks for the info! I have resubmitted the PR (https://github.com/aws/amazon-vpc-cni-k8s/pull/2957) in the amazon-vpc-cni-k8s repo.

aws / eks-charts