Describe the bug
We use OIDC enabled K8s cluster, we rotate OIDC issuer key after few months. Post rotation of the key whereabouts cannot allocate IP for newly created pods.
whereabouts.log
2024-01-02T09:06:16Z [debug] Used defaults from parsed flat file config @ /etc/cni/net.d/whereabouts.d/whereabouts.conf
2024-01-02T09:06:16Z [debug] ADD - IPAM configuration successfully read: {Name:test-sriov Type:whereabouts xxx Routes:[] GatewayStr: LeaderLeaseDuration:1500 LeaderRenewDeadline:1000 LeaderRetryPeriod:500 LogFile:/var/log/whereabouts.log LogLevel:info ReconcilerCronExpression:30 4 OverlappingRanges:true SleepForRace:0 Gateway: Kubernetes:{KubeConfigPath:/etc/cni/net.d/whereabouts.d/whereabouts.kubeconfig K8sAPIRoot:} ConfigurationPath: PodName:test-pod PodNamespace:default NetworkName:}
2024-01-02T09:06:16Z [debug] Beginning IPAM for ContainerID: 21c1209c11d5075e50ae42216e5cc633c6f9d75524eac31f883b28f81f710f21
2024-01-02T09:06:16Z [debug] Started leader election
2024-01-02T09:08:16Z [debug] OnStoppedLeading() called
2024-01-02T09:08:16Z [debug] Finished leader election
2024-01-02T09:08:16Z [debug] IPManagement: [], time limit exceeded while waiting to become leader*
2024-01-02T09:08:16Z [error] Error at storage engine: time limit exceeded while waiting to become leader
Restarting whereabouts pods help mitigate this problem.
Expected behavior
A clear and concise description of what you expected to happen.
Looking for sample config to allow whereabouts use in-pod service account token instead of kubeconfig. Service account token refreshes hourly. (we faced similar issue with Multus, and switched to multus-thick deployment, it helped!)
To Reproduce
Steps to reproduce the behavior:
Rotate oidc issuer key twice.
Create pods that use whereabouts for IP allocation
Environment: Linux
Whereabouts version : v0.6.2
Kubernetes version (use kubectl version): 1.26.3
Network-attachment-definition: N/A
Whereabouts configuration (on the host):
{ "datastore": "kubernetes", "kubernetes": { "kubeconfig": "/etc/cni/net.d/whereabouts.d/whereabouts.kubeconfig" }, "reconciler_cron_expression": "30 4 * * *" }
OS (e.g. from /etc/os-release): Ubuntu 20.04
Kernel (e.g. uname -a): 5.15.0-1045
Others: N/A
Additional info / context
Add any other information / context about the problem here.
Describe the bug We use OIDC enabled K8s cluster, we rotate OIDC issuer key after few months. Post rotation of the key whereabouts cannot allocate IP for newly created pods.
whereabouts.log
2024-01-02T09:06:16Z [debug] Used defaults from parsed flat file config @ /etc/cni/net.d/whereabouts.d/whereabouts.conf 2024-01-02T09:06:16Z [debug] ADD - IPAM configuration successfully read: {Name:test-sriov Type:whereabouts xxx Routes:[] GatewayStr: LeaderLeaseDuration:1500 LeaderRenewDeadline:1000 LeaderRetryPeriod:500 LogFile:/var/log/whereabouts.log LogLevel:info ReconcilerCronExpression:30 4 OverlappingRanges:true SleepForRace:0 Gateway: Kubernetes:{KubeConfigPath:/etc/cni/net.d/whereabouts.d/whereabouts.kubeconfig K8sAPIRoot:} ConfigurationPath: PodName:test-pod PodNamespace:default NetworkName:}
2024-01-02T09:06:16Z [debug] Beginning IPAM for ContainerID: 21c1209c11d5075e50ae42216e5cc633c6f9d75524eac31f883b28f81f710f21
2024-01-02T09:06:16Z [debug] Started leader election
2024-01-02T09:08:16Z [debug] OnStoppedLeading() called
2024-01-02T09:08:16Z [debug] Finished leader election
2024-01-02T09:08:16Z [debug] IPManagement: [], time limit exceeded while waiting to become leader*
2024-01-02T09:08:16Z [error] Error at storage engine: time limit exceeded while waiting to become leader
Restarting whereabouts pods help mitigate this problem.
Expected behavior A clear and concise description of what you expected to happen. Looking for sample config to allow whereabouts use in-pod service account token instead of kubeconfig. Service account token refreshes hourly. (we faced similar issue with Multus, and switched to multus-thick deployment, it helped!)
To Reproduce Steps to reproduce the behavior:
Environment: Linux
kubectl version): 1.26.3{ "datastore": "kubernetes", "kubernetes": { "kubeconfig": "/etc/cni/net.d/whereabouts.d/whereabouts.kubeconfig" }, "reconciler_cron_expression": "30 4 * * *" }uname -a): 5.15.0-1045Additional info / context Add any other information / context about the problem here.