caas-team / py-kube-downscaler

Scale down / "pause" Kubernetes workload (Deployments, StatefulSets, and/or HorizontalPodAutoscalers and CronJobs too !) during non-work hours.
GNU General Public License v3.0
36 stars 14 forks source link

downscaler can't access v1/pod if there is a networkpolicy #107

Closed rkashasl closed 3 weeks ago

rkashasl commented 4 weeks ago

Hello, i got an issue running kubedownscaler with networkpolicies enabled in the cluste Here is netpol for kubedownscaler, everything else is denied by default with a default-deny rule

Namespace:    kube-system
Created on:   2024-10-23 11:14:36 +0300 EEST
Labels:       kustomize.toolkit.fluxcd.io/name=default-networkpolicies
              kustomize.toolkit.fluxcd.io/namespace=flux-system
Annotations:  <none>
Spec:
  PodSelector:     <none> (Allowing the specific traffic to all pods in this namespace)
  Allowing ingress traffic:
    <none> (Selected pods are isolated for ingress connectivity)
  Allowing egress traffic:
    <none> (Selected pods are isolated for egress connectivity)
  Policy Types: Ingress, Egress
Namespace:    kube-system
Created on:   2024-10-23 11:14:36 +0300 EEST
Labels:       kustomize.toolkit.fluxcd.io/name=default-networkpolicies
              kustomize.toolkit.fluxcd.io/namespace=flux-system
Annotations:  <none>
Spec:
  PodSelector:     app.kubernetes.io/instance=kube-downscaler,app.kubernetes.io/name=kube-downscaler
  Allowing ingress traffic:
    <none> (Selected pods are isolated for ingress connectivity)
  Allowing egress traffic:
    To Port: 443/TCP
    To:
      IPBlock:
        CIDR: 172.20.0.1/32
        Except:
  Policy Types: Egress, Ingress

However i get an error

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/kube_downscaler/main.py", line 72, in run_loop
    scale(
  File "/kube_downscaler/scaler.py", line 1163, in scale
    forced_uptime = pods_force_uptime(api, namespace)
  File "/kube_downscaler/scaler.py", line 127, in pods_force_uptime
    for pod in pykube.Pod.objects(api).filter(namespace=(namespace or pykube.all)):
  File "/usr/local/lib/python3.10/site-packages/pykube/query.py", line 197, in __iter__
    return iter(self.query_cache["objects"])
  File "/usr/local/lib/python3.10/site-packages/pykube/query.py", line 187, in query_cache
    cache["response"] = self.execute().json()
  File "/usr/local/lib/python3.10/site-packages/pykube/query.py", line 161, in execute
    r = self.api.get(**kwargs)
  File "/usr/local/lib/python3.10/site-packages/pykube/http.py", line 465, in get
    return self.session.get(*args, **self.get_kwargs(**kwargs))
  File "/usr/local/lib/python3.10/site-packages/requests/sessions.py", line 602, in get
    return self.request("GET", url, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/requests/sessions.py", line 589, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/local/lib/python3.10/site-packages/requests/sessions.py", line 703, in send
    r = adapter.send(request, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/pykube/http.py", line 181, in send
    response = self._do_send(request, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/requests/adapters.py", line 688, in send
    raise ConnectTimeout(e, request=request)
requests.exceptions.ConnectTimeout: HTTPSConnectionPool(host='172.20.0.1', port=443): Max retries exceeded with url: /api/v1/pods (Caused by ConnectTimeoutError(<urllib3.connection.HTTPSConnection object at 0xffffa17eae60>, 'Connection to 172.20.0.1 timed out. (connect timeout=10)'))

Could you please give me exact ports needs to be open and to which endpoints? My netpol allows 443 at 172.20.0.1/32 which should be enough to access v1/pods i guess Thank you

rkashasl commented 3 weeks ago

Problem solved by upgrading from 0.2.4 to 0.2.10