Proxy and Redis issues

[JoaoPPCastelo]

Hi all!

I was trying to deploy Authentik on a K8s cluster made of Raspberry Pis 5 (home lab) and i'm getting some issues that haven't been able to fix...

I'm using the following config

authentik:
  secret_key: "<redacted>"
  # This sends anonymous usage-data, stack traces on errors and
  # performance data to sentry.io, and is fully opt-in
  error_reporting:
    enabled: false
  postgresql:
    password: "<redacted>"

server:
  ingress:
    # Specify kubernetes ingress controller class name
    ingressClassName: traefik
    enabled: true
    hosts:
      - <redacted>
    tls:
      - secretName: authentik-tls
        hosts:
          - <redacted>
    https: true

postgresql:
  enabled: true
  auth:
    password: "<redacted>"
redis:
  enabled: false

And so far found the following errors:

• When using redis.enabled: true, the redis pod was constantly failing with the error <jemalloc>: Unsupported system page size. I found some issues related to this, like https://github.com/bitnami/containers/issues/26062 . There seems that a fix was provided on the latest images, and i started a container with the bitnami/redis:latest and at least didn't got the error and the pod was up and running, so probably authentik just needs to update to a newer container tag/ chart version?
• To avoid the previous error and try to proceed with the deployment, deployed the helm chart with redis.enabled: false and: -- (1) both on the logs for the server and worker pods, there was a {"event": "Redis Connection failed, retrying... (Timeout connecting to server)", "level": "info", "logger": "authentik.lib.config", "timestamp": 1729369834.256011} error and the pods restarted. So or there's an issue on the config that is not propagating the redis.enabled: false to all the places it needs or redis is required? -- (2) was getting the error {"error":"authentik starting","event":"failed to proxy to backend","level":"warning","logger":"authentik.router","timestamp":"2024-10-19T20:31:39Z"} on the server pod. The pods were terminated automatically and replaced by new ones, but always with the same errors

NAME                                READY   STATUS    RESTARTS        AGE
authentik-postgresql-0              1/1     Running   0               34m
authentik-server-7d7699d4d5-bsn2f   0/1     Running   3 (4m27s ago)   34m
authentik-worker-67cf9cf89-dlmzs    0/1     Running   3 (3m26s ago)   34m

Helm chart version 2024.8.3 from https://artifacthub.io/packages/helm/goauthentik/authentik Running on a k3s cluster on Raspberry Pi 5s with version v1.30.5+k3s1

Adding the logs from pods on the comments to avoid an even bigger description

But any insight on how to get those issues fixed and get Authentik running? And thank you for the support

[JoaoPPCastelo]

Logs from the server pod:

klf authentik-server-7d7699d4d5-bsn2f -n authentik
{"event":"Loaded config","level":"debug","path":"inbuilt-default","timestamp":"2024-10-19T20:25:49Z"}
{"event":"Loaded config","level":"debug","path":"/authentik/lib/default.yml","timestamp":"2024-10-19T20:25:49Z"}
{"event":"Loaded config from environment","level":"debug","timestamp":"2024-10-19T20:25:49Z"}
{"event":"not enabling debug server, set `AUTHENTIK_DEBUG` to `true` to enable it.","level":"info","logger":"authentik.go_debugger","timestamp":"2024-10-19T20:25:49Z"}
{"event":"Starting Metrics server","level":"info","listen":"0.0.0.0:9300","logger":"authentik.router.metrics","timestamp":"2024-10-19T20:25:49Z"}
{"event":"Starting HTTP server","level":"info","listen":"0.0.0.0:9000","logger":"authentik.router","timestamp":"2024-10-19T20:25:49Z"}
{"event": "Loaded config", "level": "debug", "logger": "authentik.lib.config", "timestamp": 1729369550.3001347, "file": "/authentik/lib/default.yml"}
{"event": "Loaded environment variables", "level": "debug", "logger": "authentik.lib.config", "timestamp": 1729369550.300784, "count": 42}
{"event":"Starting HTTPS server","level":"info","listen":"0.0.0.0:9443","logger":"authentik.router","timestamp":"2024-10-19T20:25:50Z"}
{"event": "Starting authentik bootstrap", "level": "info", "logger": "authentik.lib.config", "timestamp": 1729369551.2630074}
{"event": "PostgreSQL connection failed, retrying... (connection failed: connection to server at \"10.43.102.204\", port 5432 failed: Connection refused\n\tIs the server running on that host and accepting TCP/IP connections?)", "level": "info", "logger": "authentik.lib.config", "timestamp": 1729369552.265486}
{"event": "PostgreSQL connection failed, retrying... (connection failed: connection to server at \"10.43.102.204\", port 5432 failed: Connection refused\n\tIs the server running on that host and accepting TCP/IP connections?)", "level": "info", "logger": "authentik.lib.config", "timestamp": 1729369553.267583}
{"event": "PostgreSQL connection failed, retrying... (connection failed: connection to server at \"10.43.102.204\", port 5432 failed: Connection refused\n\tIs the server running on that host and accepting TCP/IP connections?)", "level": "info", "logger": "authentik.lib.config", "timestamp": 1729369554.27205}
{"event": "PostgreSQL connection failed, retrying... (connection failed: connection to server at \"10.43.102.204\", port 5432 failed: Connection refused\n\tIs the server running on that host and accepting TCP/IP connections?)", "level": "info", "logger": "authentik.lib.config", "timestamp": 1729369555.2738996}
{"event": "PostgreSQL connection failed, retrying... (connection failed: connection to server at \"10.43.102.204\", port 5432 failed: Connection refused\n\tIs the server running on that host and accepting TCP/IP connections?)", "level": "info", "logger": "authentik.lib.config", "timestamp": 1729369556.2759786}
{"event": "PostgreSQL connection failed, retrying... (connection failed: connection to server at \"10.43.102.204\", port 5432 failed: Connection refused\n\tIs the server running on that host and accepting TCP/IP connections?)", "level": "info", "logger": "authentik.lib.config", "timestamp": 1729369557.278126}
{"event": "PostgreSQL connection failed, retrying... (connection failed: connection to server at \"10.43.102.204\", port 5432 failed: Connection refused\n\tIs the server running on that host and accepting TCP/IP connections?)", "level": "info", "logger": "authentik.lib.config", "timestamp": 1729369558.2802534}
{"event": "PostgreSQL connection failed, retrying... (connection failed: connection to server at \"10.43.102.204\", port 5432 failed: Connection refused\n\tIs the server running on that host and accepting TCP/IP connections?)", "level": "info", "logger": "authentik.lib.config", "timestamp": 1729369559.2823832}
{"error":"authentik starting","event":"failed to proxy to backend","level":"warning","logger":"authentik.router","timestamp":"2024-10-19T20:25:59Z"}
{"event": "PostgreSQL connection failed, retrying... (connection failed: connection to server at \"10.43.102.204\", port 5432 failed: Connection refused\n\tIs the server running on that host and accepting TCP/IP connections?)", "level": "info", "logger": "authentik.lib.config", "timestamp": 1729369560.2849264}
{"event": "PostgreSQL connection failed, retrying... (connection failed: connection to server at \"10.43.102.204\", port 5432 failed: Connection refused\n\tIs the server running on that host and accepting TCP/IP connections?)", "level": "info", "logger": "authentik.lib.config", "timestamp": 1729369561.287439}
{"event": "PostgreSQL connection failed, retrying... (connection failed: connection to server at \"10.43.102.204\", port 5432 failed: Connection refused\n\tIs the server running on that host and accepting TCP/IP connections?)", "level": "info", "logger": "authentik.lib.config", "timestamp": 1729369562.2903624}
{"event": "PostgreSQL connection failed, retrying... (connection failed: connection to server at \"10.43.102.204\", port 5432 failed: Connection refused\n\tIs the server running on that host and accepting TCP/IP connections?)", "level": "info", "logger": "authentik.lib.config", "timestamp": 1729369563.2934241}
{"event": "PostgreSQL connection successful", "level": "info", "logger": "authentik.lib.config", "timestamp": 1729369563.3516576}
{"error":"authentik starting","event":"failed to proxy to backend","level":"warning","logger":"authentik.router","timestamp":"2024-10-19T20:26:09Z"}
{"error":"authentik starting","event":"failed to proxy to backend","level":"warning","logger":"authentik.router","timestamp":"2024-10-19T20:26:19Z"}
{"error":"authentik starting","event":"failed to proxy to backend","level":"warning","logger":"authentik.router","timestamp":"2024-10-19T20:26:29Z"}
{"error":"authentik starting","event":"failed to proxy to backend","level":"warning","logger":"authentik.router","timestamp":"2024-10-19T20:26:39Z"}
{"error":"authentik starting","event":"failed to proxy to backend","level":"warning","logger":"authentik.router","timestamp":"2024-10-19T20:26:49Z"}
{"error":"authentik starting","event":"failed to proxy to backend","level":"warning","logger":"authentik.router","timestamp":"2024-10-19T20:26:59Z"}
{"error":"authentik starting","event":"failed to proxy to backend","level":"warning","logger":"authentik.router","timestamp":"2024-10-19T20:27:09Z"}
{"error":"authentik starting","event":"failed to proxy to backend","level":"warning","logger":"authentik.router","timestamp":"2024-10-19T20:27:19Z"}
{"error":"authentik starting","event":"failed to proxy to backend","level":"warning","logger":"authentik.router","timestamp":"2024-10-19T20:27:29Z"}
{"error":"authentik starting","event":"failed to proxy to backend","level":"warning","logger":"authentik.router","timestamp":"2024-10-19T20:27:39Z"}
{"error":"authentik starting","event":"failed to proxy to backend","level":"warning","logger":"authentik.router","timestamp":"2024-10-19T20:27:49Z"}
{"error":"authentik starting","event":"failed to proxy to backend","level":"warning","logger":"authentik.router","timestamp":"2024-10-19T20:27:59Z"}
{"error":"authentik starting","event":"failed to proxy to backend","level":"warning","logger":"authentik.router","timestamp":"2024-10-19T20:28:09Z"}
{"error":"authentik starting","event":"failed to proxy to backend","level":"warning","logger":"authentik.router","timestamp":"2024-10-19T20:28:19Z"}
{"error":"authentik starting","event":"failed to proxy to backend","level":"warning","logger":"authentik.router","timestamp":"2024-10-19T20:28:29Z"}
{"error":"authentik starting","event":"failed to proxy to backend","level":"warning","logger":"authentik.router","timestamp":"2024-10-19T20:28:39Z"}
{"error":"authentik starting","event":"failed to proxy to backend","level":"warning","logger":"authentik.router","timestamp":"2024-10-19T20:28:49Z"}
{"error":"authentik starting","event":"failed to proxy to backend","level":"warning","logger":"authentik.router","timestamp":"2024-10-19T20:28:59Z"}
{"error":"authentik starting","event":"failed to proxy to backend","level":"warning","logger":"authentik.router","timestamp":"2024-10-19T20:29:09Z"}
{"error":"authentik starting","event":"failed to proxy to backend","level":"warning","logger":"authentik.router","timestamp":"2024-10-19T20:29:19Z"}
{"error":"authentik starting","event":"failed to proxy to backend","level":"warning","logger":"authentik.router","timestamp":"2024-10-19T20:29:29Z"}
{"error":"authentik starting","event":"failed to proxy to backend","level":"warning","logger":"authentik.router","timestamp":"2024-10-19T20:29:39Z"}
{"error":"authentik starting","event":"failed to proxy to backend","level":"warning","logger":"authentik.router","timestamp":"2024-10-19T20:29:49Z"}
{"error":"authentik starting","event":"failed to proxy to backend","level":"warning","logger":"authentik.router","timestamp":"2024-10-19T20:29:59Z"}
{"error":"authentik starting","event":"failed to proxy to backend","level":"warning","logger":"authentik.router","timestamp":"2024-10-19T20:30:09Z"}
{"error":"authentik starting","event":"failed to proxy to backend","level":"warning","logger":"authentik.router","timestamp":"2024-10-19T20:30:19Z"}
{"error":"authentik starting","event":"failed to proxy to backend","level":"warning","logger":"authentik.router","timestamp":"2024-10-19T20:30:29Z"}
{"event": "Redis Connection failed, retrying... (Timeout connecting to server)", "level": "info", "logger": "authentik.lib.config", "timestamp": 1729369834.256011}
{"error":"authentik starting","event":"failed to proxy to backend","level":"warning","logger":"authentik.router","timestamp":"2024-10-19T20:30:39Z"}
{"error":"authentik starting","event":"failed to proxy to backend","level":"warning","logger":"authentik.router","timestamp":"2024-10-19T20:30:49Z"}
{"error":"authentik starting","event":"failed to proxy to backend","level":"warning","logger":"authentik.router","timestamp":"2024-10-19T20:30:59Z"}
{"error":"authentik starting","event":"failed to proxy to backend","level":"warning","logger":"authentik.router","timestamp":"2024-10-19T20:31:09Z"}
{"error":"authentik starting","event":"failed to proxy to backend","level":"warning","logger":"authentik.router","timestamp":"2024-10-19T20:31:19Z"}
{"error":"authentik starting","event":"failed to proxy to backend","level":"warning","logger":"authentik.router","timestamp":"2024-10-19T20:31:29Z"}
{"error":"authentik starting","event":"failed to proxy to backend","level":"warning","logger":"authentik.router","timestamp":"2024-10-19T20:31:39Z"}
{"error":"authentik starting","event":"failed to proxy to backend","level":"warning","logger":"authentik.router","timestamp":"2024-10-19T20:31:49Z"}
{"error":"authentik starting","event":"failed to proxy to backend","level":"warning","logger":"authentik.router","timestamp":"2024-10-19T20:31:59Z"}
{"error":"authentik starting","event":"failed to proxy to backend","level":"warning","logger":"authentik.router","timestamp":"2024-10-19T20:32:09Z"}
{"error":"authentik starting","event":"failed to proxy to backend","level":"warning","logger":"authentik.router","timestamp":"2024-10-19T20:32:19Z"}
{"error":"authentik starting","event":"failed to proxy to backend","level":"warning","logger":"authentik.router","timestamp":"2024-10-19T20:32:29Z"}
{"error":"authentik starting","event":"failed to proxy to backend","level":"warning","logger":"authentik.router","timestamp":"2024-10-19T20:32:39Z"}
{"error":"authentik starting","event":"failed to proxy to backend","level":"warning","logger":"authentik.router","timestamp":"2024-10-19T20:32:49Z"}
{"error":"authentik starting","event":"failed to proxy to backend","level":"warning","logger":"authentik.router","timestamp":"2024-10-19T20:32:59Z"}
{"error":"authentik starting","event":"failed to proxy to backend","level":"warning","logger":"authentik.router","timestamp":"2024-10-19T20:33:09Z"}
{"error":"authentik starting","event":"failed to proxy to backend","level":"warning","logger":"authentik.router","timestamp":"2024-10-19T20:33:19Z"}
{"error":"authentik starting","event":"failed to proxy to backend","level":"warning","logger":"authentik.router","timestamp":"2024-10-19T20:33:29Z"}
{"error":"authentik starting","event":"failed to proxy to backend","level":"warning","logger":"authentik.router","timestamp":"2024-10-19T20:33:39Z"}
{"error":"authentik starting","event":"failed to proxy to backend","level":"warning","logger":"authentik.router","timestamp":"2024-10-19T20:33:49Z"}
{"error":"authentik starting","event":"failed to proxy to backend","level":"warning","logger":"authentik.router","timestamp":"2024-10-19T20:33:59Z"}
{"error":"authentik starting","event":"failed to proxy to backend","level":"warning","logger":"authentik.router","timestamp":"2024-10-19T20:34:09Z"}
{"error":"authentik starting","event":"failed to proxy to backend","level":"warning","logger":"authentik.router","timestamp":"2024-10-19T20:34:19Z"}
{"error":"authentik starting","event":"failed to proxy to backend","level":"warning","logger":"authentik.router","timestamp":"2024-10-19T20:34:29Z"}
{"error":"authentik starting","event":"failed to proxy to backend","level":"warning","logger":"authentik.router","timestamp":"2024-10-19T20:34:39Z"}
{"error":"authentik starting","event":"failed to proxy to backend","level":"warning","logger":"authentik.router","timestamp":"2024-10-19T20:34:49Z"}
{"error":"authentik starting","event":"failed to proxy to backend","level":"warning","logger":"authentik.router","timestamp":"2024-10-19T20:34:59Z"}
{"event": "Redis Connection failed, retrying... (Timeout connecting to server)", "level": "info", "logger": "authentik.lib.config", "timestamp": 1729370104.5920184}
{"error":"authentik starting","event":"failed to proxy to backend","level":"warning","logger":"authentik.router","timestamp":"2024-10-19T20:35:09Z"}
{"error":"authentik starting","event":"failed to proxy to backend","level":"warning","logger":"authentik.router","timestamp":"2024-10-19T20:35:19Z"}
{"error":"authentik starting","event":"failed to proxy to backend","level":"warning","logger":"authentik.router","timestamp":"2024-10-19T20:35:29Z"}
{"error":"authentik starting","event":"failed to proxy to backend","level":"warning","logger":"authentik.router","timestamp":"2024-10-19T20:35:39Z"}
{"error":"authentik starting","event":"failed to proxy to backend","level":"warning","logger":"authentik.router","timestamp":"2024-10-19T20:35:49Z"}
{"event":"shutting down webserver","level":"info","logger":"authentik.root","timestamp":"2024-10-19T20:35:49Z"}

Logs from the worker pod:

klf authentik-worker-67cf9cf89-dlmzs -n authentik
{"event": "Not running as root, disabling permission fixes", "level": "info", "logger": "bootstrap"}
{"event": "Loaded config", "level": "debug", "logger": "authentik.lib.config", "timestamp": 1729371410.6909087, "file": "/authentik/lib/default.yml"}
{"event": "Loaded environment variables", "level": "debug", "logger": "authentik.lib.config", "timestamp": 1729371410.6915574, "count": 39}
{"event": "Starting authentik bootstrap", "level": "info", "logger": "authentik.lib.config", "timestamp": 1729371411.3758287}
{"event": "PostgreSQL connection successful", "level": "info", "logger": "authentik.lib.config", "timestamp": 1729371411.392661}
{"event": "Redis Connection failed, retrying... (Timeout connecting to server)", "level": "info", "logger": "authentik.lib.config", "timestamp": 1729371681.7588427}