Open containerckf opened 9 months ago
django.db.utils.OperationalError: connection is bad: Name or service not known
seems you have connectivity issues to your DB. So you may need to take some debugging steps to see why the connections are failing
Looks like you are setting up an internal database (running as pod inside of the same cluster as awx-task/web). is that right? if so, you shouldn't need to set the postgres_configuration_secret
. Does it work fine without setting the postgres configuration?
I experience a quite similar issue when installing AWX with the operator version 2.18.0, although my setup somewhat different.
I have my Postgres in Azure behind a VNet. I mount a custom /etc/resolv.conf
through ee_extra_volume_mounts
, task_extra_volume_mounts
, init_container_extra_volume_mounts
, etc. From there (e.g. awx-web, awx-task) my Azure Postgres Flexi server is reachable.
However, the AWX migration Job does not configured to use it, and its pods apparently can't resolve the address of my Postgres (django.db.utils.OperationalError: [Errno -2] Name or service not known
).
Is there a way to configure the the migration Job to use Azure's DNS resolver?
@sasvari-attila-bosch , have you solve this issue? I'm facing the same problem
@sasvari-attila-bosch , have you solve this issue? I'm facing the same problem
@vpelagatti, I wasn't able to resolve it using version 2.18.0, and I haven't tried with 2.19.*.
Please confirm the following
Bug Summary
When installing the recent (2.7.1) version of AWX (and others) on a v1.28 EKS Cluster, AWX does not correctly initialize.
The Web pod gets stuck in a loop of trying to complete the database migration (even though the deployment is fresh). When trying to run 'awx-manage' commands from the pod there are "connection is bad: Name or service not known" errors received.
All the pods are in Running state, but the web pod target shows as Unhealthy and when trying to access the ALB endpoint, the AWX interface does not come up. This was verified by port forwarding the web pod and hitting the IP directly, confirming the ALB was routing correctly.
AWX Operator version
2.7.1
AWX version
23.3.1
Kubernetes platform
kubernetes
Kubernetes/Platform version
1.28
Modifications
no
Steps to reproduce
Installation is performed via "kustomization" and ingress YAML per outlined here.
admin_user: admin admin_password_secret: awx-admin-password
ingress_type: ingress ingress_path: "/" ingress_path_type: Prefix hostname: awx.dev.compucom.io ingress_annotations: | alb.ingress.kubernetes.io/listen-ports: '[{"HTTPS":443}]' alb.ingress.kubernetes.io/actions.redirect: "{\"Type\": \"redirect\", \"RedirectConfig\": {\"Protocol\": \"HTTPS\", \"Port\": \"443\", \"StatusCode\": \"HTTP_301\"}}" alb.ingress.kubernetes.io/scheme: internet-facing alb.ingress.kubernetes.io/target-type: ip kubernetes.io/ingress.class: alb alb.ingress.kubernetes.io/certificate-arn: arn:aws:acm:us-east-1:123456789012:certificate/XXX alb.ingress.kubernetes.io/load-balancer-attributes: "idle_timeout.timeout_seconds=360"
postgres_configuration_secret: awx-postgres-configuration
kubectl -n awx logs awx-web-7b9777b649-5sw8g [wait-for-migrations] Waiting for database migrations... [wait-for-migrations] Attempt 1 of 30 [wait-for-migrations] Waiting 0.5 seconds before next attempt [wait-for-migrations] Attempt 2 of 30 [wait-for-migrations] Waiting 1 seconds before next attempt [wait-for-migrations] Attempt 3 of 30 ...
"playbook task failed" {"level":"error","ts":"2023-09-19T12:21:02Z","msg":"Reconciler error","controller":"awx-controller","object":{"name":"awx","namespace":"awx"},"namespace":"awx","name":"awx","reconcileID":"3fbdfdb3-552a-47bd-a28a-f1dc6c1d9d9e","error":"event runner on failed","stacktrace"
django.db.utils.OperationalError: connection is bad: Name or service not known psycopg.OperationalError: connection is bad: Name or service not known