Frontend doesn't work

[Reign1]

Environment

• Red Hat Enterprise Linux release 9.3 (Plow)
• Kubernetes v1.30.0
• AWX Operator 2.17.0

Description

As per https://github.com/kurokobo/awx-on-k3s setup everything and it seems to be fine. Play recap output:

PLAY RECAP ***** localhost : ok=90 changed=0 unreachable=0 failed=0 skipped=82 rescued=0 ignored=1

Status of AWX resources:

pod/awx-migration-24.4.0-rv74w 0/1 Completed 0 100m pod/awx-operator-controller-manager-5b9cb84bd5-g54xx 2/2 Running 2 107m pod/awx-postgres-15-0 1/1 Running 1 101m pod/awx-task-6f65778bd-wwzld 4/4 Running 4 100m pod/awx-web-988fccf6d-w5pz2 3/3 Running 3 100m service/awx-operator-controller-manager-metrics-service ClusterIP 10.99.254.163 8443/TCP 107m service/awx-postgres-15 ClusterIP None 5432/TCP 101m service/awx-service ClusterIP 10.111.108.153 80/TCP 100m deployment.apps/awx-operator-controller-manager 1/1 1 1 107m deployment.apps/awx-task 1/1 1 1 100m deployment.apps/awx-web 1/1 1 1 100m replicaset.apps/awx-operator-controller-manager-5b9cb84bd5 1 1 1 107m replicaset.apps/awx-task-6f65778bd 1 1 1 100m replicaset.apps/awx-web-988fccf6d 1 1 1 100m statefulset.apps/awx-postgres-15 1/1 101m job.batch/awx-migration-24.4.0 Complete 1/1 112s 100m

However when trying to access frontend via https://awx.tia.eu it doesn't show the AWX page.

The only output from the logs I dont get is this:

kubectl -n awx logs -f deployment/awx-task -c init error: container init is not valid for pod awx-task-6f65778bd-wwzld

kubectl -n awx logs -f job/awx-migration-24.4.0 unable to retrieve container logs for cri-o://3289ab0ca5228c3e49691db3598461bcb30961a2cf3f01f5c5a472e648f840d7

All other show nothing suspicious info.

[jpmens]

If it's any consolation there appears to be no process listening there:

$ curl -v https://awx.tia.eu
* Host awx.tia.eu:443 was resolved.
* IPv6: (none)
* IPv4: 130.61.41.227
*   Trying 130.61.41.227:443...
* connect to 130.61.41.227 port 443 from 192.168.1.140 port 64899 failed: Connection refused
* Failed to connect to awx.tia.eu port 443 after 612 ms: Couldn't connect to server
* Closing connection
curl: (7) Failed to connect to awx.tia.eu port 443 after 612 ms: Couldn't connect to server

Have you verified:

• the domain is actually connected
• there is no firewall in place
• you can connect to :443 locally from your infra

[jpmens]

This looks weird:

kubectl -n awx logs -f deployment/awx-task -c init
error: container init is not valid for pod awx-task-6f65778bd-wwzld

"container init" ?

[kurokobo]

Hi, could you please provide the output from:

kubectl -n awx get awx,all,ingress,secrets

Also try accessing your AWX from your K3s host (replace FQDN and IP address with your actual values):

# Determine the IP address of the `awx-ingress` ingress
$ kubectl -n awx get ingress
NAME          CLASS     HOSTS             ADDRESS         PORTS     AGE
awx-ingress   traefik   awx.example.com   192.168.0.219   80, 443   2m52s

# Send HTTP request with --resolve to imitate DNS name resolution. Expected output is:
$ curl --resolve awx.example.com:80:192.168.0.219 http://awx.example.com/api/
{"description":"AWX REST API","current_version":"/api/v2/","available_versions":{"v2":"/api/v2/"},"oauth2":"/api/o/","custom_logo":"","custom_login_info":"","login_redirect_override":""}

# Send HTTPS request with --resolve to imitate DNS name resolution. Expected output is:
$ curl -k --resolve awx.example.com:443:192.168.0.219 https://awx.example.com/api/
{"description":"AWX REST API","current_version":"/api/v2/","available_versions":{"v2":"/api/v2/"},"oauth2":"/api/o/","custom_logo":"","custom_login_info":"","login_redirect_override":""}

# Ensure HTTPS connection is established with correct certificate (-v is appended).
$ curl -kv --resolve awx.example.com:443:192.168.0.219 https://awx.example.com/api/
* Added awx.example.com:443:192.168.0.219 to DNS cache
* Hostname awx.example.com was found in DNS cache
*   Trying 192.168.0.219...
* TCP_NODELAY set
* Connected to awx.example.com (192.168.0.219) port 443 (#0)
...
* Server certificate:
*  subject: CN=awx.example.com
*  start date: Sep  7 12:08:42 2023 GMT
*  expire date: Dec  6 12:08:41 2023 GMT
*  issuer: C=US; O=(STAGING) Let's Encrypt; CN=(STAGING) Artificial Apricot R3
*  SSL certificate verify result: unable to get local issuer certificate (20), continuing anyway.
...

Then on your client, test your name resolution with dig or nslookup:

$ nslookup awx.example.com
...
Name:   awx.example.com
Address: 192.168.0.219

Finally test your AWX from your client:

# Test with explicitly specified IP address
$ curl -kv --resolve awx.example.com:443:192.168.0.219 https://awx.example.com/api/

# Final test
$ curl -v https://awx.example.com/api/

[kurokobo]

kubectl -n awx logs -f deployment/awx-task -c init

Sorry it's my fault, I've forgot to update the name of the init containers on this page: https://github.com/kurokobo/awx-on-k3s/blob/main/tips/troubleshooting.md I will fix this page, but in any case, I don't think the init container has much to do with this issue.

EDIT: Updated the commands on the page

[github-actions[bot]]

This issue is stale because it has been open 10 days with no activity. Remove stale label or comment or this will be closed in 4 days.

[github-actions[bot]]

This issue was closed because it has been open 2 weeks with no activity.