search

AdWerx / charts

helm charts maintained by Adwerx Engineering
MIT License
25 stars 34 forks source link

How to access AWX after installation with helm #38

Closed alokispandey closed 2 years ago

alokispandey commented 2 years ago

I am new to K8s env but i manage to install awx via helm3 on 3 node K8s cluster (installed via kubeadm) bleow are my pods/deployment.

[root@k8smaster1 pgsql]# kubectl get all -A
NAMESPACE     NAME                                    READY   STATUS    RESTARTS   AGE
awx           pod/awx-demo-64788d7b46-857n9           2/3     Running   0          9m23s
awx           pod/awx-demo-postgresql-0               1/1     Running   0          5m15s
kube-system   pod/coredns-64897985d-5th6r             1/1     Running   0          31m
kube-system   pod/coredns-64897985d-8pjn7             1/1     Running   0          31m
kube-system   pod/etcd-k8smaster                      1/1     Running   6          32m
kube-system   pod/kube-apiserver-k8smaster            1/1     Running   6          32m
kube-system   pod/kube-controller-manager-k8smaster   1/1     Running   1          32m
kube-system   pod/kube-flannel-ds-9m76k               1/1     Running   0          30m
kube-system   pod/kube-flannel-ds-k4r7n               1/1     Running   0          30m
kube-system   pod/kube-flannel-ds-rbbq2               1/1     Running   0          30m
kube-system   pod/kube-proxy-6svdn                    1/1     Running   0          30m
kube-system   pod/kube-proxy-bp2sk                    1/1     Running   0          31m
kube-system   pod/kube-proxy-nrnv5                    1/1     Running   0          30m
kube-system   pod/kube-scheduler-k8smaster            1/1     Running   6          32m

NAMESPACE     NAME                                   TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)                  AGE
awx           service/awx-demo                       ClusterIP   10.102.233.80    <none>        8090/TCP                 9m23s
awx           service/awx-demo-postgresql            ClusterIP   10.104.122.197   <none>        5432/TCP                 9m23s
awx           service/awx-demo-postgresql-headless   ClusterIP   None             <none>        5432/TCP                 9m23s
default       service/kubernetes                     ClusterIP   10.96.0.1        <none>        443/TCP                  32m
kube-system   service/kube-dns                       ClusterIP   10.96.0.10       <none>        53/UDP,53/TCP,9153/TCP   32m

NAMESPACE     NAME                             DESIRED   CURRENT   READY   UP-TO-DATE   AVAILABLE   NODE SELECTOR            AGE
kube-system   daemonset.apps/kube-flannel-ds   3         3         3       3            3           <none>                   30m
kube-system   daemonset.apps/kube-proxy        3         3         3       3            3           kubernetes.io/os=linux   32m

NAMESPACE     NAME                       READY   UP-TO-DATE   AVAILABLE   AGE
awx           deployment.apps/awx-demo   0/1     1            0           9m23s
kube-system   deployment.apps/coredns    2/2     2            2           32m

NAMESPACE     NAME                                  DESIRED   CURRENT   READY   AGE
awx           replicaset.apps/awx-demo-64788d7b46   1         1         0       9m23s
kube-system   replicaset.apps/coredns-64897985d     2         2         2       31m

NAMESPACE   NAME                                   READY   AGE
awx         statefulset.apps/awx-demo-postgresql   1/1     9m23s

Not what next? how i can access AWX-GUI on cluaster nodes/remote ? Assist with steps, please.

jbielick commented 2 years ago

You'll need to set ingress.enabled=true in your chart values as it is disabled by default. After that, access it depends on your ingress controller and cluster setup.

alokispandey commented 2 years ago

Do we need ingress ? can't we use nodePort ? all i am trying at moment is to access the AWX GUI from any GUI of cluster-node?

alokispandey commented 2 years ago

Deployment is failing with error.

Below are the steps followed to deploy step1: create "storage class" , "PV" and "pvc" sample .yaml storage calss:

cat step1-1sc.yaml
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: helmpgsql
provisioner: kubernetes.io/no-provisioner
volumeBindingMode: WaitForFirstConsumer

Sample PV.yaml

cat step1-2pv.yaml
apiVersion: v1
kind: PersistentVolume
metadata:
  name: helm-awxpv
spec:
  accessModes:
    - ReadWriteOnce
  persistentVolumeReclaimPolicy: Delete
  volumeMode: Filesystem
  capacity:
    storage: 10Gi
  storageClassName: helmawx
  local:
    path: /opt/helm/deploy/awx
  nodeAffinity:
    required:
      nodeSelectorTerms:
      - matchExpressions:
        - key: kubernetes.io/hostname
          operator: In
          values:
          - k8sworker2
          - k8sworker3
          - k8smaster1

Sample pvc.yaml

 cat ../pgsql/pvc.yaml
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  finalizers:
  - kubernetes.io/pvc-protection
  labels:
    app.kubernetes.io/instance: awx-demo
    app.kubernetes.io/name: postgresql
    role: primary
  name: data-awx-demo-postgresql-0
  namespace: awx
spec:
  accessModes:
  - ReadWriteOnce
  resources:
    requests:
      storage: 8Gi
  volumeMode: Filesystem
  storageClassName: "helmpgsql"

Staus of SC,PV and pVC

 kubectl get sc,pv,pvc
NAME                                    PROVISIONER                    RECLAIMPOLICY   VOLUMEBINDINGMODE      ALLOWVOLUMEEXPANSION   AGE
storageclass.storage.k8s.io/helmawx     kubernetes.io/no-provisioner   Delete          WaitForFirstConsumer   false                  6d22h
storageclass.storage.k8s.io/helmpgsql   kubernetes.io/no-provisioner   Delete          WaitForFirstConsumer   false                  6d21h

NAME                             CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS      CLAIM   STORAGECLASS   REASON   AGE
persistentvolume/helm-awxpv      10Gi       RWO            Delete           Available           helmawx                 6d21h
persistentvolume/helm-postgres   10Gi       RWO            Delete           Available           helmpgsql               52s

NAME                                               STATUS    VOLUME   CAPACITY   ACCESS MODES   STORAGECLASS   AGE
persistentvolumeclaim/data-awx-demo-postgresql-0   Pending                                      helmpgsql      41s

step2: Deploy deploy awx via helm Parameter set in awx-valuebundle.yaml

secretKey: awwsecretkey
secret_key: awxsecretkey
defaultAdminUser: admin
default_admin_user: admin
defaultAdminPassword: password
default_admin_password: password

postgresql:
  # -- Set to false if using external postgresql
  enabled: true
  storageClass: "helmpgsql"
  postgresqlUsername: awx
  postgresqlPassword: awxpostgres
  postgresqlDatabase: awx

Install AWX :

 helm install awx-demo adwerx/awx --version 3.4.2 -f awx-valuebundle.yaml
NAME: awx-demo
LAST DEPLOYED: Wed Mar 23 05:53:13 2022
NAMESPACE: awx
STATUS: deployed
REVISION: 1

Overall Status:

# kubectl get all
NAME                            READY   STATUS    RESTARTS   AGE
pod/awx-demo-64788d7b46-l6n4j   2/3     Running   0          3m6s
pod/awx-demo-postgresql-0       1/1     Running   0          3m6s

NAME                                   TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)    AGE
service/awx-demo                       ClusterIP   10.96.224.146    <none>        8090/TCP   3m6s
service/awx-demo-postgresql            ClusterIP   10.101.185.210   <none>        5432/TCP   3m6s
service/awx-demo-postgresql-headless   ClusterIP   None             <none>        5432/TCP   3m6s

NAME                       READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/awx-demo   0/1     1            0           3m6s

NAME                                  DESIRED   CURRENT   READY   AGE
replicaset.apps/awx-demo-64788d7b46   1         1         0       3m6s

NAME                                   READY   AGE
statefulset.apps/awx-demo-postgresql   1/1     3m6s

K8s cluster status:

[root@k8smaster1 awx]# kubectl get pods -n kube-system -o wide
NAME                                READY   STATUS    RESTARTS   AGE     IP             NODE         NOMINATED NODE   READINESS GATES
coredns-64897985d-5th6r             1/1     Running   0          6d22h   21.244.0.2     k8smaster    <none>           <none>
coredns-64897985d-8pjn7             1/1     Running   0          6d22h   21.244.0.3     k8smaster    <none>           <none>
etcd-k8smaster                      1/1     Running   6          6d22h   10.109.64.69   k8smaster    <none>           <none>
kube-apiserver-k8smaster            1/1     Running   6          6d22h   10.109.64.69   k8smaster    <none>           <none>
kube-controller-manager-k8smaster   1/1     Running   1          6d22h   10.109.64.69   k8smaster    <none>           <none>
kube-flannel-ds-9m76k               1/1     Running   0          6d22h   10.109.64.70   k8sworker2   <none>           <none>
kube-flannel-ds-k4r7n               1/1     Running   0          6d22h   10.109.64.69   k8smaster    <none>           <none>
kube-flannel-ds-rbbq2               1/1     Running   0          6d22h   10.109.64.71   k8sworker3   <none>           <none>
kube-proxy-6svdn                    1/1     Running   0          6d22h   10.109.64.70   k8sworker2   <none>           <none>
kube-proxy-bp2sk                    1/1     Running   0          6d22h   10.109.64.69   k8smaster    <none>           <none>
kube-proxy-nrnv5                    1/1     Running   0          6d22h   10.109.64.71   k8sworker3   <none>           <none>
kube-scheduler-k8smaster            1/1     Running   6          6d22h   10.109.64.69   k8smaster    <none>           <none>

### PROBLEMS/ERROR:

Erro1: Endpoint is not shwoing IP

[root@k8smaster1 awx]# kubectl get ep
NAME                           ENDPOINTS         AGE
awx-demo                                         4m45s
awx-demo-postgresql            21.244.1.8:5432   4m45s
awx-demo-postgresql-headless   21.244.1.8:5432   4m45s
[root@k8smaster1 awx]#

Erro2: Awx-demo pods facing error connectivity issue with db

[root@k8smaster1 awx]# kubectl logs awx-demo-64788d7b46-l6n4j task
Using /etc/ansible/ansible.cfg as config file
127.0.0.1 | FAILED! => {
    "ansible_facts": {
        "discovered_interpreter_python": "/usr/libexec/platform-python"
    },
    "changed": false,
    "elapsed": 315,
    "msg": "Timeout when waiting for awx-demo-postgresql:5432"
}
Using /etc/ansible/ansible.cfg as config file
An exception occurred during task execution. To see the full traceback, use -vvv. The error was: psycopg2.OperationalError: could not translate host name "awx-demo-postgresql" to address: Name or service not known
127.0.0.1 | FAILED! => {
    "ansible_facts": {
        "discovered_interpreter_python": "/usr/libexec/platform-python"
    },
    "changed": false,
    **"msg": "unable to connect to database: could not translate host name \"awx-demo-postgresql\" to address: Name or service not known\n"**
}
[root@k8smaster1 awx]#

additional info : core dns logs shows error: (10.100.70.135 is our internal DNS with no connectivty to lab)

 #]kubectl logs coredns-64897985d-5th6r -n kube-system
.:53
[INFO] plugin/reload: Running configuration MD5 = db32ca3650231d74073ff4cf814959a7
CoreDNS-1.8.6
linux/amd64, go1.17.1, 13a9191
[ERROR] plugin/errors: 2 6677645820154464786.4846276664823500229. HINFO: read udp 21.244.0.2:51023->10.100.70.135:53: i/o timeout
[ERROR] plugin/errors: 2 6677645820154464786.4846276664823500229. HINFO: read udp 21.244.0.2:32781->10.100.70.136:53: i/o timeout
[ERROR] plugin/errors: 2 6677645820154464786.4846276664823500229. HINFO: read udp 21.244.0.2:35187->10.100.70.136:53: i/o timeout
[ERROR] plugin/errors: 2 6677645820154464786.4846276664823500229. HINFO: read udp 21.244.0.2:40432->10.100.70.135:53: i/o timeout
[ERROR] plugin/errors: 2 6677645820154464786.4846276664823500229. HINFO: read udp 21.244.0.2:60061->10.100.70.135:53: i/o timeout
[ERROR] plugin/errors: 2 6677645820154464786.4846276664823500229. HINFO: read udp 21.244.0.2:57199->10.100.70.136:53: i/o timeout
[ERROR] plugin/errors: 2 6677645820154464786.4846276664823500229. HINFO: read udp 21.244.0.2:52308->10.100.70.135:53: i/o timeout
[ERROR] plugin/errors: 2 6677645820154464786.4846276664823500229. HINFO: read udp 21.244.0.2:38273->10.100.70.135:53: i/o timeout
[ERROR] plugin/errors: 2 6677645820154464786.4846276664823500229. HINFO: read udp 21.244.0.2:32993->10.100.70.135:53: i/o timeout
[ERROR] plugin/errors: 2 6677645820154464786.4846276664823500229. HINFO: read udp 21.244.0.2:54872->10.100.70.136:53: i/o timeout

ERROR3: can;t access awx dashboard.

Please suggest how to fix this as it seems DNS query between pods are failing..

jbielick commented 2 years ago

@alokispandey honestly DNS doesn't have much to do with this chart. The hostname your pod is looking up appears correct. Based on the output you've pasted here you could have an issue with flannel? There is no known issue with DNS and this chart. I suggest posting on Stack Overflow with some of this information and maybe a small reproducible test case. If there's an issue you identify specifically with this chart I could take another look.

alokispandey commented 2 years ago

Agreed, it apprears issue remains with flannel how it is using iptables to configure rules. Not sure at moment, but it seems my setup is hosting known bug "kube-proxy currently incompatible with iptables >= 1.8 #71305" as i can see my iptable version is same.

[root@k8sworker2 ~]# iptables -v iptables v1.8.4 (nf_tables):

thanks for your time just last Q: what will be default URL to access awx ? http://awx-demo:port/awx? of simple http://awx-demo:port/ ?