search

kurokobo / awx-on-k3s

An example implementation of AWX on single node K3s using AWX Operator, with easy-to-use simplified configuration with ownership of data and passwords.
MIT License
521 stars 146 forks source link

Not able to login with external DB setup #325

Closed Saravanaselvaraj closed 3 months ago

Saravanaselvaraj commented 3 months ago

Environment

Description

Hi, I am trying to setup AWX with external DB setup with 3 web and task replicas. I tried to follow the same approach as it is mentioned in the link "https://github.com/kurokobo/awx-on-k3s/blob/main/tips/external-db.md" and the mainguide but I am not able to login post installation. I can see 3 copies of task and web replicas running but I am not able to login with the password specified. Please check the attachments for the yaml files I have used. Please help me out in troubleshooting this issue.

Step to Reproduce

  1. Follow the document main.md and external-db.md
  2. URL works but not able to login

Logs

$ kubectl ...
...

Files

awx.yaml
---
apiVersion: awx.ansible.com/v1beta1
kind: AWX
metadata:
  name: awx
spec:
  # These parameters are designed for use with:
  # - AWX Operator: 2.0.0
  #   https://github.com/ansible/awx-operator/blob/2.0.0/README.md
  # - AWX: 22.0.0
  #   https://github.com/ansible/awx/blob/22.0.0/INSTALL.md

  admin_user: admin
  admin_password_secret: awx-admin-password
 #admin_password_secret: dev-awx-admin-password
  ingress_type: ingress
  ingress_hosts:
    - hostname: XXXXXX:
  #  tls_secret: awx-secret-tls

  postgres_configuration_secret: awx-postgres-configuration
  # Comment below lines for external DB configuration
  #postgres_storage_class: awx-postgres-volume
  #postgres_storage_requirements:
  #  requests:
  #    storage: 8Gi

  projects_persistence: true
  projects_existing_claim: awx-projects-claim
  replicas: 3
  web_replicas: 3
  task_replicas: 3

  web_resource_requirements: {}
  task_resource_requirements: {}
  ee_resource_requirements: {}
  init_container_resource_requirements: {}
  postgres_init_container_resource_requirements: {}
  postgres_resource_requirements: {}
  redis_resource_requirements: {}
  rsyslog_resource_requirements: {}

  # Uncomment to reveal "censored" logs
  #no_log: false
*********************************************************************
kustomization.yaml
---
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
namespace: awx

generatorOptions:
  disableNameSuffixHash: true

secretGenerator:
  - name: awx-secret-tls
    type: kubernetes.io/tls
    files:
      - tls.crt
      - tls.key

  - name: awx-postgres-configuration
    type: Opaque
    literals:
     # - host=awx-postgres-13
     # - port=5432
     # - database=awx
     # - username=awx
     # - password=Ansible123!
     # - type=managed
       - host=1XXXXXXXX
       - port=5432
       - database=awx
       - username=awx_usr
       - password=cola4Y_pKdu2y
       - sslmode=prefer
       - type=unmanaged

  - name: awx-admin-password
    type: Opaque
    literals:
       - password=Ansibleqa123!

  # If you want to specify SECRET_KEY for your AWX manually, uncomment following lines and change the value.
  # Refer AAC documentation for detail about SECRET_KEY.
  # https://docs.ansible.com/automation-controller/latest/html/administration/secret_handling.html
  #- name: awx-secret-key
  #  type: Opaque
  #  literals:
  #    - secret_key=dev-awx-admin-password

resources:
  - pv.yaml
  - pvc.yaml
  - awx.yaml
*********************************************************************
pv.yaml
#External DB Configuration
##---
#apiVersion: v1
#kind: PersistentVolume
#metadata:
#  name: awx-postgres-13-volume
#spec:
#  accessModes:
#    - ReadWriteOnce
#  persistentVolumeReclaimPolicy: Retain
#  capacity:
#    storage: 8Gi
#  storageClassName: awx-postgres-volume
#  hostPath:
#    path: /data/postgres-13

---
apiVersion: v1
kind: PersistentVolume
metadata:
  name: awx-projects-volume
spec:
  accessModes:
    - ReadWriteOnce
  persistentVolumeReclaimPolicy: Retain
  capacity:
    storage: 2Gi
  storageClassName: awx-projects-volume
  hostPath:
    path: /data/projects
kurokobo commented 3 months ago

@Saravanaselvaraj Hi, thanks for using my guide. Let me introduce some points for further investigation.

  1. Try using password used in the first deployments
    • AWX Operator does not update password if the admin user already exists in DB. Since you are using external DB, If you are retrying multiple deployments using the same DB, the admin user in the DB may have the password specified for the very first deployment.
    • So you can try using password used in the first deployment.
  2. Retry deployment after wiping DB
    • For the same reason as above, wiping the DB before deployment will reflect the password specified in the secret.
    • So you can try wiping DB and redeploy AWX.
  3. Reset admin password
    • You can reset admin password by following command. 
      $ kubectl -n awx exec -it deployment/awx-task -- awx-manage changepassword admin
Changing password for user 'admin'
Password: 
Password (again): 
Password changed successfully for user 'admin'
  4. Investigate logs from web pod
    • If you cannot log in, your password may be incorrect or your CSRF Trusted Origins may be incorrectly configured.
    • Monitor logs from web pod and try logging in again, then check the error log displayed at login.
    • kubectl -n awx logs -f deployment/awx-web -c awx-web
Saravanaselvaraj commented 3 months ago

Hello @kurokobo ,

Thanks for the reply. Yes I did some R&D in the beginning with this password thing.

  1. I do not want to expose the password for admin user and my awxdb user in kustomization.yaml, hence I created a secret.yaml , Converted my password to base64 using echo password | base64 , I tried to convert the password (Ansibleqa123!) '''

    apiVersion: v1 kind: Secret metadata: name: dev-awx-admin-password namespace: awx type: Opaque data: password: QW5zaWJsZXFhMTIzIQo= '''

Initially i set this secret in awx.yaml, as admin_password_secret: dev-awx-admin-password But it did not work, And then I specified this in kustomization.yaml like this below , But still did not work

If this would have worked, I wanted to try the same with DB password too. Can you please let me know what was wrong here ? Not sure What I was missing but then I have decided to follow your approach. I will try resetting the password and will let you know.

Thanks and Regards

Saravanaselvaraj commented 3 months ago

Hello @kurokobo , I confirm that step 3 (Changing the password) trick worked.

Thank you so much, But please can respond how we can achieve above configuration ?

Thanks and Regards

kurokobo commented 3 months ago

@Saravanaselvaraj

using echo password | base64

Your approach is correct, but above command is the cause. You have to use echo -n instead of echo, since echo without -n appends a trailing line break (\n), which causes your password to include trailing line break.

AWX Operator does not trim trailing line breaks in custom password, so inputting Ansibleqa123! in web UI never match with actual password that contains trailing line break. This is why you can't login with the password Ansibleqa123!.

# By decoding encoded string from your comment, we can see trailing `0x0a` (means `\n`)
$ echo "QW5zaWJsZXFhMTIzIQo=" | base64 -d | od -tx1z -Ax
000000 41 6e 73 69 62 6c 65 71 61 31 32 33 21 0a        >Ansibleqa123!.<
00000e

# By comparing `echo` and `echo -n`, we can see a little difference (trailing `o=` vs. `==`)
$ echo "Ansibleqa123!" | base64
QW5zaWJsZXFhMTIzIQo=
$ echo -n "Ansibleqa123!" | base64
QW5zaWJsZXFhMTIzIQ==

# By deconding strings encoded by `echo -n`, we can see there is no trailing new line (`0x0a`)
$ echo "QW5zaWJsZXFhMTIzIQ==" | base64 -d | od -tx1z -Ax
000000 41 6e 73 69 62 6c 65 71 61 31 32 33 21           >Ansibleqa123!<
00000d
Saravanaselvaraj commented 3 months ago

Thank you @kurokobo for the response.You are right, Looks like that converting command caused problems. By the way Which one is correct ? To confirm again, Adding this line "admin_password_secret: dev-awx-admin-password" in awx.yaml and specifying in kustomization.yaml will work as expected right ?

And if I want to specify the same for DB password as well , Do I have to follow 7.2 in this link https://access.redhat.com/documentation/en-us/red_hat_ansible_automation_platform/2.4/html/red_hat_ansible_automation_platform_operations_guide/encrypting-plaintext-passwords ? Please confirm if we have only this way of encryption for this or can i use secret.yaml reference for this password too ?

kurokobo commented 3 months ago

@Saravanaselvaraj

Adding this line "admin_password_secret: dev-awx-admin-password" in awx.yaml and specifying in kustomization.yaml will work as expected right ?

Adding the line admin_password_secret: dev-awx-admin-password in awx.yaml is enough to specify custom password for admin user. awx-secret-key is not for a secret for a password, but a key to encrypt/decryot credentials in DB, so in most cases you don't need to modify this in kustomization.yaml.

And if I want to specify the same for DB password as well , Do I have to follow 7.2 in this link https://access.redhat.com/documentation/en-us/red_hat_ansible_automation_platform/2.4/html/red_hat_ansible_automation_platform_operations_guide/encrypting-plaintext-passwords ? Please confirm if we have only this way of encryption for this or can i use secret.yaml reference for this password too ?

7.2 is not required. You can apply the same manner as admin password; just create following secret for example, and specify it in awx.yaml as postgres_configuration_secret: dev-awx-postgres-configuration.

apiVersion: v1
kind: Secret
metadata:
  namespace: awx
  name: dev-awx-postgres-configuration
type: Opaque
data:
  host: cG9zdGdyZXMuZXhhbXBsZS5pbnRlcm5hbA==   # echo -n "postgres.example.internal" | base64
  port: NTQzMg==                               # echo -n "5432" | base64
  database: YXd4                               # echo -n "awx" | base64
  username: YXd4                               # echo -n "awx" | base64
  password: UG9zdGdyZVNRTDEyMyE=               # echo -n "PostgreSQL123!" | base64
  sslmode: cHJlZmVy                            # echo -n "prefer" | base64
  type: dW5tYW5hZ2Vk                           # echo -n "unmanaged" | base64
Saravanaselvaraj commented 3 months ago

Thanks @kurokobo , I will try this today. But the order would be first apply secret.yaml and then kustomisation.yaml( and internally AWX, pv and PVC are being called) . Am I right ?

Or I can apply using -k base as mentioned in the doc ?

kurokobo commented 3 months ago

But the order would be first apply secret.yaml and then kustomisation.yaml( and internally AWX, pv and PVC are being called) . Am I right ?

Correct.

Or I can apply using -k base as mentioned in the doc ?

If you want to use -k base to deploy AWX with your customized Secrets at the same time, place your secrets.yaml under base and add - secrets.yaml under resources: in kustomization.yaml.

Saravanaselvaraj commented 3 months ago

Thank you. It is pretty clear.