ERROR: must be owner of database awx

ISSUE TYPE

Bug Report

SUMMARY

I have differents errors on postgres pod after one fresh install

ENVIRONMENT

AWX version: 9.3
AWX install method: kubernetes
Ansible version: 2.9.4
Operating System: Ubuntu 18.04
Web Browser: Firefox / chromium

Helm postgresql chart: postgresql-8.3.0 (app version 11.6.0)

STEPS TO REPRODUCE

Inventory file

localhost ansible_connection=local ansible_python_interpreter="/usr/bin/env python"

[all:vars]
dockerhub_base=ansible
# Kubernetes Install
kubernetes_context=awx-prp
kubernetes_namespace=awx-prp
kubernetes_deployment_name=awx-prp
kubernetes_deployment_replica_size=1
tiller_namespace=kube-system

pg_volume_capacity=20

# Kubernetes Ingress Configuration
kubernetes_ingress_hostname=cloud-awx-z00n-prp.cloud.620nm.net
kubernetes_ingress_annotations={'kubernetes.io/ingress.class': 'nginx'}
kubernetes_ingress_tls_secret=wildcard.cloud.620nm.net

# Common Docker parameters
awx_task_hostname=awx
awx_web_hostname=awxweb
postgres_data_dir="~/.awx/pgdocker"
host_port=80
host_port_ssl=443
#ssl_certificate=
docker_compose_dir="~/.awx/awxcompose"

# Set pg_hostname if you have an external postgres server, otherwise
# a new postgres service will be created
# pg_hostname=postgresql
pg_username=awx
# pg_password should be random 10 character alphanumeric string, when postgresql is running on kubernetes
# NB: it's a limitation of the "official" postgres helm chart
pg_password="<10 character alphanumeric string >"
pg_database=awx
pg_port=5432
#pg_sslmode=require

# pg_admin_password=postgrespass

# RabbitMQ Configuration
rabbitmq_password="< rabbitmq_password >"
rabbitmq_erlang_cookie=cookiemonster

# This will create or update a default admin (superuser) account in AWX, if not provided
# then these default values are used
admin_user=admin
admin_password="< awx_password >"

# Whether or not to create preload data for demonstration purposes
create_preload_data=False

# AWX Secret key
# It's *very* important that this stay the same between upgrades or you will lose the ability to decrypt
# your credentials
secret_key=awxsecret

EXPECTED RESULTS

No error in postgres pod

ACTUAL RESULTS

On the first playbook execution, i have a failed at "kubernetes : create django super user if it does not exist" task.

TASK [kubernetes : Check for Tower Super users] *******************************************************************************************************************************************************************
fatal: [localhost]: FAILED! => {"changed": true, "cmd": "kubectl -n awx-prp exec ansible-tower-management --  bash -c \"echo 'from django.contrib.auth.models import User; nsu = User.objects.filter(is_superuser=True).count(); exit(0 if nsu > 0 else 1)' | awx-manage shell\"\n", "delta": "0:00:02.624181", "end": "2020-03-31 12:28:16.431460", "msg": "non-zero return code", "rc": 1, "start": "2020-03-31 12:28:13.807279", "stderr": "command terminated with exit code 1", "stderr_lines": ["command terminated with exit code 1"], "stdout": "", "stdout_lines": []}
...ignoring

TASK [kubernetes : create django super user if it does not exist] *************************************************************************************************************************************************
fatal: [localhost]: FAILED! => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": true}

If i re lunch the playbook, it's good for ansible, but in postgres log, i have

ERROR:  must be owner of database awx
STATEMENT:  ALTER DATABASE "awx" OWNER TO "awx"

kubectl exec -it awx-prp-postgresql-postgresql-0 -- /bin/bash
I have no name!@awx-prp-postgresql-postgresql-0:/$ 
I have no name!@awx-prp-postgresql-postgresql-0:/$ psql --user awx
Password for user awx: 
psql (11.6)
Type "help" for help.

awx=> \l
                                  List of databases
   Name    |  Owner   | Encoding |   Collate   |    Ctype    |   Access privileges   
-----------+----------+----------+-------------+-------------+-----------------------
 awx       | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 | =Tc/postgres         +
           |          |          |             |             | postgres=CTc/postgres+
           |          |          |             |             | awx=CTc/postgres
 postgres  | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 | 
 template0 | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 | =c/postgres          +
           |          |          |             |             | postgres=CTc/postgres
 template1 | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 | =c/postgres          +
           |          |          |             |             | postgres=CTc/postgres
(4 rows)

awx=> \du
                                   List of roles
 Role name |                         Attributes                         | Member of 
-----------+------------------------------------------------------------+-----------
 awx       | Create DB                                                  | {}
 postgres  | Superuser, Create role, Create DB, Replication, Bypass RLS | {}

awx=>

ADDITIONAL INFORMATION

ansible-playbook install.yml -i ../inventory_os_k8s_prp

playbook logs

``` PLAY [Build and deploy AWX] *************************************************************************************************************************************************************************************** TASK [Gathering Facts] ******************************************************************************************************************************************************************************************** ok: [localhost] TASK [check_vars : include_tasks] ********************************************************************************************************************************************************************************* skipping: [localhost] TASK [check_vars : include_tasks] ********************************************************************************************************************************************************************************* included: /home/mrkw3241/work/k8s/awx/installer/roles/check_vars/tasks/check_docker.yml for localhost TASK [check_vars : postgres_data_dir should be defined] *********************************************************************************************************************************************************** ok: [localhost] => { "changed": false, "msg": "All assertions passed" } TASK [check_vars : host_port should be defined] ******************************************************************************************************************************************************************* ok: [localhost] => { "changed": false, "msg": "All assertions passed" } TASK [image_build : Set global version if not provided] *********************************************************************************************************************************************************** skipping: [localhost] TASK [image_build : Verify awx-logos directory exists for official install] *************************************************************************************************************************************** skipping: [localhost] TASK [image_build : Copy logos for inclusion in sdist] ************************************************************************************************************************************************************ skipping: [localhost] TASK [image_build : Set sdist file name] ************************************************************************************************************************************************************************** skipping: [localhost] TASK [image_build : AWX Distribution] ***************************************************************************************************************************************************************************** skipping: [localhost] TASK [image_build : Stat distribution file] *********************************************************************************************************************************************************************** skipping: [localhost] TASK [image_build : Clean distribution] *************************************************************************************************************************************************************************** skipping: [localhost] TASK [image_build : Build sdist builder image] ******************************************************************************************************************************************************************** skipping: [localhost] TASK [image_build : Build AWX distribution using container] ******************************************************************************************************************************************************* skipping: [localhost] TASK [image_build : Build AWX distribution locally] *************************************************************************************************************************************************************** skipping: [localhost] TASK [image_build : Set docker build base path] ******************************************************************************************************************************************************************* skipping: [localhost] TASK [image_build : Set awx_web image name] *********************************************************************************************************************************************************************** skipping: [localhost] TASK [image_build : Set awx_task image name] ********************************************************************************************************************************************************************** skipping: [localhost] TASK [image_build : Ensure directory exists] ********************************************************************************************************************************************************************** skipping: [localhost] TASK [image_build : Stage sdist] ********************************************************************************************************************************************************************************** skipping: [localhost] TASK [image_build : Template web Dockerfile] ********************************************************************************************************************************************************************** skipping: [localhost] TASK [image_build : Template task Dockerfile] ********************************************************************************************************************************************************************* skipping: [localhost] TASK [image_build : Stage launch_awx] ***************************************************************************************************************************************************************************** skipping: [localhost] TASK [image_build : Stage launch_awx_task] ************************************************************************************************************************************************************************ skipping: [localhost] TASK [image_build : Stage google-cloud-sdk.repo] ****************************************************************************************************************************************************************** skipping: [localhost] TASK [image_build : Stage supervisor.conf] ************************************************************************************************************************************************************************ skipping: [localhost] TASK [image_build : Stage supervisor_task.conf] ******************************************************************************************************************************************************************* skipping: [localhost] TASK [image_build : Stage settings.py] **************************************************************************************************************************************************************************** skipping: [localhost] TASK [image_build : Stage requirements] *************************************************************************************************************************************************************************** skipping: [localhost] TASK [image_build : Stage config watcher] ************************************************************************************************************************************************************************* skipping: [localhost] TASK [image_build : Stage Makefile] ******************************************************************************************************************************************************************************* skipping: [localhost] TASK [image_build : Build base web image] ************************************************************************************************************************************************************************* skipping: [localhost] TASK [image_build : Build base task image] ************************************************************************************************************************************************************************ skipping: [localhost] TASK [image_build : Tag task and web images as latest] ************************************************************************************************************************************************************ skipping: [localhost] TASK [image_build : Clean docker base directory] ****************************************************************************************************************************************************************** skipping: [localhost] TASK [image_push : Authenticate with Docker registry if registry password given] ********************************************************************************************************************************** skipping: [localhost] TASK [image_push : Remove web image] ****************************************************************************************************************************************************************************** skipping: [localhost] TASK [image_push : Remove task image] ***************************************************************************************************************************************************************************** skipping: [localhost] TASK [image_push : Tag and push web image to registry] ************************************************************************************************************************************************************ skipping: [localhost] TASK [image_push : Tag and push task image to registry] *********************************************************************************************************************************************************** skipping: [localhost] TASK [image_push : Set full image path for Registry] ************************************************************************************************************************************************************** skipping: [localhost] TASK [kubernetes : fail] ****************************************************************************************************************************************************************************************** skipping: [localhost] TASK [kubernetes : include_tasks] ********************************************************************************************************************************************************************************* skipping: [localhost] => (item=openshift_auth.yml) skipping: [localhost] => (item=openshift.yml) TASK [kubernetes : include_tasks] ********************************************************************************************************************************************************************************* included: /home/mrkw3241/work/k8s/awx/installer/roles/kubernetes/tasks/kubernetes_auth.yml for localhost included: /home/mrkw3241/work/k8s/awx/installer/roles/kubernetes/tasks/kubernetes.yml for localhost TASK [kubernetes : Set the Kubernetes Context] ******************************************************************************************************************************************************************** changed: [localhost] TASK [kubernetes : Get Namespace Detail] ************************************************************************************************************************************************************************** changed: [localhost] TASK [kubernetes : Create AWX Kubernetes Project] ***************************************************************************************************************************************************************** skipping: [localhost] TASK [kubernetes : Set postgresql service name] ******************************************************************************************************************************************************************* ok: [localhost] TASK [kubernetes : Get Kubernetes API version] ******************************************************************************************************************************************************************** changed: [localhost] TASK [kubernetes : Extract server version from command output] **************************************************************************************************************************************************** ok: [localhost] TASK [kubernetes : Use kubectl or oc] ***************************************************************************************************************************************************************************** ok: [localhost] TASK [kubernetes : set_fact] ************************************************************************************************************************************************************************************** ok: [localhost] TASK [kubernetes : Record deployment size] ************************************************************************************************************************************************************************ fatal: [localhost]: FAILED! => {"changed": true, "cmd": "kubectl get sts awx-prp -n awx-prp -o=jsonpath='{.status.replicas}'\n", "delta": "0:00:00.453192", "end": "2020-03-31 12:25:54.961335", "msg": "non-zero return code", "rc": 1, "start": "2020-03-31 12:25:54.508143", "stderr": "Error from server (NotFound): statefulsets.apps \"awx-prp\" not found", "stderr_lines": ["Error from server (NotFound): statefulsets.apps \"awx-prp\" not found"], "stdout": "", "stdout_lines": []} ...ignoring TASK [kubernetes : Set expected post-deployment Replicas value] *************************************************************************************************************************************************** skipping: [localhost] TASK [kubernetes : Delete existing Deployment] ******************************************************************************************************************************************************************** skipping: [localhost] TASK [kubernetes : Get Postgres Service Detail] ******************************************************************************************************************************************************************* fatal: [localhost]: FAILED! => {"changed": true, "cmd": "kubectl describe svc awx-prp-postgresql -n awx-prp", "delta": "0:00:00.367317", "end": "2020-03-31 12:25:55.641523", "msg": "non-zero return code", "rc": 1, "start": "2020-03-31 12:25:55.274206", "stderr": "Error from server (NotFound): services \"awx-prp-postgresql\" not found", "stderr_lines": ["Error from server (NotFound): services \"awx-prp-postgresql\" not found"], "stdout": "", "stdout_lines": []} ...ignoring TASK [kubernetes : Template PostgreSQL Deployment (OpenShift)] **************************************************************************************************************************************************** skipping: [localhost] TASK [kubernetes : Deploy and Activate Postgres (OpenShift)] ****************************************************************************************************************************************************** skipping: [localhost] TASK [kubernetes : Create Temporary Values File (Kubernetes)] ***************************************************************************************************************************************************** changed: [localhost] TASK [kubernetes : Populate Temporary Values File (Kubernetes)] *************************************************************************************************************************************************** changed: [localhost] TASK [kubernetes : Deploy and Activate Postgres (Kubernetes)] ***************************************************************************************************************************************************** changed: [localhost] TASK [kubernetes : Remove tempfile] ******************************************************************************************************************************************************************************* changed: [localhost] TASK [kubernetes : Set postgresql hostname to helm package service (Kubernetes)] ********************************************************************************************************************************** ok: [localhost] TASK [kubernetes : Wait for Postgres to activate] ***************************************************************************************************************************************************************** Pausing for 60 seconds (ctrl+C then 'C' = continue early, ctrl+C then 'A' = abort) ok: [localhost] TASK [kubernetes : Check if Postgres 9.6 is being used] *********************************************************************************************************************************************************** skipping: [localhost] TASK [kubernetes : Set new pg image] ****************************************************************************************************************************************************************************** skipping: [localhost] TASK [kubernetes : Wait for change to take affect] **************************************************************************************************************************************************************** skipping: [localhost] TASK [kubernetes : Set env var for pg upgrade] ******************************************************************************************************************************************************************** skipping: [localhost] TASK [kubernetes : Wait for change to take affect] **************************************************************************************************************************************************************** skipping: [localhost] TASK [kubernetes : Set env var for new pg version] **************************************************************************************************************************************************************** skipping: [localhost] TASK [kubernetes : Wait for Postgres to redeploy] ***************************************************************************************************************************************************************** skipping: [localhost] TASK [kubernetes : Wait for Postgres to finish upgrading] ********************************************************************************************************************************************************* skipping: [localhost] TASK [kubernetes : Unset upgrade env var] ************************************************************************************************************************************************************************* skipping: [localhost] TASK [kubernetes : Wait for Postgres to redeploy] ***************************************************************************************************************************************************************** skipping: [localhost] TASK [kubernetes : Set task image name] *************************************************************************************************************************************************************************** skipping: [localhost] TASK [kubernetes : Set web image name] **************************************************************************************************************************************************************************** skipping: [localhost] TASK [kubernetes : Generate SSL certificates for RabbitMQ, if needed] ********************************************************************************************************************************************* skipping: [localhost] TASK [kubernetes : Determine StatefulSet api version] ************************************************************************************************************************************************************* ok: [localhost] TASK [kubernetes : Render deployment templates] ******************************************************************************************************************************************************************* ok: [localhost] => (item=None) ok: [localhost] => (item=None) ok: [localhost] => (item=None) ok: [localhost] TASK [kubernetes : Apply Deployment] ****************************************************************************************************************************************************************************** changed: [localhost] => (item=None) changed: [localhost] => (item=None) changed: [localhost] => (item=None) changed: [localhost] TASK [kubernetes : Delete any existing management pod] ************************************************************************************************************************************************************ changed: [localhost] TASK [kubernetes : Template management pod] *********************************************************************************************************************************************************************** ok: [localhost] TASK [kubernetes : Create management pod] ************************************************************************************************************************************************************************* changed: [localhost] TASK [kubernetes : Wait for management pod to start] ************************************************************************************************************************************************************** FAILED - RETRYING: Wait for management pod to start (60 retries left). changed: [localhost] TASK [kubernetes : Migrate database] ****************************************************************************************************************************************************************************** changed: [localhost] TASK [kubernetes : Check for Tower Super users] ******************************************************************************************************************************************************************* fatal: [localhost]: FAILED! => {"changed": true, "cmd": "kubectl -n awx-prp exec ansible-tower-management -- bash -c \"echo 'from django.contrib.auth.models import User; nsu = User.objects.filter(is_superuser=True).count(); exit(0 if nsu > 0 else 1)' | awx-manage shell\"\n", "delta": "0:00:02.624181", "end": "2020-03-31 12:28:16.431460", "msg": "non-zero return code", "rc": 1, "start": "2020-03-31 12:28:13.807279", "stderr": "command terminated with exit code 1", "stderr_lines": ["command terminated with exit code 1"], "stdout": "", "stdout_lines": []} ...ignoring TASK [kubernetes : create django super user if it does not exist] ************************************************************************************************************************************************* fatal: [localhost]: FAILED! => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": true} PLAY RECAP ******************************************************************************************************************************************************************************************************** localhost : ok=30 changed=15 unreachable=0 failed=1 skipped=57 rescued=0 ignored=3 ```

kubectl logs awx-prp-postgresql-postgresql-0

postgres logs

``` + id current user id: uid=0(root) gid=0(root) groups=0(root),1001 + echo current user id: uid=0(root) gid=0(root) groups=0(root),1001 + mkdir -p /bitnami/postgresql/data + chmod 700 /bitnami/postgresql/data + find /bitnami/postgresql -mindepth 1 -maxdepth 1 -not -name .snapshot -not -name lost+found + xargs chown -R 1001:1001 + chmod -R 777 /dev/shm postgresql 10:26:17.28 postgresql 10:26:17.29 Welcome to the Bitnami postgresql container postgresql 10:26:17.29 Subscribe to project updates by watching https://github.com/bitnami/bitnami-docker-postgresql postgresql 10:26:17.29 Submit issues and feature requests at https://github.com/bitnami/bitnami-docker-postgresql/issues postgresql 10:26:17.29 Send us your feedback at containers@bitnami.com postgresql 10:26:17.29 postgresql 10:26:17.31 INFO ==> ** Starting PostgreSQL setup ** postgresql 10:26:17.36 INFO ==> Validating settings in POSTGRESQL_* env vars.. postgresql 10:26:17.36 INFO ==> Loading custom pre-init scripts... postgresql 10:26:17.36 INFO ==> Initializing PostgreSQL database... postgresql 10:26:17.38 INFO ==> postgresql.conf file not detected. Generating it... postgresql 10:26:17.39 INFO ==> pg_hba.conf file not detected. Generating it... postgresql 10:26:17.39 INFO ==> Generating local authentication configuration postgresql 10:26:18.53 INFO ==> Starting PostgreSQL in background... postgresql 10:26:18.91 INFO ==> Creating user awx postgresql 10:26:18.92 INFO ==> Grating access to "awx" to the database "awx" postgresql 10:26:18.94 INFO ==> Configuring replication parameters postgresql 10:26:18.96 INFO ==> Configuring fsync postgresql 10:26:18.97 INFO ==> Loading custom scripts... postgresql 10:26:18.97 INFO ==> Enabling remote connections postgresql 10:26:18.98 INFO ==> Stopping PostgreSQL... postgresql 10:26:19.99 INFO ==> ** PostgreSQL setup finished! ** postgresql 10:26:20.04 INFO ==> ** Starting PostgreSQL ** 2020-03-31 10:26:20.063 GMT [1] LOG: listening on IPv4 address "0.0.0.0", port 5432 2020-03-31 10:26:20.063 GMT [1] LOG: listening on IPv6 address "::", port 5432 2020-03-31 10:26:20.066 GMT [1] LOG: listening on Unix socket "/tmp/.s.PGSQL.5432" 2020-03-31 10:26:20.080 GMT [213] LOG: database system was shut down at 2020-03-31 10:26:19 GMT 2020-03-31 10:26:20.085 GMT [1] LOG: database system is ready to accept connections 2020-03-31 10:27:12.953 GMT [268] LOG: incomplete startup packet 2020-03-31 10:27:14.254 GMT [269] LOG: incomplete startup packet 2020-03-31 10:27:25.429 GMT [283] ERROR: relation "conf_setting" does not exist at character 158 2020-03-31 10:27:25.429 GMT [283] STATEMENT: SELECT "conf_setting"."id", "conf_setting"."created", "conf_setting"."modified", "conf_setting"."key", "conf_setting"."value", "conf_setting"."user_id" FROM "conf_setting" WHERE ("conf_setting"."key" IN ('ACTIVITY_STREAM_ENABLED', 'ACTIVITY_STREAM_ENABLED_FOR_INVENTORY_SYNC', 'ORG_ADMINS_CAN_SEE_ALL_USERS', 'MANAGE_ORGANIZATION_AUTH', 'TOWER_URL_BASE', 'PROXY_IP_WHITELIST', 'LICENSE', 'REDHAT_USERNAME', 'REDHAT_PASSWORD', 'INSTALL_UUID', 'CUSTOM_VENV_PATHS', 'AD_HOC_COMMANDS', 'ALLOW_JINJA_IN_EXTRA_VARS', 'AWX_PROOT_ENABLED', 'AWX_PROOT_BASE_PATH', 'AWX_PROOT_HIDE_PATHS', 'AWX_PROOT_SHOW_PATHS', 'AWX_ISOLATED_VERBOSITY', 'AWX_ISOLATED_CHECK_INTERVAL', 'AWX_ISOLATED_LAUNCH_TIMEOUT', 'AWX_ISOLATED_CONNECTION_TIMEOUT', 'AWX_ISOLATED_HOST_KEY_CHECKING', 'AWX_ISOLATED_KEY_GENERATION', 'AWX_ISOLATED_PRIVATE_KEY', 'AWX_ISOLATED_PUBLIC_KEY', 'AWX_RESOURCE_PROFILING_ENABLED', 'AWX_RESOURCE_PROFILING_CPU_POLL_INTERVAL', 'AWX_RESOURCE_PROFILING_MEMORY_POLL_INTERVAL', 'AWX_RESOURCE_PROFILING_PID_POLL_INTERVAL', 'AWX_TASK_ENV', 'INSIGHTS_TRACKING_STATE', 'PROJECT_UPDATE_VVV', 'AWX_ROLES_ENABLED', 'AWX_COLLECTIONS_ENABLED', 'PRIMARY_GALAXY_URL', 'PRIMARY_GALAXY_USERNAME', 'PRIMARY_GALAXY_PASSWORD', 'PRIMARY_GALAXY_TOKEN', 'PRIMARY_GALAXY_AUTH_URL', 'PUBLIC_GALAXY_ENABLED', 'GALAXY_IGNORE_CERTS', 'STDOUT_MAX_BYTES_DISPLAY', 'EVENT_STDOUT_MAX_BYTES_DISPLAY', 'SCHEDULE_MAX_JOBS', 'AWX_ANSIBLE_CALLBACK_PLUGINS', 'DEFAULT_JOB_TIMEOUT', 'DEFAULT_INVENTORY_UPDATE_TIMEOUT', 'DEFAULT_PROJECT_UPDATE_TIMEOUT', 'ANSIBLE_FACT_CACHE_TIMEOUT', 'MAX_FORKS', 'LOG_AGGREGATOR_HOST', 'LOG_AGGREGATOR_PORT', 'LOG_AGGREGATOR_TYPE', 'LOG_AGGREGATOR_USERNAME', 'LOG_AGGREGATOR_PASSWORD', 'LOG_AGGREGATOR_LOGGERS', 'LOG_AGGREGATOR_INDIVIDUAL_FACTS', 'LOG_AGGREGATOR_ENABLED', 'LOG_AGGREGATOR_TOWER_UUID', 'LOG_AGGREGATOR_PROTOCOL', 'LOG_AGGREGATOR_TCP_TIMEOUT', 'LOG_AGGREGATOR_VERIFY_CERT', 'LOG_AGGREGATOR_LEVEL', 'LOG_AGGREGATOR_AUDIT', 'BROKER_DURABILITY', 'AUTOMATION_ANALYTICS_LAST_GATHER', 'AUTOMATION_ANALYTICS_GATHER_INTERVAL', 'SESSION_COOKIE_AGE', 'SESSIONS_PER_USER', 'AUTH_BASIC_ENABLED', 'OAUTH2_PROVIDER', 'ALLOW_OAUTH2_FOR_EXTERNAL_USERS', 'LOGIN_REDIRECT_OVERRIDE', 'PENDO_TRACKING_STATE', 'CUSTOM_LOGIN_INFO', 'CUSTOM_LOGO', 'MAX_UI_JOB_EVENTS', 'UI_LIVE_UPDATES_ENABLED', 'SOCIAL_AUTH_ORGANIZATION_MAP', 'SOCIAL_AUTH_TEAM_MAP', 'SOCIAL_AUTH_USER_FIELDS', 'AUTH_LDAP_SERVER_URI', 'AUTH_LDAP_BIND_DN', 'AUTH_LDAP_BIND_PASSWORD', 'AUTH_LDAP_START_TLS', 'AUTH_LDAP_CONNECTION_OPTIONS', 'AUTH_LDAP_USER_SEARCH', 'AUTH_LDAP_USER_DN_TEMPLATE', 'AUTH_LDAP_USER_ATTR_MAP', 'AUTH_LDAP_GROUP_SEARCH', 'AUTH_LDAP_GROUP_TYPE', 'AUTH_LDAP_GROUP_TYPE_PARAMS', 'AUTH_LDAP_REQUIRE_GROUP', 'AUTH_LDAP_DENY_GROUP', 'AUTH_LDAP_USER_FLAGS_BY_GROUP', 'AUTH_LDAP_ORGANIZATION_MAP', 'AUTH_LDAP_TEAM_MAP', 'AUTH_LDAP_1_SERVER_URI', 'AUTH_LDAP_1_BIND_DN', 'AUTH_LDAP_1_BIND_PASSWORD', 'AUTH_LDAP_1_START_TLS', 'AUTH_LDAP_1_CONNECTION_OPTIONS', 'AUTH_LDAP_1_USER_SEARCH', 'AUTH_LDAP_1_USER_DN_TEMPLATE', 'AUTH_LDAP_1_USER_ATTR_MAP', 'AUTH_LDAP_1_GROUP_SEARCH', 'AUTH_LDAP_1_GROUP_TYPE', 'AUTH_LDAP_1_GROUP_TYPE_PARAMS', 'AUTH_LDAP_1_REQUIRE_GROUP', 'AUTH_LDAP_1_DENY_GROUP', 'AUTH_LDAP_1_USER_FLAGS_BY_GROUP', 'AUTH_LDAP_1_ORGANIZATION_MAP', 'AUTH_LDAP_1_TEAM_MAP', 'AUTH_LDAP_2_SERVER_URI', 'AUTH_LDAP_2_BIND_DN', 'AUTH_LDAP_2_BIND_PASSWORD', 'AUTH_LDAP_2_START_TLS', 'AUTH_LDAP_2_CONNECTION_OPTIONS', 'AUTH_LDAP_2_USER_SEARCH', 'AUTH_LDAP_2_USER_DN_TEMPLATE', 'AUTH_LDAP_2_USER_ATTR_MAP', 'AUTH_LDAP_2_GROUP_SEARCH', 'AUTH_LDAP_2_GROUP_TYPE', 'AUTH_LDAP_2_GROUP_TYPE_PARAMS', 'AUTH_LDAP_2_REQUIRE_GROUP', 'AUTH_LDAP_2_DENY_GROUP', 'AUTH_LDAP_2_USER_FLAGS_BY_GROUP', 'AUTH_LDAP_2_ORGANIZATION_MAP', 'AUTH_LDAP_2_TEAM_MAP', 'AUTH_LDAP_3_SERVER_URI', 'AUTH_LDAP_3_BIND_DN', 'AUTH_LDAP_3_BIND_PASSWORD', 'AUTH_LDAP_3_START_TLS', 'AUTH_LDAP_3_CONNECTION_OPTIONS', 'AUTH_LDAP_3_USER_SEARCH', 'AUTH_LDAP_3_USER_DN_TEMPLATE', 'AUTH_LDAP_3_USER_ATTR_MAP', 'AUTH_LDAP_3_GROUP_SEARCH', 'AUTH_LDAP_3_GROUP_TYPE', 'AUTH_LDAP_3_GROUP_TYPE_PARAMS', 'AUTH_LDAP_3_REQUIRE_GROUP', 'AUTH_LDAP_3_DENY_GROUP', 'AUTH_LDAP_3_USER_FLAGS_BY_GROUP', 'AUTH_LDAP_3_ORGANIZATION_MAP', 'AUTH_LDAP_3_TEAM_MAP', 'AUTH_LDAP_4_SERVER_URI', 'AUTH_LDAP_4_BIND_DN', 'AUTH_LDAP_4_BIND_PASSWORD', 'AUTH_LDAP_4_START_TLS', 'AUTH_LDAP_4_CONNECTION_OPTIONS', 'AUTH_LDAP_4_USER_SEARCH', 'AUTH_LDAP_4_USER_DN_TEMPLATE', 'AUTH_LDAP_4_USER_ATTR_MAP', 'AUTH_LDAP_4_GROUP_SEARCH', 'AUTH_LDAP_4_GROUP_TYPE', 'AUTH_LDAP_4_GROUP_TYPE_PARAMS', 'AUTH_LDAP_4_REQUIRE_GROUP', 'AUTH_LDAP_4_DENY_GROUP', 'AUTH_LDAP_4_USER_FLAGS_BY_GROUP', 'AUTH_LDAP_4_ORGANIZATION_MAP', 'AUTH_LDAP_4_TEAM_MAP', 'AUTH_LDAP_5_SERVER_URI', 'AUTH_LDAP_5_BIND_DN', 'AUTH_LDAP_5_BIND_PASSWORD', 'AUTH_LDAP_5_START_TLS', 'AUTH_LDAP_5_CONNECTION_OPTIONS', 'AUTH_LDAP_5_USER_SEARCH', 'AUTH_LDAP_5_USER_DN_TEMPLATE', 'AUTH_LDAP_5_USER_ATTR_MAP', 'AUTH_LDAP_5_GROUP_SEARCH', 'AUTH_LDAP_5_GROUP_TYPE', 'AUTH_LDAP_5_GROUP_TYPE_PARAMS', 'AUTH_LDAP_5_REQUIRE_GROUP', 'AUTH_LDAP_5_DENY_GROUP', 'AUTH_LDAP_5_USER_FLAGS_BY_GROUP', 'AUTH_LDAP_5_ORGANIZATION_MAP', 'AUTH_LDAP_5_TEAM_MAP', 'RADIUS_SERVER', 'RADIUS_PORT', 'RADIUS_SECRET', 'TACACSPLUS_HOST', 'TACACSPLUS_PORT', 'TACACSPLUS_SECRET', 'TACACSPLUS_SESSION_TIMEOUT', 'TACACSPLUS_AUTH_PROTOCOL', 'SOCIAL_AUTH_GOOGLE_OAUTH2_KEY', 'SOCIAL_AUTH_GOOGLE_OAUTH2_SECRET', 'SOCIAL_AUTH_GOOGLE_OAUTH2_WHITELISTED_DOMAINS', 'SOCIAL_AUTH_GOOGLE_OAUTH2_AUTH_EXTRA_ARGUMENTS', 'SOCIAL_AUTH_GOOGLE_OAUTH2_ORGANIZATION_MAP', 'SOCIAL_AUTH_GOOGLE_OAUTH2_TEAM_MAP', 'SOCIAL_AUTH_GITHUB_KEY', 'SOCIAL_AUTH_GITHUB_SECRET', 'SOCIAL_AUTH_GITHUB_ORGANIZATION_MAP', 'SOCIAL_AUTH_GITHUB_TEAM_MAP', 'SOCIAL_AUTH_GITHUB_ORG_KEY', 'SOCIAL_AUTH_GITHUB_ORG_SECRET', 'SOCIAL_AUTH_GITHUB_ORG_NAME', 'SOCIAL_AUTH_GITHUB_ORG_ORGANIZATION_MAP', 'SOCIAL_AUTH_GITHUB_ORG_TEAM_MAP', 'SOCIAL_AUTH_GITHUB_TEAM_KEY', 'SOCIAL_AUTH_GITHUB_TEAM_SECRET', 'SOCIAL_AUTH_GITHUB_TEAM_ID', 'SOCIAL_AUTH_GITHUB_TEAM_ORGANIZATION_MAP', 'SOCIAL_AUTH_GITHUB_TEAM_TEAM_MAP', 'SOCIAL_AUTH_AZUREAD_OAUTH2_KEY', 'SOCIAL_AUTH_AZUREAD_OAUTH2_SECRET', 'SOCIAL_AUTH_AZUREAD_OAUTH2_ORGANIZATION_MAP', 'SOCIAL_AUTH_AZUREAD_OAUTH2_TEAM_MAP', 'SOCIAL_AUTH_SAML_SP_ENTITY_ID', 'SOCIAL_AUTH_SAML_SP_PUBLIC_CERT', 'SOCIAL_AUTH_SAML_SP_PRIVATE_KEY', 'SOCIAL_AUTH_SAML_ORG_INFO', 'SOCIAL_AUTH_SAML_TECHNICAL_CONTACT', 'SOCIAL_AUTH_SAML_SUPPORT_CONTACT', 'SOCIAL_AUTH_SAML_ENABLED_IDPS', 'SOCIAL_AUTH_SAML_SECURITY_CONFIG', 'SOCIAL_AUTH_SAML_SP_EXTRA', 'SOCIAL_AUTH_SAML_EXTRA_DATA', 'SOCIAL_AUTH_SAML_ORGANIZATION_MAP', 'SOCIAL_AUTH_SAML_TEAM_MAP', 'SOCIAL_AUTH_SAML_ORGANIZATION_ATTR', 'SOCIAL_AUTH_SAML_TEAM_ATTR') AND "conf_setting"."user_id" IS NULL) ORDER BY "conf_setting"."id" ASC 2020-03-31 10:27:25.647 GMT [283] ERROR: relation "django_migrations" does not exist at character 124 2020-03-31 10:27:25.647 GMT [283] STATEMENT: SELECT "django_migrations"."id", "django_migrations"."app", "django_migrations"."name", "django_migrations"."applied" FROM "django_migrations" WHERE ("django_migrations"."app" = 'main' AND "django_migrations"."name" IN ('0005a_squashed_v310_v313_updates', '0005b_squashed_v310_v313_updates')) 2020-03-31 10:27:25.662 GMT [283] ERROR: relation "django_migrations" does not exist at character 124 2020-03-31 10:27:25.662 GMT [283] STATEMENT: SELECT "django_migrations"."id", "django_migrations"."app", "django_migrations"."name", "django_migrations"."applied" FROM "django_migrations" WHERE ("django_migrations"."app" = 'main' AND NOT ("django_migrations"."name"::text LIKE '%squashed%')) ORDER BY "django_migrations"."id" DESC LIMIT 1 2020-03-31 10:27:25.665 GMT [283] ERROR: relation "django_migrations" does not exist at character 124 2020-03-31 10:27:25.665 GMT [283] STATEMENT: SELECT "django_migrations"."id", "django_migrations"."app", "django_migrations"."name", "django_migrations"."applied" FROM "django_migrations" WHERE ("django_migrations"."app" = 'main' AND NOT ("django_migrations"."name"::text LIKE '%squashed%')) ORDER BY "django_migrations"."id" DESC LIMIT 1 2020-03-31 10:27:25.676 GMT [283] ERROR: relation "django_migrations" does not exist at character 124 2020-03-31 10:27:25.676 GMT [283] STATEMENT: SELECT "django_migrations"."id", "django_migrations"."app", "django_migrations"."name", "django_migrations"."applied" FROM "django_migrations" WHERE ("django_migrations"."app" = 'main' AND NOT ("django_migrations"."name"::text LIKE '%squashed%')) ORDER BY "django_migrations"."id" DESC LIMIT 1 2020-03-31 10:27:25.681 GMT [283] ERROR: relation "django_migrations" does not exist at character 124 2020-03-31 10:27:25.681 GMT [283] STATEMENT: SELECT "django_migrations"."id", "django_migrations"."app", "django_migrations"."name", "django_migrations"."applied" FROM "django_migrations" WHERE ("django_migrations"."app" = 'main' AND NOT ("django_migrations"."name"::text LIKE '%squashed%')) ORDER BY "django_migrations"."id" DESC LIMIT 1 2020-03-31 10:27:25.690 GMT [283] ERROR: relation "django_migrations" does not exist at character 124 2020-03-31 10:27:25.690 GMT [283] STATEMENT: SELECT "django_migrations"."id", "django_migrations"."app", "django_migrations"."name", "django_migrations"."applied" FROM "django_migrations" WHERE ("django_migrations"."app" = 'main' AND NOT ("django_migrations"."name"::text LIKE '%squashed%')) ORDER BY "django_migrations"."id" DESC LIMIT 1 2020-03-31 10:27:25.692 GMT [283] ERROR: relation "django_migrations" does not exist at character 124 2020-03-31 10:27:25.692 GMT [283] STATEMENT: SELECT "django_migrations"."id", "django_migrations"."app", "django_migrations"."name", "django_migrations"."applied" FROM "django_migrations" WHERE ("django_migrations"."app" = 'main' AND NOT ("django_migrations"."name"::text LIKE '%squashed%')) ORDER BY "django_migrations"."id" DESC LIMIT 1 2020-03-31 10:28:19.603 GMT [359] ERROR: column "deleted_actor" of relation "main_activitystream" does not exist at character 85 2020-03-31 10:28:19.603 GMT [359] STATEMENT: INSERT INTO "main_activitystream" ("actor_id", "operation", "timestamp", "changes", "deleted_actor", "action_node", "object_relationship_type", "object1", "object2", "setting") VALUES (NULL, 'create', '2020-03-31T10:28:19.598577+00:00'::timestamptz, '{"username": "admin", "first_name": "", "last_name": "", "email": "root@localhost", "is_superuser": true, "password": "hidden", "id": 1}', NULL, 'ansible-tower-management', '', 'user', '', '{}') RETURNING "main_activitystream"."id" ```

kubectl logs awx-prp-0 -f --all-containers

all containers awx-prp-0 logs

``` Using /etc/ansible/ansible.cfg as config file 127.0.0.1 | SUCCESS => { "ansible_facts": { "discovered_interpreter_python": "/usr/libexec/platform-python" }, "changed": false, 2020-03-31 10:27:21.608 [info] <0.8.0> Feature flags: list of feature flags found: 2020-03-31 10:27:21.608 [info] <0.8.0> Feature flags: feature flag states written to disk: yes 2020-03-31 10:27:21.634 [info] <0.239.0> "elapsed": 0, "match_groupdict": {}, "match_groups": [], "path": null, "port": 5432, "search_regex": null, "state": "started" } Using /etc/ansible/ansible.cfg as config file 127.0.0.1 | SUCCESS => { "ansible_facts": { "discovered_interpreter_python": "/usr/libexec/platform-python" Starting RabbitMQ 3.7.21 on Erlang 22.1.7 Copyright (C) 2007-2019 Pivotal Software, Inc. Licensed under the MPL. See https://www.rabbitmq.com/ ## ## ## ## RabbitMQ 3.7.21. Copyright (C) 2007-2019 Pivotal Software, Inc. ########## Licensed under the MPL. See https://www.rabbitmq.com/ ###### ## ########## Logs: Starting broker... 2020-03-31 10:27:21.635 [info] <0.239.0> node : rabbit@10.233.93.26 home dir : /var/lib/rabbitmq config file(s) : /etc/rabbitmq/rabbitmq.conf cookie hash : at619UOZzsenF44tSK3ulA== }, log(s) : database dir : /var/lib/rabbitmq/mnesia/rabbit@10.233.93.26 2020-03-31 10:27:21.650 [info] <0.239.0> Running boot step pre_boot defined by app rabbit "changed": false, "elapsed": 1, "match_groupdict": {}, "match_groups": [], "path": null, "port": 11211, "search_regex": null, "state": "started" } Using /etc/ansible/ansible.cfg as config file Using /etc/ansible/ansible.cfg as config file 127.0.0.1 | SUCCESS => { "ansible_facts": { "discovered_interpreter_python": "/usr/libexec/platform-python" 2020-03-31 10:27:21.650 [info] <0.239.0> Running boot step rabbit_core_metrics defined by app rabbit 2020-03-31 10:27:21.650 [info] <0.239.0> Running boot step rabbit_alarm defined by app rabbit 2020-03-31 10:27:21.654 [info] <0.245.0> Memory high watermark set to 6416 MiB (6727725875 bytes) of 16040 MiB (16819314688 bytes) total 2020-03-31 10:27:21.658 [info] <0.247.0> Enabling free disk space monitoring 2020-03-31 10:27:21.658 [info] <0.247.0> Disk free limit set to 50MB 2020-03-31 10:27:21.661 [info] <0.239.0> Running boot step code_server_cache defined by app rabbit 2020-03-31 10:27:21.662 [info] <0.239.0> Running boot step file_handle_cache defined by app rabbit 2020-03-31 10:27:21.662 [info] <0.250.0> Limiting to approx 1048476 file handles (943626 sockets) 2020-03-31 10:27:21.662 [info] <0.251.0> FHC read buffering: OFF 2020-03-31 10:27:21.663 [info] <0.251.0> FHC write buffering: ON 2020-03-31 10:27:21.663 [info] <0.239.0> Running boot step worker_pool defined by app rabbit }, "changed": false, "elapsed": 0, "match_groupdict": {}, "match_groups": [], 2020-03-31 10:27:21.663 [info] <0.240.0> Will use 4 processes for default worker pool 2020-03-31 10:27:21.663 [info] <0.240.0> Starting worker pool 'worker_pool' with 4 processes in it 2020-03-31 10:27:21.664 [info] <0.239.0> Running boot step database defined by app rabbit 2020-03-31 10:27:21.664 [info] <0.239.0> Node database directory at /var/lib/rabbitmq/mnesia/rabbit@10.233.93.26 is empty. Assuming we need to join an existing cluster or initialise from scratch... 2020-03-31 10:27:21.664 [info] <0.239.0> Configured peer discovery backend: rabbit_peer_discovery_k8s 2020-03-31 10:27:21.665 [info] <0.239.0> Will try to lock with peer discovery backend rabbit_peer_discovery_k8s 2020-03-31 10:27:21.665 [info] <0.239.0> Peer discovery backend does not support locking, falling back to randomized delay 2020-03-31 10:27:21.665 [info] <0.239.0> Peer discovery backend rabbit_peer_discovery_k8s does not support registration, skipping randomized startup delay. 2020-03-31 10:27:21.686 [info] <0.239.0> k8s endpoint listing returned nodes not yet ready: 10.233.93.26 2020-03-31 10:27:21.686 [info] <0.239.0> All discovered existing cluster peers: 2020-03-31 10:27:21.686 [info] <0.239.0> Discovered no peer nodes to cluster with 2020-03-31 10:27:21.691 [info] <0.43.0> Application mnesia exited with reason: stopped 2020-03-31 10:27:21.743 [info] <0.239.0> Waiting for Mnesia tables for 30000 ms, 9 retries left 2020-03-31 10:27:21.771 [info] <0.239.0> Waiting for Mnesia tables for 30000 ms, 9 retries left 2020-03-31 10:27:21.799 [info] <0.239.0> Waiting for Mnesia tables for 30000 ms, 9 retries left 2020-03-31 10:27:21.799 [info] <0.239.0> Peer discovery backend rabbit_peer_discovery_k8s does not support registration, skipping registration. 2020-03-31 10:27:21.800 [info] <0.239.0> Running boot step database_sync defined by app rabbit 2020-03-31 10:27:21.800 [info] <0.239.0> Running boot step feature_flags defined by app rabbit 2020-03-31 10:27:21.800 [info] <0.239.0> Running boot step codec_correctness_check defined by app rabbit 2020-03-31 10:27:21.800 [info] <0.239.0> Running boot step external_infrastructure defined by app rabbit 2020-03-31 10:27:21.800 [info] <0.239.0> Running boot step rabbit_registry defined by app rabbit 2020-03-31 10:27:21.801 [info] <0.239.0> Running boot step rabbit_auth_mechanism_cr_demo defined by app rabbit 2020-03-31 10:27:21.801 [info] <0.239.0> Running boot step rabbit_queue_location_random defined by app rabbit 2020-03-31 10:27:21.801 [info] <0.239.0> Running boot step rabbit_event defined by app rabbit 2020-03-31 10:27:21.801 [info] <0.239.0> Running boot step rabbit_auth_mechanism_amqplain defined by app rabbit 2020-03-31 10:27:21.802 [info] <0.239.0> Running boot step rabbit_auth_mechanism_plain defined by app rabbit 2020-03-31 10:27:21.802 [info] <0.239.0> Running boot step rabbit_exchange_type_direct defined by app rabbit 2020-03-31 10:27:21.802 [info] <0.239.0> Running boot step rabbit_exchange_type_fanout defined by app rabbit 2020-03-31 10:27:21.802 [info] <0.239.0> Running boot step rabbit_exchange_type_headers defined by app rabbit 2020-03-31 10:27:21.802 [info] <0.239.0> Running boot step rabbit_exchange_type_topic defined by app rabbit 2020-03-31 10:27:21.803 [info] <0.239.0> Running boot step rabbit_mirror_queue_mode_all defined by app rabbit 2020-03-31 10:27:21.803 [info] <0.239.0> Running boot step rabbit_mirror_queue_mode_exactly defined by app rabbit 2020-03-31 10:27:21.803 [info] <0.239.0> Running boot step rabbit_mirror_queue_mode_nodes defined by app rabbit 2020-03-31 10:27:21.803 [info] <0.239.0> Running boot step rabbit_priority_queue defined by app rabbit "path": null, "port": 5432, "search_regex": null, "state": "started" } Using /etc/ansible/ansible.cfg as config file 127.0.0.1 | SUCCESS => { 2020-03-31 10:27:21.803 [info] <0.239.0> Priority queues enabled, real BQ is rabbit_variable_queue 2020-03-31 10:27:21.803 [info] <0.239.0> Running boot step rabbit_queue_location_client_local defined by app rabbit 2020-03-31 10:27:21.803 [info] <0.239.0> Running boot step rabbit_queue_location_min_masters defined by app rabbit 2020-03-31 10:27:21.803 [info] <0.239.0> Running boot step kernel_ready defined by app rabbit 2020-03-31 10:27:21.803 [info] <0.239.0> Running boot step rabbit_sysmon_minder defined by app rabbit 2020-03-31 10:27:21.804 [info] <0.239.0> Running boot step rabbit_epmd_monitor defined by app rabbit 2020-03-31 10:27:21.804 [info] <0.424.0> epmd monitor knows us, inter-node communication (distribution) port: 25672 2020-03-31 10:27:21.805 [info] <0.239.0> Running boot step guid_generator defined by app rabbit 2020-03-31 10:27:21.806 [info] <0.239.0> Running boot step rabbit_node_monitor defined by app rabbit 2020-03-31 10:27:21.806 [info] <0.428.0> Starting rabbit_node_monitor 2020-03-31 10:27:21.807 [info] <0.239.0> Running boot step delegate_sup defined by app rabbit 2020-03-31 10:27:21.807 [info] <0.239.0> Running boot step rabbit_memory_monitor defined by app rabbit 2020-03-31 10:27:21.807 [info] <0.239.0> Running boot step core_initialized defined by app rabbit 2020-03-31 10:27:21.807 [info] <0.239.0> Running boot step upgrade_queues defined by app rabbit 2020-03-31 10:27:21.833 [info] <0.239.0> message_store upgrades: 1 to apply 2020-03-31 10:27:21.833 [info] <0.239.0> message_store upgrades: Applying rabbit_variable_queue:move_messages_to_vhost_store 2020-03-31 10:27:21.833 [info] <0.239.0> message_store upgrades: No durable queues found. Skipping message store migration 2020-03-31 10:27:21.833 [info] <0.239.0> message_store upgrades: Removing the old message store data 2020-03-31 10:27:21.834 [info] <0.239.0> message_store upgrades: All upgrades applied successfully 2020-03-31 10:27:21.866 [info] <0.239.0> Running boot step rabbit_connection_tracking defined by app rabbit 2020-03-31 10:27:21.866 [info] <0.239.0> Running boot step rabbit_connection_tracking_handler defined by app rabbit 2020-03-31 10:27:21.866 [info] <0.239.0> Running boot step rabbit_exchange_parameters defined by app rabbit 2020-03-31 10:27:21.866 [info] <0.239.0> Running boot step rabbit_mirror_queue_misc defined by app rabbit 2020-03-31 10:27:21.867 [info] <0.239.0> Running boot step rabbit_policies defined by app rabbit "ansible_facts": { "discovered_interpreter_python": "/usr/libexec/platform-python" }, "changed": false, "elapsed": 3, "match_groupdict": {}, "match_groups": [], "path": null, "port": 11211, 2020-03-31 10:27:21.867 [info] <0.239.0> Running boot step rabbit_policy defined by app rabbit 2020-03-31 10:27:21.868 [info] <0.239.0> Running boot step rabbit_queue_location_validator defined by app rabbit 2020-03-31 10:27:21.868 [info] <0.239.0> Running boot step rabbit_vhost_limit defined by app rabbit 2020-03-31 10:27:21.868 [info] <0.239.0> Running boot step rabbit_mgmt_reset_handler defined by app rabbitmq_management 2020-03-31 10:27:21.868 [info] <0.239.0> Running boot step rabbit_mgmt_db_handler defined by app rabbitmq_management_agent 2020-03-31 10:27:21.868 [info] <0.239.0> Management plugin: using rates mode 'basic' 2020-03-31 10:27:21.868 [info] <0.239.0> Running boot step recovery defined by app rabbit 2020-03-31 10:27:21.869 [info] <0.239.0> Running boot step load_definitions defined by app rabbitmq_management 2020-03-31 10:27:21.869 [info] <0.239.0> Applying definitions from /etc/rabbitmq/rabbitmq_definitions.json 2020-03-31 10:27:21.869 [info] <0.239.0> Asked to import definitions. Acting user: rmq-internal 2020-03-31 10:27:21.870 [info] <0.239.0> Importing users... "search_regex": null, "state": "started" } Using /etc/ansible/ansible.cfg as config file 2020-03-31 10:27:21.870 [info] <0.239.0> Creating user 'awx' 2020-03-31 10:27:21.871 [info] <0.239.0> Setting user tags for user 'awx' to [administrator] 2020-03-31 10:27:21.872 [info] <0.239.0> Importing vhosts... 2020-03-31 10:27:21.872 [info] <0.239.0> Adding vhost 'awx' 2020-03-31 10:27:21.880 [info] <0.471.0> Making sure data directory '/var/lib/rabbitmq/mnesia/rabbit@10.233.93.26/msg_stores/vhosts/5A7PFSQL4N28PQOL0INDFDDLU' for vhost 'awx' exists 2020-03-31 10:27:21.882 [info] <0.471.0> Starting message stores for vhost 'awx' 2020-03-31 10:27:21.883 [info] <0.475.0> Message store "5A7PFSQL4N28PQOL0INDFDDLU/msg_store_transient": using rabbit_msg_store_ets_index to provide index 2020-03-31 10:27:21.884 [info] <0.471.0> Started message store of type transient for vhost 'awx' 2020-03-31 10:27:21.884 [info] <0.478.0> Message store "5A7PFSQL4N28PQOL0INDFDDLU/msg_store_persistent": using rabbit_msg_store_ets_index to provide index 2020-03-31 10:27:21.884 [warning] <0.478.0> Message store "5A7PFSQL4N28PQOL0INDFDDLU/msg_store_persistent": rebuilding indices from scratch 2020-03-31 10:27:21.885 [info] <0.471.0> Started message store of type persistent for vhost 'awx' 2020-03-31 10:27:21.886 [info] <0.239.0> Importing user permissions... 2020-03-31 10:27:21.886 [info] <0.239.0> Setting permissions for 'awx' in 'awx' to '.*', '.*', '.*' 2020-03-31 10:27:21.887 [info] <0.239.0> Importing topic permissions... 2020-03-31 10:27:21.887 [info] <0.239.0> Importing parameters... 2020-03-31 10:27:21.888 [info] <0.239.0> Importing global parameters... 2020-03-31 10:27:21.888 [info] <0.239.0> Importing policies... 2020-03-31 10:27:21.890 [info] <0.239.0> Importing queues... 2020-03-31 10:27:21.890 [info] <0.239.0> Importing exchanges... 2020-03-31 10:27:21.890 [info] <0.239.0> Importing bindings... 2020-03-31 10:27:21.891 [info] <0.239.0> Running boot step empty_db_check defined by app rabbit 2020-03-31 10:27:21.891 [info] <0.239.0> Running boot step rabbit_looking_glass defined by app rabbit 2020-03-31 10:27:21.891 [info] <0.239.0> Running boot step rabbit_core_metrics_gc defined by app rabbit 2020-03-31 10:27:21.891 [info] <0.239.0> Running boot step background_gc defined by app rabbit 2020-03-31 10:27:21.891 [info] <0.239.0> Running boot step connection_tracking defined by app rabbit 2020-03-31 10:27:21.893 [info] <0.239.0> Setting up a table for connection tracking on this node: 'tracked_connection_on_node_rabbit@10.233.93.26' 2020-03-31 10:27:21.895 [info] <0.239.0> Setting up a table for per-vhost connection counting on this node: 'tracked_connection_per_vhost_on_node_rabbit@10.233.93.26' 2020-03-31 10:27:21.896 [info] <0.239.0> Running boot step routing_ready defined by app rabbit 2020-03-31 10:27:21.896 [info] <0.239.0> Running boot step pre_flight defined by app rabbit 2020-03-31 10:27:21.896 [info] <0.239.0> Running boot step notify_cluster defined by app rabbit 2020-03-31 10:27:21.896 [info] <0.239.0> Running boot step networking defined by app rabbit 2020-03-31 10:27:21.899 [info] <0.524.0> started TCP listener on [::]:5672 2020-03-31 10:27:21.899 [info] <0.239.0> Running boot step direct_client defined by app rabbit 2020-03-31 10:27:21.901 [info] <0.570.0> Peer discovery: enabling node cleanup (will remove nodes not known to the discovery backend). Check interval: 10 seconds. 2020-03-31 10:27:21.929 [info] <0.580.0> Management plugin: HTTP (non-TLS) listener started on port 15672 2020-03-31 10:27:21.929 [info] <0.686.0> Statistics database started. 2020-03-31 10:27:21.929 [info] <0.685.0> Starting worker pool 'management_worker_pool' with 3 processes in it 2020-03-31 10:27:22.027 [info] <0.8.0> Server startup complete; 5 plugins started. * rabbitmq_management * rabbitmq_web_dispatch * rabbitmq_peer_discovery_k8s * rabbitmq_peer_discovery_common * rabbitmq_management_agent completed with 5 plugins. ```

ansible / awx