Closed bskou57 closed 7 months ago
kurokobo
commented
7 months ago backup_dir: /data/backup/tower
Your /data/backup on k3s host will be mounted in the worker container as /backups, so if your backup directory is /data/backup/tower on k3s host, you should specify /backups/tower for backup_dir.
bskou57
commented
7 months ago backup_dir is not the location of backup files ?
I set it as /backups/tower but still have an issue
apiVersion: awx.ansible.com/v1beta1 kind: AWXRestore metadata: name: awxrestore-2023-12-03 namespace: awx spec: deployment_name: awx
backup_pvc: awx-backup-claim backup_dir: /backup/tower
no_log: false [IAAS - root@vmt86rhel937 awx-on-k3s]$ ll /data/backup/tower total 243780 -rw-------. 1 1001 root 1271 Dec 3 17:45 awx_object -rw-------. 1 1001 root 20089 Dec 3 17:45 secrets.yml -rw-rw----. 1 root root 249604579 Dec 3 17:45 tower.db [IAAS - root@vmt86rhel937 awx-on-k3s]$
--------------------------- Ansible Task StdOut -------------------------------
TASK [Check to make sure backup directory exists on PVC] **** fatal: [localhost]: FAILED! => {"changed": true, "rc": 1, "return_code": 1, "stderr": "stat: cannot statx '/backup/tower': No such file or directory\n", "stderr_lines": ["stat: cannot statx '/backup/tower': No such file or directory"], "stdout": "", "stdout_lines": []}
{"level":"error","ts":"2023-12-06T13:38:41Z","logger":"logging_event_handler","msg":"","name":"awxrestore-2023-12-03","namespace":"awx","gvk":"awx.ansible.com/v1beta1, Kind=AWXRestore","event_type":"runner_on_failed","job":"3768719885826966812","EventData.Task":"Check to make sure backup directory exists on PVC","EventData.TaskArgs":"","EventData.FailedTaskPath":"/opt/ansible/roles/restore/tasks/init.yml:97","error":"[playbook task failed]","stacktrace":"github.com/operator-framework/operator-sdk/internal/ansible/events.loggingEventHandler.Handle\n\t/workspace/internal/ansible/events/log_events.go:111"}
{"level":"error","ts":"2023-12-06T13:38:42Z","logger":"runner","msg":"ansible-playbook [core 2.15.2]\r\n config file = /etc/ansible/ansible.cfg\r\n configured module search path = ['/usr/share/ansible/openshift']\r\n ansible python module location = /usr/local/lib/python3.9/site-packages/ansible\r\n ansible collection location = /opt/ansible/.ansible/collections:/usr/share/ansible/collections\r\n executable location = /usr/local/bin/ansible-playbook\r\n python version = 3.9.16 (main, May 31 2023, 12:21:58) [GCC 8.5.0 20210514 (Red Hat 8.5.0-18)] (/usr/bin/python3)\r\n jinja version = 3.1.2\r\n libyaml = True\r\nUsing /etc/ansible/ansible.cfg as config file\r\nSkipping callback 'awx_display', as we already have a stdout callback.\nSkipping callback 'default', as we already have a stdout callback.\nSkipping callback 'minimal', as we already have a stdout callback.\nSkipping callback 'oneline', as we already have a stdout callback.\n\r\nPLAYBOOK: ac6b7ea742ae4222a7117b44436517bf *\n1 plays in /tmp/ansible-operator/runner/awx.ansible.com/v1beta1/AWXRestore/awx/awxrestore-2023-12-03/project/ac6b7ea742ae4222a7117b44436517bf\n[WARNING]: Found variable using reserved name: no_log\n\r\nPLAY [localhost] ***\n\r\nTASK [common : Get information about the cluster] **\r\ntask path: /opt/ansible/roles/common/tasks/main.yml:3\nredirecting (type: lookup) ansible.builtin.k8s to kubernetes.core.k8s\nok: [localhost] => {\"ansible_facts\": {\"api_groups\": [\"\", \"apiregistration.k8s.io\", \"apps\", \"events.k8s.io\", \"authentication.k8s.io\", \"authorization.k8s.io\", \"autoscaling\", \"batch\", \"certificates.k8s.io\", \"networking.k8s.io\", \"policy\", \"rbac.authorization.k8s.io\", \"storage.k8s.io\", \"admissionregistration.k8s.io\", \"apiextensions.k8s.io\", \"scheduling.k8s.io\", \"coordination.k8s.io\", \"node.k8s.io\", \"discovery.k8s.io\", \"flowcontrol.apiserver.k8s.io\", \"helm.cattle.io\", \"k3s.cattle.io\", \"traefik.containo.us\", \"traefik.io\", \"awx.ansible.com\", \"metrics.k8s.io\"]}, \"changed\": false}\n\r\nTASK [common : Determine the cluster type] ***\r\ntask path: /opt/ansible/roles/common/tasks/main.yml:10\nok: [localhost] => {\"ansible_facts\": {\"is_k8s\": true, \"is_openshift\": false}, \"changed\": false}\n\r\nTASK [common : debug] **\r\ntask path: /opt/ansible/roles/common/tasks/main.yml:19\nok: [localhost] => {\r\n \"msg\": \"CLUSTER TYPE: is_openshift=False; is_k8s=True\"\r\n}\n\r\nTASK [restore : Patching labels to AWXRestore kind] ****\r\ntask path: /opt/ansible/roles/restore/tasks/main.yml:2\nok: [localhost] => {\"changed\": false, \"method\": \"update\", \"result\": {\"apiVersion\": \"awx.ansible.com/v1beta1\", \"kind\": \"AWXRestore\", \"metadata\": {\"annotations\": {\"kubectl.kubernetes.io/last-applied-configuration\": \"{\\"apiVersion\\":\\"awx.ansible.com/v1beta1\\",\\"kind\\":\\"AWXRestore\\",\\"metadata\\":{\\"annotations\\":{},\\"name\\":\\"awxrestore-2023-12-03\\",\\"namespace\\":\\"awx\\"},\\"spec\\":{\\"backup_dir\\":\\"/backup/tower\\",\\"backup_pvc\\":\\"awx-backup-claim\\",\\"deployment_name\\":\\"awx\\",\\"no_log\\":false}}\n\"}, \"creationTimestamp\": \"2023-12-06T09:43:54Z\", \"generation\": 3, \"labels\": {\"app.kubernetes.io/component\": \"awx\", \"app.kubernetes.io/managed-by\": \"awx-operator\", \"app.kubernetes.io/operator-version\": \"2.8.0\", \"app.kubernetes.io/part-of\": \"awxrestore-2023-12-03\"}, \"managedFields\": [{\"apiVersion\": \"awx.ansible.com/v1beta1\", \"fieldsType\": \"FieldsV1\", \"fieldsV1\": {\"f:metadata\": {\"f:labels\": {\".\": {}, \"f:app.kubernetes.io/component\": {}, \"f:app.kubernetes.io/managed-by\": {}, \"f:app.kubernetes.io/operator-version\": {}, \"f:app.kubernetes.io/part-of\": {}}}}, \"manager\": \"OpenAPI-Generator\", \"operation\": \"Update\", \"time\": \"2023-12-06T09:43:58Z\"}, {\"apiVersion\": \"awx.ansible.com/v1beta1\", \"fieldsType\": \"FieldsV1\", \"fieldsV1\": {\"f:metadata\": {\"f:annotations\": {\".\": {}, \"f:kubectl.kubernetes.io/last-applied-configuration\": {}}}, \"f:spec\": {\".\": {}, \"f:backup_dir\": {}, \"f:backup_pvc\": {}, \"f:deployment_name\": {}, \"f:image_pull_policy\": {}, \"f:no_log\": {}, \"f:set_self_labels\": {}}}, \"manager\": \"kubectl-client-side-apply\", \"operation\": \"Update\", \"time\": \"2023-12-06T13:37:04Z\"}, {\"apiVersion\": \"awx.ansible.com/v1beta1\", \"fieldsType\": \"FieldsV1\", \"fieldsV1\": {\"f:status\": {\".\": {}, \"f:conditions\": {}}}, \"manager\": \"ansible-operator\", \"operation\": \"Update\", \"subresource\": \"status\", \"time\": \"2023-12-06T13:37:50Z\"}], \"name\": \"awxrestore-2023-12-03\", \"namespace\": \"awx\", \"resourceVersion\": \"79034\", \"uid\": \"b74d2c66-0e79-4afc-b74a-0f88e3f3dd2d\"}, \"spec\": {\"backup_dir\": \"/backup/tower\", \"backup_pvc\": \"awx-backup-claim\", \"deployment_name\": \"awx\", \"image_pull_policy\": \"IfNotPresent\", \"no_log\": false, \"set_self_labels\": true}, \"status\": {\"conditions\": [{\"lastTransitionTime\": \"2023-12-06T09:44:04Z\", \"reason\": \"\", \"status\": \"False\", \"type\": \"Successful\"}, {\"lastTransitionTime\": \"2023-12-06T13:37:50Z\", \"reason\": \"Failed\", \"status\": \"False\", \"type\": \"Failure\"}, {\"lastTransitionTime\": \"2023-12-06T13:37:50Z\", \"reason\": \"Running\", \"status\": \"True\", \"type\": \"Running\"}]}}}\n\r\nTASK [restore : Look up details for this restore object] *\r\ntask path: /opt/ansible/roles/restore/tasks/main.yml:16\nok: [localhost] => {\"api_found\": true, \"changed\": false, \"resources\": [{\"apiVersion\": \"awx.ansible.com/v1beta1\", \"kind\": \"AWXRestore\", \"metadata\": {\"annotations\": {\"kubectl.kubernetes.io/last-applied-configuration\": \"{\\"apiVersion\\":\\"awx.ansible.com/v1beta1\\",\\"kind\\":\\"AWXRestore\\",\\"metadata\\":{\\"annotations\\":{},\\"name\\":\\"awxrestore-2023-12-03\\",\\"namespace\\":\\"awx\\"},\\"spec\\":{\\"backup_dir\\":\\"/backup/tower\\",\\"backup_pvc\\":\\"awx-backup-claim\\",\\"deployment_name\\":\\"awx\\",\\"no_log\\":false}}\n\"}, \"creationTimestamp\": \"2023-12-06T09:43:54Z\", \"generation\": 3, \"labels\": {\"app.kubernetes.io/component\": \"awx\", \"app.kubernetes.io/managed-by\": \"awx-operator\", \"app.kubernetes.io/operator-version\": \"2.8.0\", \"app.kubernetes.io/part-of\": \"awxrestore-2023-12-03\"}, \"managedFields\": [{\"apiVersion\": \"awx.ansible.com/v1beta1\", \"fieldsType\": \"FieldsV1\", \"fieldsV1\": {\"f:metadata\": {\"f:labels\": {\".\": {}, \"f:app.kubernetes.io/component\": {}, \"f:app.kubernetes.io/managed-by\": {}, \"f:app.kubernetes.io/operator-version\": {}, \"f:app.kubernetes.io/part-of\": {}}}}, \"manager\": \"OpenAPI-Generator\", \"operation\": \"Update\", \"time\": \"2023-12-06T09:43:58Z\"}, {\"apiVersion\": \"awx.ansible.com/v1beta1\", \"fieldsType\": \"FieldsV1\", \"fieldsV1\": {\"f:metadata\": {\"f:annotations\": {\".\": {}, \"f:kubectl.kubernetes.io/last-applied-configuration\": {}}}, \"f:spec\": {\".\": {}, \"f:backup_dir\": {}, \"f:backup_pvc\": {}, \"f:deployment_name\": {}, \"f:image_pull_policy\": {}, \"f:no_log\": {}, \"f:set_self_labels\": {}}}, \"manager\": \"kubectl-client-side-apply\", \"operation\": \"Update\", \"time\": \"2023-12-06T13:37:04Z\"}, {\"apiVersion\": \"awx.ansible.com/v1beta1\", \"fieldsType\": \"FieldsV1\", \"fieldsV1\": {\"f:status\": {\".\": {}, \"f:conditions\": {}}}, \"manager\": \"ansible-operator\", \"operation\": \"Update\", \"subresource\": \"status\", \"time\": \"2023-12-06T13:37:50Z\"}], \"name\": \"awxrestore-2023-12-03\", \"namespace\": \"awx\", \"resourceVersion\": \"79034\", \"uid\": \"b74d2c66-0e79-4afc-b74a-0f88e3f3dd2d\"}, \"spec\": {\"backup_dir\": \"/backup/tower\", \"backup_pvc\": \"awx-backup-claim\", \"deployment_name\": \"awx\", \"image_pull_policy\": \"IfNotPresent\", \"no_log\": false, \"set_self_labels\": true}, \"status\": {\"conditions\": [{\"lastTransitionTime\": \"2023-12-06T09:44:04Z\", \"reason\": \"\", \"status\": \"False\", \"type\": \"Successful\"}, {\"lastTransitionTime\": \"2023-12-06T13:37:50Z\", \"reason\": \"Failed\", \"status\": \"False\", \"type\": \"Failure\"}, {\"lastTransitionTime\": \"2023-12-06T13:37:50Z\", \"reason\": \"Running\", \"status\": \"True\", \"type\": \"Running\"}]}}]}\n\r\nTASK [restore : Build additional_labels_items labels from additional_labels] *\r\ntask path: /opt/ansible/roles/restore/tasks/main.yml:24\nskipping: [localhost] => {\"changed\": false, \"false_condition\": \"additional_labels | length\", \"skip_reason\": \"Conditional result was False\"}\n\r\nTASK [restore : include_tasks] ***\r\ntask path: /opt/ansible/roles/restore/tasks/main.yml:35\nstatically imported: /opt/ansible/roles/restore/tasks/error_handling.yml\nstatically imported: /opt/ansible/roles/restore/tasks/error_handling.yml\nstatically imported: /opt/ansible/roles/restore/tasks/error_handling.yml\nincluded: /opt/ansible/roles/restore/tasks/init.yml for localhost\n\r\nTASK [restore : Look up details for the backup] ****\r\ntask path: /opt/ansible/roles/restore/tasks/init.yml:5\nskipping: [localhost] => {\"changed\": false, \"false_condition\": \"backup_name is defined and backup_name != ''\", \"skip_reason\": \"Conditional result was False\"}\n\r\nTASK [restore : Set error message] ***\r\ntask path: /opt/ansible/roles/restore/tasks/init.yml:15\nskipping: [localhost] => {\"changed\": false, \"false_condition\": \"backup_name is defined and backup_name != ''\", \"skip_reason\": \"Conditional result was False\"}\n\r\nTASK [restore : Determine the timestamp] *\r\ntask path: /opt/ansible/roles/restore/tasks/error_handling.yml:3\nskipping: [localhost] => {\"changed\": false, \"false_condition\": \"backup_name is defined and backup_name != ''\", \"skip_reason\": \"Conditional result was False\"}\n\r\nTASK [restore : Emit ocp event with error] ***\r\ntask path: /opt/ansible/roles/restore/tasks/error_handling.yml:7\nskipping: [localhost] => {\"changed\": false, \"false_condition\": \"backup_name is defined and backup_name != ''\", \"skip_reason\": \"Conditional result was False\"}\n\r\nTASK [restore : Fail early if pvc is defined but does not exist] *\r\ntask path: /opt/ansible/roles/restore/tasks/init.yml:22\nskipping: [localhost] => {\"changed\": false, \"false_condition\": \"backup_name is defined and backup_name != ''\", \"skip_reason\": \"Conditional result was False\"}\n\r\nTASK [restore : Set backup facts] **\r\ntask path: /opt/ansible/roles/restore/tasks/init.yml:32\nskipping: [localhost] => {\"changed\": false, \"false_condition\": \"backup_name is defined and backup_name != ''\", \"skip_reason\": \"Conditional result was False\"}\n\r\nTASK [restore : Check provided PVC exists] *\r\ntask path: /opt/ansible/roles/restore/tasks/init.yml:40\nok: [localhost] => {\"api_found\": true, \"changed\": false, \"resources\": [{\"apiVersion\": \"v1\", \"kind\": \"PersistentVolumeClaim\", \"metadata\": {\"annotations\": {\"kubectl.kubernetes.io/last-applied-configuration\": \"{\\"apiVersion\\":\\"v1\\",\\"kind\\":\\"PersistentVolumeClaim\\",\\"metadata\\":{\\"annotations\\":{},\\"name\\":\\"awx-backup-claim\\",\\"namespace\\":\\"awx\\"},\\"spec\\":{\\"accessModes\\":[\\"ReadWriteOnce\\"],\\"resources\\":{\\"requests\\":{\\"storage\\":\\"4Gi\\"}},\\"storageClassName\\":\\"awx-backup-volume\\",\\"volumeMode\\":\\"Filesystem\\"}}\n\", \"pv.kubernetes.io/bind-completed\": \"yes\", \"pv.kubernetes.io/bound-by-controller\": \"yes\"}, \"creationTimestamp\": \"2023-12-05T22:36:11Z\", \"finalizers\": [\"kubernetes.io/pvc-protection\"], \"managedFields\": [{\"apiVersion\": \"v1\", \"fieldsType\": \"FieldsV1\", \"fieldsV1\": {\"f:metadata\": {\"f:annotations\": {\"f:pv.kubernetes.io/bind-completed\": {}, \"f:pv.kubernetes.io/bound-by-controller\": {}}}, \"f:spec\": {\"f:volumeName\": {}}}, \"manager\": \"k3s\", \"operation\": \"Update\", \"time\": \"2023-12-05T22:36:11Z\"}, {\"apiVersion\": \"v1\", \"fieldsType\": \"FieldsV1\", \"fieldsV1\": {\"f:status\": {\"f:accessModes\": {}, \"f:capacity\": {\".\": {}, \"f:storage\": {}}, \"f:phase\": {}}}, \"manager\": \"k3s\", \"operation\": \"Update\", \"subresource\": \"status\", \"time\": \"2023-12-05T22:36:11Z\"}, {\"apiVersion\": \"v1\", \"fieldsType\": \"FieldsV1\", \"fieldsV1\": {\"f:metadata\": {\"f:annotations\": {\".\": {}, \"f:kubectl.kubernetes.io/last-applied-configuration\": {}}}, \"f:spec\": {\"f:accessModes\": {}, \"f:resources\": {\"f:requests\": {\".\": {}, \"f:storage\": {}}}, \"f:storageClassName\": {}, \"f:volumeMode\": {}}}, \"manager\": \"kubectl-client-side-apply\", \"operation\": \"Update\", \"time\": \"2023-12-05T22:36:11Z\"}], \"name\": \"awx-backup-claim\", \"namespace\": \"awx\", \"resourceVersion\": \"2589\", \"uid\": \"366e3c25-52c3-42d5-a5ff-d5deb176d4e1\"}, \"spec\": {\"accessModes\": [\"ReadWriteOnce\"], \"resources\": {\"requests\": {\"storage\": \"4Gi\"}}, \"storageClassName\": \"awx-backup-volume\", \"volumeMode\": \"Filesystem\", \"volumeName\": \"awx-backup-volume\"}, \"status\": {\"accessModes\": [\"ReadWriteOnce\"], \"capacity\": {\"storage\": \"4Gi\"}, \"phase\": \"Bound\"}}]}\n\r\nTASK [restore : Set error message] *****\r\ntask path: /opt/ansible/roles/restore/tasks/init.yml:51\nskipping: [localhost] => {\"changed\": false, \"false_condition\": \"provided_pvc.resources | length == 0\", \"skip_reason\": \"Conditional result was False\"}\n\r\nTASK [restore : Determine the timestamp] **\r\ntask path: /opt/ansible/roles/restore/tasks/error_handling.yml:3\nskipping: [localhost] => {\"changed\": false, \"false_condition\": \"provided_pvc.resources | length == 0\", \"skip_reason\": \"Conditional result was False\"}\n\r\nTASK [restore : Emit ocp event with error] \r\ntask path: /opt/ansible/roles/restore/tasks/error_handling.yml:7\nskipping: [localhost] => {\"changed\": false, \"false_condition\": \"provided_pvc.resources | length == 0\", \"skip_reason\": \"Conditional result was False\"}\n\r\nTASK [restore : Fail early if pvc is defined but does not exist] *\r\ntask path: /opt/ansible/roles/restore/tasks/init.yml:58\nskipping: [localhost] => {\"changed\": false, \"false_condition\": \"provided_pvc.resources | length == 0\", \"skip_reason\": \"Conditional result was False\"}\n\r\nTASK [restore : Delete any existing management pod] **\r\ntask path: /opt/ansible/roles/restore/tasks/init.yml:65\nchanged: [localhost] => {\"changed\": true, \"duration\": 35, \"method\": \"delete\", \"result\": {\"apiVersion\": \"v1\", \"kind\": \"Pod\", \"metadata\": {\"creationTimestamp\": \"2023-12-06T13:37:44Z\", \"deletionGracePeriodSeconds\": 30, \"deletionTimestamp\": \"2023-12-06T13:38:28Z\", \"labels\": {\"app.kubernetes.io/component\": \"awx\", \"app.kubernetes.io/managed-by\": \"awx-operator\", \"app.kubernetes.io/operator-version\": \"2.8.0\", \"app.kubernetes.io/part-of\": \"awxrestore-2023-12-03\"}, \"managedFields\": [{\"apiVersion\": \"v1\", \"fieldsType\": \"FieldsV1\", \"fieldsV1\": {\"f:metadata\": {\"f:labels\": {\".\": {}, \"f:app.kubernetes.io/component\": {}, \"f:app.kubernetes.io/managed-by\": {}, \"f:app.kubernetes.io/operator-version\": {}, \"f:app.kubernetes.io/part-of\": {}}, \"f:ownerReferences\": {\".\": {}, \"k:{\\"uid\\":\\"b74d2c66-0e79-4afc-b74a-0f88e3f3dd2d\\"}\": {}}}, \"f:spec\": {\"f:containers\": {\"k:{\\"name\\":\\"awxrestore-2023-12-03-db-management\\"}\": {\".\": {}, \"f:command\": {}, \"f:image\": {}, \"f:imagePullPolicy\": {}, \"f:name\": {}, \"f:resources\": {\".\": {}, \"f:limits\": {\".\": {}, \"f:cpu\": {}, \"f:memory\": {}}, \"f:requests\": {\".\": {}, \"f:cpu\": {}, \"f:memory\": {}}}, \"f:terminationMessagePath\": {}, \"f:terminationMessagePolicy\": {}, \"f:volumeMounts\": {\".\": {}, \"k:{\\"mountPath\\":\\"/backups\\"}\": {\".\": {}, \"f:mountPath\": {}, \"f:name\": {}}}}}, \"f:dnsPolicy\": {}, \"f:enableServiceLinks\": {}, \"f:restartPolicy\": {}, \"f:schedulerName\": {}, \"f:securityContext\": {}, \"f:terminationGracePeriodSeconds\": {}, \"f:volumes\": {\".\": {}, \"k:{\\"name\\":\\"awxrestore-2023-12-03-backup\\"}\": {\".\": {}, \"f:name\": {}, \"f:persistentVolumeClaim\": {\".\": {}, \"f:claimName\": {}}}}}}, \"manager\": \"OpenAPI-Generator\", \"operation\": \"Update\", \"time\": \"2023-12-06T13:37:44Z\"}, {\"apiVersion\": \"v1\", \"fieldsType\": \"FieldsV1\", \"fieldsV1\": {\"f:status\": {\"f:conditions\": {\"k:{\\"type\\":\\"ContainersReady\\"}\": {\".\": {}, \"f:lastProbeTime\": {}, \"f:lastTransitionTime\": {}, \"f:status\": {}, \"f:type\": {}}, \"k:{\\"type\\":\\"Initialized\\"}\": {\".\": {}, \"f:lastProbeTime\": {}, \"f:lastTransitionTime\": {}, \"f:status\": {}, \"f:type\": {}}, \"k:{\\"type\\":\\"Ready\\"}\": {\".\": {}, \"f:lastProbeTime\": {}, \"f:lastTransitionTime\": {}, \"f:status\": {}, \"f:type\": {}}}, \"f:containerStatuses\": {}, \"f:hostIP\": {}, \"f:phase\": {}, \"f:podIP\": {}, \"f:podIPs\": {\".\": {}, \"k:{\\"ip\\":\\"10.42.0.81\\"}\": {\".\": {}, \"f:ip\": {}}}, \"f:startTime\": {}}}, \"manager\": \"k3s\", \"operation\": \"Update\", \"subresource\": \"status\", \"time\": \"2023-12-06T13:37:45Z\"}], \"name\": \"awxrestore-2023-12-03-db-management\", \"namespace\": \"awx\", \"ownerReferences\": [{\"apiVersion\": \"awx.ansible.com/v1beta1\", \"kind\": \"AWXRestore\", \"name\": \"awxrestore-2023-12-03\", \"uid\": \"b74d2c66-0e79-4afc-b74a-0f88e3f3dd2d\"}], \"resourceVersion\": \"79046\", \"uid\": \"0ebd042d-7cac-4cf5-b941-3d3c157bafce\"}, \"spec\": {\"containers\": [{\"command\": [\"sleep\", \"infinity\"], \"image\": \"postgres:13\", \"imagePullPolicy\": \"IfNotPresent\", \"name\": \"awxrestore-2023-12-03-db-management\", \"resources\": {\"limits\": {\"cpu\": \"1\", \"memory\": \"4Gi\"}, \"requests\": {\"cpu\": \"25m\", \"memory\": \"32Mi\"}}, \"terminationMessagePath\": \"/dev/termination-log\", \"terminationMessagePolicy\": \"File\", \"volumeMounts\": [{\"mountPath\": \"/backups\", \"name\": \"awxrestore-2023-12-03-backup\"}, {\"mountPath\": \"/var/run/secrets/kubernetes.io/serviceaccount\", \"name\": \"kube-api-access-fldhr\", \"readOnly\": true}]}], \"dnsPolicy\": \"ClusterFirst\", \"enableServiceLinks\": true, \"nodeName\": \"vmt86rhel937\", \"preemptionPolicy\": \"PreemptLowerPriority\", \"priority\": 0, \"restartPolicy\": \"Never\", \"schedulerName\": \"default-scheduler\", \"securityContext\": {}, \"serviceAccount\": \"default\", \"serviceAccountName\": \"default\", \"terminationGracePeriodSeconds\": 30, \"tolerations\": [{\"effect\": \"NoExecute\", \"key\": \"node.kubernetes.io/not-ready\", \"operator\": \"Exists\", \"tolerationSeconds\": 300}, {\"effect\": \"NoExecute\", \"key\": \"node.kubernetes.io/unreachable\", \"operator\": \"Exists\", \"tolerationSeconds\": 300}], \"volumes\": [{\"name\": \"awxrestore-2023-12-03-backup\", \"persistentVolumeClaim\": {\"claimName\": \"awx-backup-claim\"}}, {\"name\": \"kube-api-access-fldhr\", \"projected\": {\"defaultMode\": 420, \"sources\": [{\"serviceAccountToken\": {\"expirationSeconds\": 3607, \"path\": \"token\"}}, {\"configMap\": {\"items\": [{\"key\": \"ca.crt\", \"path\": \"ca.crt\"}], \"name\": \"kube-root-ca.crt\"}}, {\"downwardAPI\": {\"items\": [{\"fieldRef\": {\"apiVersion\": \"v1\", \"fieldPath\": \"metadata.namespace\"}, \"path\": \"namespace\"}]}}]}}]}, \"status\": {\"conditions\": [{\"lastProbeTime\": null, \"lastTransitionTime\": \"2023-12-06T13:37:44Z\", \"status\": \"True\", \"type\": \"Initialized\"}, {\"lastProbeTime\": null, \"lastTransitionTime\": \"2023-12-06T13:37:45Z\", \"status\": \"True\", \"type\": \"Ready\"}, {\"lastProbeTime\": null, \"lastTransitionTime\": \"2023-12-06T13:37:45Z\", \"status\": \"True\", \"type\": \"ContainersReady\"}, {\"lastProbeTime\": null, \"lastTransitionTime\": \"2023-12-06T13:37:44Z\", \"status\": \"True\", \"type\": \"PodScheduled\"}], \"containerStatuses\": [{\"containerID\": \"containerd://a6d071fb7bf76dc461ea6fef4cf2bf76edda9ea711a52a620bc291132e05adb2\", \"image\": \"docker.io/library/postgres:13\", \"imageID\": \"docker.io/library/postgres@sha256:9167bfa128429fb9c36df04b6d4c2f2d1751f3db105e00d60b514248f959c5bb\", \"lastState\": {}, \"name\": \"awxrestore-2023-12-03-db-management\", \"ready\": true, \"restartCount\": 0, \"started\": true, \"state\": {\"running\": {\"startedAt\": \"2023-12-06T13:37:44Z\"}}}], \"hostIP\": \"10.245.157.108\", \"phase\": \"Running\", \"podIP\": \"10.42.0.81\", \"podIPs\": [{\"ip\": \"10.42.0.81\"}], \"qosClass\": \"Burstable\", \"startTime\": \"2023-12-06T13:37:44Z\"}}}\n\r\nTASK [restore : Set default postgres image] ****\r\ntask path: /opt/ansible/roles/restore/tasks/init.yml:74\nok: [localhost] => {\"ansible_facts\": {\"_default_postgres_image\": \"postgres:13\"}, \"changed\": false}\n\r\nTASK [restore : Set user provided postgres image] **\r\ntask path: /opt/ansible/roles/restore/tasks/init.yml:78\nskipping: [localhost] => {\"changed\": false, \"false_condition\": \"postgres_image | default([]) | length\", \"skip_reason\": \"Conditional result was False\"}\n\r\nTASK [restore : Set Postgres image URL] ****\r\ntask path: /opt/ansible/roles/restore/tasks/init.yml:85\nok: [localhost] => {\"ansible_facts\": {\"_postgres_image\": \"postgres:13\"}, \"changed\": false}\n\r\nTASK [restore : Create management pod from templated deployment config] ****\r\ntask path: /opt/ansible/roles/restore/tasks/init.yml:89\nchanged: [localhost] => {\"changed\": true, \"duration\": 5, \"method\": \"create\", \"result\": {\"apiVersion\": \"v1\", \"kind\": \"Pod\", \"metadata\": {\"creationTimestamp\": \"2023-12-06T13:38:35Z\", \"labels\": {\"app.kubernetes.io/component\": \"awx\", \"app.kubernetes.io/managed-by\": \"awx-operator\", \"app.kubernetes.io/operator-version\": \"2.8.0\", \"app.kubernetes.io/part-of\": \"awxrestore-2023-12-03\"}, \"managedFields\": [{\"apiVersion\": \"v1\", \"fieldsType\": \"FieldsV1\", \"fieldsV1\": {\"f:metadata\": {\"f:labels\": {\".\": {}, \"f:app.kubernetes.io/component\": {}, \"f:app.kubernetes.io/managed-by\": {}, \"f:app.kubernetes.io/operator-version\": {}, \"f:app.kubernetes.io/part-of\": {}}, \"f:ownerReferences\": {\".\": {}, \"k:{\\"uid\\":\\"b74d2c66-0e79-4afc-b74a-0f88e3f3dd2d\\"}\": {}}}, \"f:spec\": {\"f:containers\": {\"k:{\\"name\\":\\"awxrestore-2023-12-03-db-management\\"}\": {\".\": {}, \"f:command\": {}, \"f:image\": {}, \"f:imagePullPolicy\": {}, \"f:name\": {}, \"f:resources\": {\".\": {}, \"f:limits\": {\".\": {}, \"f:cpu\": {}, \"f:memory\": {}}, \"f:requests\": {\".\": {}, \"f:cpu\": {}, \"f:memory\": {}}}, \"f:terminationMessagePath\": {}, \"f:terminationMessagePolicy\": {}, \"f:volumeMounts\": {\".\": {}, \"k:{\\"mountPath\\":\\"/backups\\"}\": {\".\": {}, \"f:mountPath\": {}, \"f:name\": {}}}}}, \"f:dnsPolicy\": {}, \"f:enableServiceLinks\": {}, \"f:restartPolicy\": {}, \"f:schedulerName\": {}, \"f:securityContext\": {}, \"f:terminationGracePeriodSeconds\": {}, \"f:volumes\": {\".\": {}, \"k:{\\"name\\":\\"awxrestore-2023-12-03-backup\\"}\": {\".\": {}, \"f:name\": {}, \"f:persistentVolumeClaim\": {\".\": {}, \"f:claimName\": {}}}}}}, \"manager\": \"OpenAPI-Generator\", \"operation\": \"Update\", \"time\": \"2023-12-06T13:38:35Z\"}, {\"apiVersion\": \"v1\", \"fieldsType\": \"FieldsV1\", \"fieldsV1\": {\"f:status\": {\"f:conditions\": {\"k:{\\"type\\":\\"ContainersReady\\"}\": {\".\": {}, \"f:lastProbeTime\": {}, \"f:lastTransitionTime\": {}, \"f:status\": {}, \"f:type\": {}}, \"k:{\\"type\\":\\"Initialized\\"}\": {\".\": {}, \"f:lastProbeTime\": {}, \"f:lastTransitionTime\": {}, \"f:status\": {}, \"f:type\": {}}, \"k:{\\"type\\":\\"Ready\\"}\": {\".\": {}, \"f:lastProbeTime\": {}, \"f:lastTransitionTime\": {}, \"f:status\": {}, \"f:type\": {}}}, \"f:containerStatuses\": {}, \"f:hostIP\": {}, \"f:phase\": {}, \"f:podIP\": {}, \"f:podIPs\": {\".\": {}, \"k:{\\"ip\\":\\"10.42.0.82\\"}\": {\".\": {}, \"f:ip\": {}}}, \"f:startTime\": {}}}, \"manager\": \"k3s\", \"operation\": \"Update\", \"subresource\": \"status\", \"time\": \"2023-12-06T13:38:36Z\"}], \"name\": \"awxrestore-2023-12-03-db-management\", \"namespace\": \"awx\", \"ownerReferences\": [{\"apiVersion\": \"awx.ansible.com/v1beta1\", \"kind\": \"AWXRestore\", \"name\": \"awxrestore-2023-12-03\", \"uid\": \"b74d2c66-0e79-4afc-b74a-0f88e3f3dd2d\"}], \"resourceVersion\": \"79108\", \"uid\": \"3587f67e-096a-4dab-91af-575c1aebe4f1\"}, \"spec\": {\"containers\": [{\"command\": [\"sleep\", \"infinity\"], \"image\": \"postgres:13\", \"imagePullPolicy\": \"IfNotPresent\", \"name\": \"awxrestore-2023-12-03-db-management\", \"resources\": {\"limits\": {\"cpu\": \"1\", \"memory\": \"4Gi\"}, \"requests\": {\"cpu\": \"25m\", \"memory\": \"32Mi\"}}, \"terminationMessagePath\": \"/dev/termination-log\", \"terminationMessagePolicy\": \"File\", \"volumeMounts\": [{\"mountPath\": \"/backups\", \"name\": \"awxrestore-2023-12-03-backup\"}, {\"mountPath\": \"/var/run/secrets/kubernetes.io/serviceaccount\", \"name\": \"kube-api-access-rsb9b\", \"readOnly\": true}]}], \"dnsPolicy\": \"ClusterFirst\", \"enableServiceLinks\": true, \"nodeName\": \"vmt86rhel937\", \"preemptionPolicy\": \"PreemptLowerPriority\", \"priority\": 0, \"restartPolicy\": \"Never\", \"schedulerName\": \"default-scheduler\", \"securityContext\": {}, \"serviceAccount\": \"default\", \"serviceAccountName\": \"default\", \"terminationGracePeriodSeconds\": 30, \"tolerations\": [{\"effect\": \"NoExecute\", \"key\": \"node.kubernetes.io/not-ready\", \"operator\": \"Exists\", \"tolerationSeconds\": 300}, {\"effect\": \"NoExecute\", \"key\": \"node.kubernetes.io/unreachable\", \"operator\": \"Exists\", \"tolerationSeconds\": 300}], \"volumes\": [{\"name\": \"awxrestore-2023-12-03-backup\", \"persistentVolumeClaim\": {\"claimName\": \"awx-backup-claim\"}}, {\"name\": \"kube-api-access-rsb9b\", \"projected\": {\"defaultMode\": 420, \"sources\": [{\"serviceAccountToken\": {\"expirationSeconds\": 3607, \"path\": \"token\"}}, {\"configMap\": {\"items\": [{\"key\": \"ca.crt\", \"path\": \"ca.crt\"}], \"name\": \"kube-root-ca.crt\"}}, {\"downwardAPI\": {\"items\": [{\"fieldRef\": {\"apiVersion\": \"v1\", \"fieldPath\": \"metadata.namespace\"}, \"path\": \"namespace\"}]}}]}}]}, \"status\": {\"conditions\": [{\"lastProbeTime\": null, \"lastTransitionTime\": \"2023-12-06T13:38:35Z\", \"status\": \"True\", \"type\": \"Initialized\"}, {\"lastProbeTime\": null, \"lastTransitionTime\": \"2023-12-06T13:38:36Z\", \"status\": \"True\", \"type\": \"Ready\"}, {\"lastProbeTime\": null, \"lastTransitionTime\": \"2023-12-06T13:38:36Z\", \"status\": \"True\", \"type\": \"ContainersReady\"}, {\"lastProbeTime\": null, \"lastTransitionTime\": \"2023-12-06T13:38:35Z\", \"status\": \"True\", \"type\": \"PodScheduled\"}], \"containerStatuses\": [{\"containerID\": \"containerd://06f453b00308a8bf50a8120e396e37ae653f77f45d22220dbfb88d47bbe374ca\", \"image\": \"docker.io/library/postgres:13\", \"imageID\": \"docker.io/library/postgres@sha256:9167bfa128429fb9c36df04b6d4c2f2d1751f3db105e00d60b514248f959c5bb\", \"lastState\": {}, \"name\": \"awxrestore-2023-12-03-db-management\", \"ready\": true, \"restartCount\": 0, \"started\": true, \"state\": {\"running\": {\"startedAt\": \"2023-12-06T13:38:36Z\"}}}], \"hostIP\": \"10.245.157.108\", \"phase\": \"Running\", \"podIP\": \"10.42.0.82\", \"podIPs\": [{\"ip\": \"10.42.0.82\"}], \"qosClass\": \"Burstable\", \"startTime\": \"2023-12-06T13:38:35Z\"}}}\n\r\nTASK [restore : Check to make sure backup directory exists on PVC] *\r\ntask path: /opt/ansible/roles/restore/tasks/init.yml:97\n[DEPRECATION WARNING]: The 'return_code' return key is being renamed to 'rc'. \r\nBoth keys are being returned for now to allow users to migrate their \r\nautomation. This feature will be removed from kubernetes.core in version 4.0.0.\r\n Deprecation warnings can be disabled by setting deprecation_warnings=False in \r\nansible.cfg.\r\nfatal: [localhost]: FAILED! => {\"changed\": true, \"rc\": 1, \"return_code\": 1, \"stderr\": \"stat: cannot statx '/backup/tower': No such file or directory\n\", \"stderr_lines\": [\"stat: cannot statx '/backup/tower': No such file or directory\"], \"stdout\": \"\", \"stdout_lines\": []}\n\r\nPLAY RECAP *****\r\nlocalhost : ok=11 changed=2 unreachable=0 failed=1 skipped=12 rescued=0 ignored=0 \n","job":"3768719885826966812","name":"awxrestore-2023-12-03","namespace":"awx","error":"exit status 2","stacktrace":"github.com/operator-framework/operator-sdk/internal/ansible/runner.(*runner).Run.func1\n\t/workspace/internal/ansible/runner/runner.go:269"}
----- Ansible Task Status Event StdOut (awx.ansible.com/v1beta1, Kind=AWXRestore, awxrestore-2023-12-03/awx) -----
PLAY RECAP ***** localhost : ok=11 changed=2 unreachable=0 failed=1 skipped=12 rescued=0 ignored=0
{"level":"error","ts":"2023-12-06T13:38:42Z","msg":"Reconciler error","controller":"awxrestore-controller","object":{"name":"awxrestore-2023-12-03","namespace":"awx"},"namespace":"awx","name":"awxrestore-2023-12-03","reconcileID":"180ed268-ce65-4eab-8c34-95f72f1061bd","error":"event runner on failed","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/controller.(Controller).reconcileHandler\n\t/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.14.5/pkg/internal/controller/controller.go:329\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(Controller).processNextWorkItem\n\t/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.14.5/pkg/internal/controller/controller.go:274\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.14.5/pkg/internal/controller/controller.go:235"} {"level":"info","ts":"2023-12-06T13:38:44Z","logger":"logging_event_handler","msg":"[playbook debug]","name":"awxrestore-2023-12-03","namespace":"awx","gvk":"awx.ansible.com/v1beta1, Kind=AWXRestore","event_type":"runner_on_ok","job":"7840835804381702915","EventData.TaskArgs":""}
❌ :q! :q!
bskou57
commented
7 months ago hoops my fault "s" was missing in backup_dir: "/backups/tower"
bskou57
commented
7 months ago Hi
restore complete successfully now but still have an issue with web interface : "404 page not found"
Thanks for your support
kurokobo
commented
7 months ago restore complete successfully now but still have an issue with web interface : "404 page not found"
I do not have enough information at all. Please do your own investigation and let me know what you have confirmed and tried and the results.
bskou57
commented
7 months ago will do it thanks
bskou57
commented
7 months ago Hi
I just finish the restore from files but I encountered an error when using web interface (error 404)
{"level":"info","ts":"2023-12-07T10:26:05Z","logger":"logging_event_handler","msg":"[playbook task start]","name":"awxrestore-2023-12-03","namespace":"awx","gvk":"awx.ansible.com/v1beta1, Kind=AWXRestore","event_type":"playbook_on_task_start","job":"1442269809012458303","EventData.Name":"restore : Update status variables"}
--------------------------- Ansible Task StdOut -------------------------------
TASK [restore : Update status variables] *** task path: /opt/ansible/roles/restore/tasks/main.yml:54
{"level":"info","ts":"2023-12-07T10:26:05Z","logger":"logging_event_handler","msg":"[playbook task start]","name":"awxrestore-2023-12-03","namespace":"awx","gvk":"awx.ansible.com/v1beta1, Kind=AWXRestore","event_type":"playbook_on_task_start","job":"1442269809012458303","EventData.Name":"restore : Update CR Restore status"}
--------------------------- Ansible Task StdOut -------------------------------
TASK [restore : Update CR Restore status] ** task path: /opt/ansible/roles/restore/tasks/update_status.yml:3
{"level":"info","ts":"2023-12-07T10:26:05Z","logger":"runner","msg":"Ansible-runner exited successfully","job":"1442269809012458303","name":"awxrestore-2023-12-03","namespace":"awx"}
----- Ansible Task Status Event StdOut (awx.ansible.com/v1beta1, Kind=AWXRestore, awxrestore-2023-12-03/awx) -----
PLAY RECAP ***** localhost : ok=6 changed=0 unreachable=0 failed=0 skipped=9 rescued=0 ignored=0
[IAAS - root@vmt86rhel937 ~]$ kubectl get pods -n awx NAME READY STATUS RESTARTS AGE awx-operator-controller-manager-6678865c69-xpjhl 2/2 Running 0 120m awx-postgres-13-0 1/1 Running 0 8m25s awx-task-58d8f4dcfb-snh9m 4/4 Running 0 8m4s awx-web-6497bc6fd8-pc85p 3/3 Running 0 7m56s
Was not able to lgin to web interface: 404 page not found
Thanks for your support
kurokobo
commented
7 months ago
Environment
$ k3s --version k3s version v1.27.7+k3s2 (575bce76) go version go1.20.10
Description
Hello Kurokobo
One more issue from my side
I tryed to restore backup from different awx instance but get an error message 👍
TASK [Check to make sure backup directory exists on PVC] **** fatal: [localhost]: FAILED! => {"changed": true, "rc": 1, "return_code": 1, "stderr": "stat: cannot statx '/data/backup/tower': No such file or directory\n", "stderr_lines": ["stat: cannot statx '/data/backup/tower': No such file or directory"], "stdout": "", "stdout_lines": []}
Thanks for your support
Step to Reproduce
install new awx instance with awx-on-k3s Try to restore from backup files (without pv objects)
kubectl apply -k restore
kubectl apply -f restore/awxrestore.yaml
[IAAS - root@vmt86rhel937 awx-on-k3s]$ cat restore/awxrestore.yaml
apiVersion: awx.ansible.com/v1beta1 kind: AWXRestore metadata: name: awxrestore-2023-12-03 namespace: awx spec: deployment_name: awx
Parameters to restore from AWXBackup object
backup_name: awxbackup-2023-26-11
Parameters to restore from existing files on PVC (without AWXBackup object)
backup_pvc: awx-backup-claim backup_dir: /data/backup/tower
Uncomment to reveal "censored" logs
no_log: false
Logs
Normal Scheduled 27s default-scheduler Successfully assigned awx/awxrestore-2023-12-03-db-management to vmt86rhel937 Normal Pulled 27s kubelet Container image "postgres:13" already present on machine Normal Created 27s kubelet Created container awxrestore-2023-12-03-db-management Normal Started 27s kubelet Started container awxrestore-2023-12-03-db-management Normal Killing 13s kubelet Stopping container awxrestore-2023-12-03-db-management [IAAS - root@vmt86rhel937 awx-on-k3s]$ kubectl get awxrestore -n awx NAME AGE awxrestore-2023-12-03 51m [IAAS - root@vmt86rhel937 awx-on-k3s]$
{"level":"error","ts":"2023-12-06T10:27:43Z","logger":"logging_event_handler","msg":"","name":"awxrestore-2023-12-03","namespace":"awx","gvk":"awx.ansible.com/v1beta1, Kind=AWXRestore","event_type":"runner_on_failed","job":"1906987945759275168","EventData.Task":"Check to make sure backup directory exists on PVC","EventData.TaskArgs":"","EventData.FailedTaskPath":"/opt/ansible/roles/restore/tasks/init.yml:97","error":"[playbook task failed]","stacktrace":"github.com/operator-framework/operator-sdk/internal/ansible/events.loggingEventHandler.Handle\n\t/workspace/internal/ansible/events/log_events.go:111"} {"level":"error","ts":"2023-12-06T10:27:43Z","logger":"runner","msg":"ansible-playbook [core 2.15.2]\r\n config file = /etc/ansible/ansible.cfg\r\n configured module search path = ['/usr/share/ansible/openshift']\r\n ansible python module location = /usr/local/lib/python3.9/site-packages/ansible\r\n ansible collection location = /opt/ansible/.ansible/collections:/usr/share/ansible/collections\r\n executable location = /usr/local/bin/ansible-playbook\r\n python version = 3.9.16 (main, May 31 2023, 12:21:58) [GCC 8.5.0 20210514 (Red Hat 8.5.0-18)] (/usr/bin/python3)\r\n jinja version = 3.1.2\r\n libyaml = True\r\nUsing /etc/ansible/ansible.cfg as config file\r\nSkipping callback 'awx_display', as we already have a stdout callback.\nSkipping callback 'default', as we already have a stdout callback.\nSkipping callback 'minimal', as we already have a stdout callback.\nSkipping callback 'oneline', as we already have a stdout callback.\n\r\nPLAYBOOK: ea29887dbbeb411998d22bf901c68de3 *\n1 plays in /tmp/ansible-operator/runner/awx.ansible.com/v1beta1/AWXRestore/awx/awxrestore-2023-12-03/project/ea29887dbbeb411998d22bf901c68de3\n[WARNING]: Found variable using reserved name: no_log\n\r\nPLAY [localhost] ***\n\r\nTASK [common : Get information about the cluster] **\r\ntask path: /opt/ansible/roles/common/tasks/main.yml:3\nredirecting (type: lookup) ansible.builtin.k8s to kubernetes.core.k8s\nok: [localhost] => {\"ansible_facts\": {\"api_groups\": [\"\", \"apiregistration.k8s.io\", \"apps\", \"events.k8s.io\", \"authentication.k8s.io\", \"authorization.k8s.io\", \"autoscaling\", \"batch\", \"certificates.k8s.io\", \"networking.k8s.io\", \"policy\", \"rbac.authorization.k8s.io\", \"storage.k8s.io\", \"admissionregistration.k8s.io\", \"apiextensions.k8s.io\", \"scheduling.k8s.io\", \"coordination.k8s.io\", \"node.k8s.io\", \"discovery.k8s.io\", \"flowcontrol.apiserver.k8s.io\", \"helm.cattle.io\", \"k3s.cattle.io\", \"traefik.containo.us\", \"traefik.io\", \"awx.ansible.com\", \"metrics.k8s.io\"]}, \"changed\": false}\n\r\nTASK [common : Determine the cluster type] ***\r\ntask path: /opt/ansible/roles/common/tasks/main.yml:10\nok: [localhost] => {\"ansible_facts\": {\"is_k8s\": true, \"is_openshift\": false}, \"changed\": false}\n\r\nTASK [common : debug] **\r\ntask path: /opt/ansible/roles/common/tasks/main.yml:19\nok: [localhost] => {\r\n \"msg\": \"CLUSTER TYPE: is_openshift=False; is_k8s=True\"\r\n}\n\r\nTASK [restore : Patching labels to AWXRestore kind] ****\r\ntask path: /opt/ansible/roles/restore/tasks/main.yml:2\nok: [localhost] => {\"changed\": false, \"method\": \"update\", \"result\": {\"apiVersion\": \"awx.ansible.com/v1beta1\", \"kind\": \"AWXRestore\", \"metadata\": {\"annotations\": {\"kubectl.kubernetes.io/last-applied-configuration\": \"{\\"apiVersion\\":\\"awx.ansible.com/v1beta1\\",\\"kind\\":\\"AWXRestore\\",\\"metadata\\":{\\"annotations\\":{},\\"name\\":\\"awxrestore-2023-12-03\\",\\"namespace\\":\\"awx\\"},\\"spec\\":{\\"backup_dir\\":\\"/data/backup/tower\\",\\"backup_pvc\\":\\"awx-backup-claim\\",\\"deployment_name\\":\\"awx\\",\\"no_log\\":false}}\n\"}, \"creationTimestamp\": \"2023-12-06T09:43:54Z\", \"generation\": 2, \"labels\": {\"app.kubernetes.io/component\": \"awx\", \"app.kubernetes.io/managed-by\": \"awx-operator\", \"app.kubernetes.io/operator-version\": \"2.8.0\", \"app.kubernetes.io/part-of\": \"awxrestore-2023-12-03\"}, \"managedFields\": [{\"apiVersion\": \"awx.ansible.com/v1beta1\", \"fieldsType\": \"FieldsV1\", \"fieldsV1\": {\"f:metadata\": {\"f:labels\": {\".\": {}, \"f:app.kubernetes.io/component\": {}, \"f:app.kubernetes.io/managed-by\": {}, \"f:app.kubernetes.io/operator-version\": {}, \"f:app.kubernetes.io/part-of\": {}}}}, \"manager\": \"OpenAPI-Generator\", \"operation\": \"Update\", \"time\": \"2023-12-06T09:43:58Z\"}, {\"apiVersion\": \"awx.ansible.com/v1beta1\", \"fieldsType\": \"FieldsV1\", \"fieldsV1\": {\"f:metadata\": {\"f:annotations\": {\".\": {}, \"f:kubectl.kubernetes.io/last-applied-configuration\": {}}}, \"f:spec\": {\".\": {}, \"f:backup_dir\": {}, \"f:backup_pvc\": {}, \"f:deployment_name\": {}, \"f:image_pull_policy\": {}, \"f:no_log\": {}, \"f:set_self_labels\": {}}}, \"manager\": \"kubectl-client-side-apply\", \"operation\": \"Update\", \"time\": \"2023-12-06T09:46:53Z\"}, {\"apiVersion\": \"awx.ansible.com/v1beta1\", \"fieldsType\": \"FieldsV1\", \"fieldsV1\": {\"f:status\": {\".\": {}, \"f:conditions\": {}}}, \"manager\": \"ansible-operator\", \"operation\": \"Update\", \"subresource\": \"status\", \"time\": \"2023-12-06T10:26:51Z\"}], \"name\": \"awxrestore-2023-12-03\", \"namespace\": \"awx\", \"resourceVersion\": \"59726\", \"uid\": \"b74d2c66-0e79-4afc-b74a-0f88e3f3dd2d\"}, \"spec\": {\"backup_dir\": \"/data/backup/tower\", \"backup_pvc\": \"awx-backup-claim\", \"deployment_name\": \"awx\", \"image_pull_policy\": \"IfNotPresent\", \"no_log\": false, \"set_self_labels\": true}, \"status\": {\"conditions\": [{\"lastTransitionTime\": \"2023-12-06T09:44:04Z\", \"reason\": \"\", \"status\": \"False\", \"type\": \"Successful\"}, {\"lastTransitionTime\": \"2023-12-06T10:26:51Z\", \"reason\": \"Failed\", \"status\": \"False\", \"type\": \"Failure\"}, {\"lastTransitionTime\": \"2023-12-06T10:26:51Z\", \"reason\": \"Running\", \"status\": \"True\", \"type\": \"Running\"}]}}}\n\r\nTASK [restore : Look up details for this restore object] *\r\ntask path: /opt/ansible/roles/restore/tasks/main.yml:16\nok: [localhost] => {\"api_found\": true, \"changed\": false, \"resources\": [{\"apiVersion\": \"awx.ansible.com/v1beta1\", \"kind\": \"AWXRestore\", \"metadata\": {\"annotations\": {\"kubectl.kubernetes.io/last-applied-configuration\": \"{\\"apiVersion\\":\\"awx.ansible.com/v1beta1\\",\\"kind\\":\\"AWXRestore\\",\\"metadata\\":{\\"annotations\\":{},\\"name\\":\\"awxrestore-2023-12-03\\",\\"namespace\\":\\"awx\\"},\\"spec\\":{\\"backup_dir\\":\\"/data/backup/tower\\",\\"backup_pvc\\":\\"awx-backup-claim\\",\\"deployment_name\\":\\"awx\\",\\"no_log\\":false}}\n\"}, \"creationTimestamp\": \"2023-12-06T09:43:54Z\", \"generation\": 2, \"labels\": {\"app.kubernetes.io/component\": \"awx\", \"app.kubernetes.io/managed-by\": \"awx-operator\", \"app.kubernetes.io/operator-version\": \"2.8.0\", \"app.kubernetes.io/part-of\": \"awxrestore-2023-12-03\"}, \"managedFields\": [{\"apiVersion\": \"awx.ansible.com/v1beta1\", \"fieldsType\": \"FieldsV1\", \"fieldsV1\": {\"f:metadata\": {\"f:labels\": {\".\": {}, \"f:app.kubernetes.io/component\": {}, \"f:app.kubernetes.io/managed-by\": {}, \"f:app.kubernetes.io/operator-version\": {}, \"f:app.kubernetes.io/part-of\": {}}}}, \"manager\": \"OpenAPI-Generator\", \"operation\": \"Update\", \"time\": \"2023-12-06T09:43:58Z\"}, {\"apiVersion\": \"awx.ansible.com/v1beta1\", \"fieldsType\": \"FieldsV1\", \"fieldsV1\": {\"f:metadata\": {\"f:annotations\": {\".\": {}, \"f:kubectl.kubernetes.io/last-applied-configuration\": {}}}, \"f:spec\": {\".\": {}, \"f:backup_dir\": {}, \"f:backup_pvc\": {}, \"f:deployment_name\": {}, \"f:image_pull_policy\": {}, \"f:no_log\": {}, \"f:set_self_labels\": {}}}, \"manager\": \"kubectl-client-side-apply\", \"operation\": \"Update\", \"time\": \"2023-12-06T09:46:53Z\"}, {\"apiVersion\": \"awx.ansible.com/v1beta1\", \"fieldsType\": \"FieldsV1\", \"fieldsV1\": {\"f:status\": {\".\": {}, \"f:conditions\": {}}}, \"manager\": \"ansible-operator\", \"operation\": \"Update\", \"subresource\": \"status\", \"time\": \"2023-12-06T10:26:51Z\"}], \"name\": \"awxrestore-2023-12-03\", \"namespace\": \"awx\", \"resourceVersion\": \"59726\", \"uid\": \"b74d2c66-0e79-4afc-b74a-0f88e3f3dd2d\"}, \"spec\": {\"backup_dir\": \"/data/backup/tower\", \"backup_pvc\": \"awx-backup-claim\", \"deployment_name\": \"awx\", \"image_pull_policy\": \"IfNotPresent\", \"no_log\": false, \"set_self_labels\": true}, \"status\": {\"conditions\": [{\"lastTransitionTime\": \"2023-12-06T09:44:04Z\", \"reason\": \"\", \"status\": \"False\", \"type\": \"Successful\"}, {\"lastTransitionTime\": \"2023-12-06T10:26:51Z\", \"reason\": \"Failed\", \"status\": \"False\", \"type\": \"Failure\"}, {\"lastTransitionTime\": \"2023-12-06T10:26:51Z\", \"reason\": \"Running\", \"status\": \"True\", \"type\": \"Running\"}]}}]}\n\r\nTASK [restore : Build
additional_labels_itemslabels fromadditional_labels] *\r\ntask path: /opt/ansible/roles/restore/tasks/main.yml:24\nskipping: [localhost] => {\"changed\": false, \"false_condition\": \"additional_labels | length\", \"skip_reason\": \"Conditional result was False\"}\n\r\nTASK [restore : include_tasks] ***\r\ntask path: /opt/ansible/roles/restore/tasks/main.yml:35\nstatically imported: /opt/ansible/roles/restore/tasks/error_handling.yml\nstatically imported: /opt/ansible/roles/restore/tasks/error_handling.yml\nstatically imported: /opt/ansible/roles/restore/tasks/error_handling.yml\nincluded: /opt/ansible/roles/restore/tasks/init.yml for localhost\n\r\nTASK [restore : Look up details for the backup] ****\r\ntask path: /opt/ansible/roles/restore/tasks/init.yml:5\nskipping: [localhost] => {\"changed\": false, \"false_condition\": \"backup_name is defined and backup_name != ''\", \"skip_reason\": \"Conditional result was False\"}\n\r\nTASK [restore : Set error message] ***\r\ntask path: /opt/ansible/roles/restore/tasks/init.yml:15\nskipping: [localhost] => {\"changed\": false, \"false_condition\": \"backup_name is defined and backup_name != ''\", \"skip_reason\": \"Conditional result was False\"}\n\r\nTASK [restore : Determine the timestamp] *\r\ntask path: /opt/ansible/roles/restore/tasks/error_handling.yml:3\nskipping: [localhost] => {\"changed\": false, \"false_condition\": \"backup_name is defined and backup_name != ''\", \"skip_reason\": \"Conditional result was False\"}\n\r\nTASK [restore : Emit ocp event with error] ***\r\ntask path: /opt/ansible/roles/restore/tasks/error_handling.yml:7\nskipping: [localhost] => {\"changed\": false, \"false_condition\": \"backup_name is defined and backup_name != ''\", \"skip_reason\": \"Conditional result was False\"}\n\r\nTASK [restore : Fail early if pvc is defined but does not exist] *\r\ntask path: /opt/ansible/roles/restore/tasks/init.yml:22\nskipping: [localhost] => {\"changed\": false, \"false_condition\": \"backup_name is defined and backup_name != ''\", \"skip_reason\": \"Conditional result was False\"}\n\r\nTASK [restore : Set backup facts] **\r\ntask path: /opt/ansible/roles/restore/tasks/init.yml:32\nskipping: [localhost] => {\"changed\": false, \"false_condition\": \"backup_name is defined and backup_name != ''\", \"skip_reason\": \"Conditional result was False\"}\n\r\nTASK [restore : Check provided PVC exists] *\r\ntask path: /opt/ansible/roles/restore/tasks/init.yml:40\nok: [localhost] => {\"api_found\": true, \"changed\": false, \"resources\": [{\"apiVersion\": \"v1\", \"kind\": \"PersistentVolumeClaim\", \"metadata\": {\"annotations\": {\"kubectl.kubernetes.io/last-applied-configuration\": \"{\\"apiVersion\\":\\"v1\\",\\"kind\\":\\"PersistentVolumeClaim\\",\\"metadata\\":{\\"annotations\\":{},\\"name\\":\\"awx-backup-claim\\",\\"namespace\\":\\"awx\\"},\\"spec\\":{\\"accessModes\\":[\\"ReadWriteOnce\\"],\\"resources\\":{\\"requests\\":{\\"storage\\":\\"4Gi\\"}},\\"storageClassName\\":\\"awx-backup-volume\\",\\"volumeMode\\":\\"Filesystem\\"}}\n\", \"pv.kubernetes.io/bind-completed\": \"yes\", \"pv.kubernetes.io/bound-by-controller\": \"yes\"}, \"creationTimestamp\": \"2023-12-05T22:36:11Z\", \"finalizers\": [\"kubernetes.io/pvc-protection\"], \"managedFields\": [{\"apiVersion\": \"v1\", \"fieldsType\": \"FieldsV1\", \"fieldsV1\": {\"f:metadata\": {\"f:annotations\": {\"f:pv.kubernetes.io/bind-completed\": {}, \"f:pv.kubernetes.io/bound-by-controller\": {}}}, \"f:spec\": {\"f:volumeName\": {}}}, \"manager\": \"k3s\", \"operation\": \"Update\", \"time\": \"2023-12-05T22:36:11Z\"}, {\"apiVersion\": \"v1\", \"fieldsType\": \"FieldsV1\", \"fieldsV1\": {\"f:status\": {\"f:accessModes\": {}, \"f:capacity\": {\".\": {}, \"f:storage\": {}}, \"f:phase\": {}}}, \"manager\": \"k3s\", \"operation\": \"Update\", \"subresource\": \"status\", \"time\": \"2023-12-05T22:36:11Z\"}, {\"apiVersion\": \"v1\", \"fieldsType\": \"FieldsV1\", \"fieldsV1\": {\"f:metadata\": {\"f:annotations\": {\".\": {}, \"f:kubectl.kubernetes.io/last-applied-configuration\": {}}}, \"f:spec\": {\"f:accessModes\": {}, \"f:resources\": {\"f:requests\": {\".\": {}, \"f:storage\": {}}}, \"f:storageClassName\": {}, \"f:volumeMode\": {}}}, \"manager\": \"kubectl-client-side-apply\", \"operation\": \"Update\", \"time\": \"2023-12-05T22:36:11Z\"}], \"name\": \"awx-backup-claim\", \"namespace\": \"awx\", \"resourceVersion\": \"2589\", \"uid\": \"366e3c25-52c3-42d5-a5ff-d5deb176d4e1\"}, \"spec\": {\"accessModes\": [\"ReadWriteOnce\"], \"resources\": {\"requests\": {\"storage\": \"4Gi\"}}, \"storageClassName\": \"awx-backup-volume\", \"volumeMode\": \"Filesystem\", \"volumeName\": \"awx-backup-volume\"}, \"status\": {\"accessModes\": [\"ReadWriteOnce\"], \"capacity\": {\"storage\": \"4Gi\"}, \"phase\": \"Bound\"}}]}\n\r\nTASK [restore : Set error message] *****\r\ntask path: /opt/ansible/roles/restore/tasks/init.yml:51\nskipping: [localhost] => {\"changed\": false, \"false_condition\": \"provided_pvc.resources | length == 0\", \"skip_reason\": \"Conditional result was False\"}\n\r\nTASK [restore : Determine the timestamp] **\r\ntask path: /opt/ansible/roles/restore/tasks/error_handling.yml:3\nskipping: [localhost] => {\"changed\": false, \"false_condition\": \"provided_pvc.resources | length == 0\", \"skip_reason\": \"Conditional result was False\"}\n\r\nTASK [restore : Emit ocp event with error] \r\ntask path: /opt/ansible/roles/restore/tasks/error_handling.yml:7\nskipping: [localhost] => {\"changed\": false, \"false_condition\": \"provided_pvc.resources | length == 0\", \"skip_reason\": \"Conditional result was False\"}\n\r\nTASK [restore : Fail early if pvc is defined but does not exist] *\r\ntask path: /opt/ansible/roles/restore/tasks/init.yml:58\nskipping: [localhost] => {\"changed\": false, \"false_condition\": \"provided_pvc.resources | length == 0\", \"skip_reason\": \"Conditional result was False\"}\n\r\nTASK [restore : Delete any existing management pod] **\r\ntask path: /opt/ansible/roles/restore/tasks/init.yml:65\nchanged: [localhost] => {\"changed\": true, \"duration\": 35, \"method\": \"delete\", \"result\": {\"apiVersion\": \"v1\", \"kind\": \"Pod\", \"metadata\": {\"creationTimestamp\": \"2023-12-06T10:26:45Z\", \"deletionGracePeriodSeconds\": 30, \"deletionTimestamp\": \"2023-12-06T10:27:30Z\", \"labels\": {\"app.kubernetes.io/component\": \"awx\", \"app.kubernetes.io/managed-by\": \"awx-operator\", \"app.kubernetes.io/operator-version\": \"2.8.0\", \"app.kubernetes.io/part-of\": \"awxrestore-2023-12-03\"}, \"managedFields\": [{\"apiVersion\": \"v1\", \"fieldsType\": \"FieldsV1\", \"fieldsV1\": {\"f:metadata\": {\"f:labels\": {\".\": {}, \"f:app.kubernetes.io/component\": {}, \"f:app.kubernetes.io/managed-by\": {}, \"f:app.kubernetes.io/operator-version\": {}, \"f:app.kubernetes.io/part-of\": {}}, \"f:ownerReferences\": {\".\": {}, \"k:{\\"uid\\":\\"b74d2c66-0e79-4afc-b74a-0f88e3f3dd2d\\"}\": {}}}, \"f:spec\": {\"f:containers\": {\"k:{\\"name\\":\\"awxrestore-2023-12-03-db-management\\"}\": {\".\": {}, \"f:command\": {}, \"f:image\": {}, \"f:imagePullPolicy\": {}, \"f:name\": {}, \"f:resources\": {\".\": {}, \"f:limits\": {\".\": {}, \"f:cpu\": {}, \"f:memory\": {}}, \"f:requests\": {\".\": {}, \"f:cpu\": {}, \"f:memory\": {}}}, \"f:terminationMessagePath\": {}, \"f:terminationMessagePolicy\": {}, \"f:volumeMounts\": {\".\": {}, \"k:{\\"mountPath\\":\\"/backups\\"}\": {\".\": {}, \"f:mountPath\": {}, \"f:name\": {}}}}}, \"f:dnsPolicy\": {}, \"f:enableServiceLinks\": {}, \"f:restartPolicy\": {}, \"f:schedulerName\": {}, \"f:securityContext\": {}, \"f:terminationGracePeriodSeconds\": {}, \"f:volumes\": {\".\": {}, \"k:{\\"name\\":\\"awxrestore-2023-12-03-backup\\"}\": {\".\": {}, \"f:name\": {}, \"f:persistentVolumeClaim\": {\".\": {}, \"f:claimName\": {}}}}}}, \"manager\": \"OpenAPI-Generator\", \"operation\": \"Update\", \"time\": \"2023-12-06T10:26:45Z\"}, {\"apiVersion\": \"v1\", \"fieldsType\": \"FieldsV1\", \"fieldsV1\": {\"f:status\": {\"f:conditions\": {\"k:{\\"type\\":\\"ContainersReady\\"}\": {\".\": {}, \"f:lastProbeTime\": {}, \"f:lastTransitionTime\": {}, \"f:status\": {}, \"f:type\": {}}, \"k:{\\"type\\":\\"Initialized\\"}\": {\".\": {}, \"f:lastProbeTime\": {}, \"f:lastTransitionTime\": {}, \"f:status\": {}, \"f:type\": {}}, \"k:{\\"type\\":\\"Ready\\"}\": {\".\": {}, \"f:lastProbeTime\": {}, \"f:lastTransitionTime\": {}, \"f:status\": {}, \"f:type\": {}}}, \"f:containerStatuses\": {}, \"f:hostIP\": {}, \"f:phase\": {}, \"f:podIP\": {}, \"f:podIPs\": {\".\": {}, \"k:{\\"ip\\":\\"10.42.0.106\\"}\": {\".\": {}, \"f:ip\": {}}}, \"f:startTime\": {}}}, \"manager\": \"k3s\", \"operation\": \"Update\", \"subresource\": \"status\", \"time\": \"2023-12-06T10:26:46Z\"}], \"name\": \"awxrestore-2023-12-03-db-management\", \"namespace\": \"awx\", \"ownerReferences\": [{\"apiVersion\": \"awx.ansible.com/v1beta1\", \"kind\": \"AWXRestore\", \"name\": \"awxrestore-2023-12-03\", \"uid\": \"b74d2c66-0e79-4afc-b74a-0f88e3f3dd2d\"}], \"resourceVersion\": \"59737\", \"uid\": \"784e7ae0-e20e-4183-9611-776fa2580295\"}, \"spec\": {\"containers\": [{\"command\": [\"sleep\", \"infinity\"], \"image\": \"postgres:13\", \"imagePullPolicy\": \"IfNotPresent\", \"name\": \"awxrestore-2023-12-03-db-management\", \"resources\": {\"limits\": {\"cpu\": \"1\", \"memory\": \"4Gi\"}, \"requests\": {\"cpu\": \"25m\", \"memory\": \"32Mi\"}}, \"terminationMessagePath\": \"/dev/termination-log\", \"terminationMessagePolicy\": \"File\", \"volumeMounts\": [{\"mountPath\": \"/backups\", \"name\": \"awxrestore-2023-12-03-backup\"}, {\"mountPath\": \"/var/run/secrets/kubernetes.io/serviceaccount\", \"name\": \"kube-api-access-75bnr\", \"readOnly\": true}]}], \"dnsPolicy\": \"ClusterFirst\", \"enableServiceLinks\": true, \"nodeName\": \"vmt86rhel937\", \"preemptionPolicy\": \"PreemptLowerPriority\", \"priority\": 0, \"restartPolicy\": \"Never\", \"schedulerName\": \"default-scheduler\", \"securityContext\": {}, \"serviceAccount\": \"default\", \"serviceAccountName\": \"default\", \"terminationGracePeriodSeconds\": 30, \"tolerations\": [{\"effect\": \"NoExecute\", \"key\": \"node.kubernetes.io/not-ready\", \"operator\": \"Exists\", \"tolerationSeconds\": 300}, {\"effect\": \"NoExecute\", \"key\": \"node.kubernetes.io/unreachable\", \"operator\": \"Exists\", \"tolerationSeconds\": 300}], \"volumes\": [{\"name\": \"awxrestore-2023-12-03-backup\", \"persistentVolumeClaim\": {\"claimName\": \"awx-backup-claim\"}}, {\"name\": \"kube-api-access-75bnr\", \"projected\": {\"defaultMode\": 420, \"sources\": [{\"serviceAccountToken\": {\"expirationSeconds\": 3607, \"path\": \"token\"}}, {\"configMap\": {\"items\": [{\"key\": \"ca.crt\", \"path\": \"ca.crt\"}], \"name\": \"kube-root-ca.crt\"}}, {\"downwardAPI\": {\"items\": [{\"fieldRef\": {\"apiVersion\": \"v1\", \"fieldPath\": \"metadata.namespace\"}, \"path\": \"namespace\"}]}}]}}]}, \"status\": {\"conditions\": [{\"lastProbeTime\": null, \"lastTransitionTime\": \"2023-12-06T10:26:45Z\", \"status\": \"True\", \"type\": \"Initialized\"}, {\"lastProbeTime\": null, \"lastTransitionTime\": \"2023-12-06T10:26:46Z\", \"status\": \"True\", \"type\": \"Ready\"}, {\"lastProbeTime\": null, \"lastTransitionTime\": \"2023-12-06T10:26:46Z\", \"status\": \"True\", \"type\": \"ContainersReady\"}, {\"lastProbeTime\": null, \"lastTransitionTime\": \"2023-12-06T10:26:45Z\", \"status\": \"True\", \"type\": \"PodScheduled\"}], \"containerStatuses\": [{\"containerID\": \"containerd://e07d9514fe39e5e996d541dfd7cadf40649d04bb55863e32239b9f3ac0478137\", \"image\": \"docker.io/library/postgres:13\", \"imageID\": \"docker.io/library/postgres@sha256:9167bfa128429fb9c36df04b6d4c2f2d1751f3db105e00d60b514248f959c5bb\", \"lastState\": {}, \"name\": \"awxrestore-2023-12-03-db-management\", \"ready\": true, \"restartCount\": 0, \"started\": true, \"state\": {\"running\": {\"startedAt\": \"2023-12-06T10:26:45Z\"}}}], \"hostIP\": \"10.245.157.108\", \"phase\": \"Running\", \"podIP\": \"10.42.0.106\", \"podIPs\": [{\"ip\": \"10.42.0.106\"}], \"qosClass\": \"Burstable\", \"startTime\": \"2023-12-06T10:26:45Z\"}}}\n\r\nTASK [restore : Set default postgres image] ****\r\ntask path: /opt/ansible/roles/restore/tasks/init.yml:74\nok: [localhost] => {\"ansible_facts\": {\"_default_postgres_image\": \"postgres:13\"}, \"changed\": false}\n\r\nTASK [restore : Set user provided postgres image] **\r\ntask path: /opt/ansible/roles/restore/tasks/init.yml:78\nskipping: [localhost] => {\"changed\": false, \"false_condition\": \"postgres_image | default([]) | length\", \"skip_reason\": \"Conditional result was False\"}\n\r\nTASK [restore : Set Postgres image URL] ****\r\ntask path: /opt/ansible/roles/restore/tasks/init.yml:85\nok: [localhost] => {\"ansible_facts\": {\"_postgres_image\": \"postgres:13\"}, \"changed\": false}\n\r\nTASK [restore : Create management pod from templated deployment config] ****\r\ntask path: /opt/ansible/roles/restore/tasks/init.yml:89\nchanged: [localhost] => {\"changed\": true, \"duration\": 5, \"method\": \"create\", \"result\": {\"apiVersion\": \"v1\", \"kind\": \"Pod\", \"metadata\": {\"creationTimestamp\": \"2023-12-06T10:27:36Z\", \"labels\": {\"app.kubernetes.io/component\": \"awx\", \"app.kubernetes.io/managed-by\": \"awx-operator\", \"app.kubernetes.io/operator-version\": \"2.8.0\", \"app.kubernetes.io/part-of\": \"awxrestore-2023-12-03\"}, \"managedFields\": [{\"apiVersion\": \"v1\", \"fieldsType\": \"FieldsV1\", \"fieldsV1\": {\"f:metadata\": {\"f:labels\": {\".\": {}, \"f:app.kubernetes.io/component\": {}, \"f:app.kubernetes.io/managed-by\": {}, \"f:app.kubernetes.io/operator-version\": {}, \"f:app.kubernetes.io/part-of\": {}}, \"f:ownerReferences\": {\".\": {}, \"k:{\\"uid\\":\\"b74d2c66-0e79-4afc-b74a-0f88e3f3dd2d\\"}\": {}}}, \"f:spec\": {\"f:containers\": {\"k:{\\"name\\":\\"awxrestore-2023-12-03-db-management\\"}\": {\".\": {}, \"f:command\": {}, \"f:image\": {}, \"f:imagePullPolicy\": {}, \"f:name\": {}, \"f:resources\": {\".\": {}, \"f:limits\": {\".\": {}, \"f:cpu\": {}, \"f:memory\": {}}, \"f:requests\": {\".\": {}, \"f:cpu\": {}, \"f:memory\": {}}}, \"f:terminationMessagePath\": {}, \"f:terminationMessagePolicy\": {}, \"f:volumeMounts\": {\".\": {}, \"k:{\\"mountPath\\":\\"/backups\\"}\": {\".\": {}, \"f:mountPath\": {}, \"f:name\": {}}}}}, \"f:dnsPolicy\": {}, \"f:enableServiceLinks\": {}, \"f:restartPolicy\": {}, \"f:schedulerName\": {}, \"f:securityContext\": {}, \"f:terminationGracePeriodSeconds\": {}, \"f:volumes\": {\".\": {}, \"k:{\\"name\\":\\"awxrestore-2023-12-03-backup\\"}\": {\".\": {}, \"f:name\": {}, \"f:persistentVolumeClaim\": {\".\": {}, \"f:claimName\": {}}}}}}, \"manager\": \"OpenAPI-Generator\", \"operation\": \"Update\", \"time\": \"2023-12-06T10:27:36Z\"}, {\"apiVersion\": \"v1\", \"fieldsType\": \"FieldsV1\", \"fieldsV1\": {\"f:status\": {\"f:conditions\": {\"k:{\\"type\\":\\"ContainersReady\\"}\": {\".\": {}, \"f:lastProbeTime\": {}, \"f:lastTransitionTime\": {}, \"f:status\": {}, \"f:type\": {}}, \"k:{\\"type\\":\\"Initialized\\"}\": {\".\": {}, \"f:lastProbeTime\": {}, \"f:lastTransitionTime\": {}, \"f:status\": {}, \"f:type\": {}}, \"k:{\\"type\\":\\"Ready\\"}\": {\".\": {}, \"f:lastProbeTime\": {}, \"f:lastTransitionTime\": {}, \"f:status\": {}, \"f:type\": {}}}, \"f:containerStatuses\": {}, \"f:hostIP\": {}, \"f:phase\": {}, \"f:podIP\": {}, \"f:podIPs\": {\".\": {}, \"k:{\\"ip\\":\\"10.42.0.107\\"}\": {\".\": {}, \"f:ip\": {}}}, \"f:startTime\": {}}}, \"manager\": \"k3s\", \"operation\": \"Update\", \"subresource\": \"status\", \"time\": \"2023-12-06T10:27:37Z\"}], \"name\": \"awxrestore-2023-12-03-db-management\", \"namespace\": \"awx\", \"ownerReferences\": [{\"apiVersion\": \"awx.ansible.com/v1beta1\", \"kind\": \"AWXRestore\", \"name\": \"awxrestore-2023-12-03\", \"uid\": \"b74d2c66-0e79-4afc-b74a-0f88e3f3dd2d\"}], \"resourceVersion\": \"59797\", \"uid\": \"6b63c64a-c50e-444c-b84c-32b8b31db24b\"}, \"spec\": {\"containers\": [{\"command\": [\"sleep\", \"infinity\"], \"image\": \"postgres:13\", \"imagePullPolicy\": \"IfNotPresent\", \"name\": \"awxrestore-2023-12-03-db-management\", \"resources\": {\"limits\": {\"cpu\": \"1\", \"memory\": \"4Gi\"}, \"requests\": {\"cpu\": \"25m\", \"memory\": \"32Mi\"}}, \"terminationMessagePath\": \"/dev/termination-log\", \"terminationMessagePolicy\": \"File\", \"volumeMounts\": [{\"mountPath\": \"/backups\", \"name\": \"awxrestore-2023-12-03-backup\"}, {\"mountPath\": \"/var/run/secrets/kubernetes.io/serviceaccount\", \"name\": \"kube-api-access-5zt5g\", \"readOnly\": true}]}], \"dnsPolicy\": \"ClusterFirst\", \"enableServiceLinks\": true, \"nodeName\": \"vmt86rhel937\", \"preemptionPolicy\": \"PreemptLowerPriority\", \"priority\": 0, \"restartPolicy\": \"Never\", \"schedulerName\": \"default-scheduler\", \"securityContext\": {}, \"serviceAccount\": \"default\", \"serviceAccountName\": \"default\", \"terminationGracePeriodSeconds\": 30, \"tolerations\": [{\"effect\": \"NoExecute\", \"key\": \"node.kubernetes.io/not-ready\", \"operator\": \"Exists\", \"tolerationSeconds\": 300}, {\"effect\": \"NoExecute\", \"key\": \"node.kubernetes.io/unreachable\", \"operator\": \"Exists\", \"tolerationSeconds\": 300}], \"volumes\": [{\"name\": \"awxrestore-2023-12-03-backup\", \"persistentVolumeClaim\": {\"claimName\": \"awx-backup-claim\"}}, {\"name\": \"kube-api-access-5zt5g\", \"projected\": {\"defaultMode\": 420, \"sources\": [{\"serviceAccountToken\": {\"expirationSeconds\": 3607, \"path\": \"token\"}}, {\"configMap\": {\"items\": [{\"key\": \"ca.crt\", \"path\": \"ca.crt\"}], \"name\": \"kube-root-ca.crt\"}}, {\"downwardAPI\": {\"items\": [{\"fieldRef\": {\"apiVersion\": \"v1\", \"fieldPath\": \"metadata.namespace\"}, \"path\": \"namespace\"}]}}]}}]}, \"status\": {\"conditions\": [{\"lastProbeTime\": null, \"lastTransitionTime\": \"2023-12-06T10:27:36Z\", \"status\": \"True\", \"type\": \"Initialized\"}, {\"lastProbeTime\": null, \"lastTransitionTime\": \"2023-12-06T10:27:37Z\", \"status\": \"True\", \"type\": \"Ready\"}, {\"lastProbeTime\": null, \"lastTransitionTime\": \"2023-12-06T10:27:37Z\", \"status\": \"True\", \"type\": \"ContainersReady\"}, {\"lastProbeTime\": null, \"lastTransitionTime\": \"2023-12-06T10:27:36Z\", \"status\": \"True\", \"type\": \"PodScheduled\"}], \"containerStatuses\": [{\"containerID\": \"containerd://3bacb529051b77560c785a67d93cde9aaf4fad2a4eb52af6282a17130179a67e\", \"image\": \"docker.io/library/postgres:13\", \"imageID\": \"docker.io/library/postgres@sha256:9167bfa128429fb9c36df04b6d4c2f2d1751f3db105e00d60b514248f959c5bb\", \"lastState\": {}, \"name\": \"awxrestore-2023-12-03-db-management\", \"ready\": true, \"restartCount\": 0, \"started\": true, \"state\": {\"running\": {\"startedAt\": \"2023-12-06T10:27:37Z\"}}}], \"hostIP\": \"10.245.157.108\", \"phase\": \"Running\", \"podIP\": \"10.42.0.107\", \"podIPs\": [{\"ip\": \"10.42.0.107\"}], \"qosClass\": \"Burstable\", \"startTime\": \"2023-12-06T10:27:36Z\"}}}\n\r\nTASK [restore : Check to make sure backup directory exists on PVC] *\r\ntask path: /opt/ansible/roles/restore/tasks/init.yml:97\nansible.cfg.\nfatal: [localhost]: FAILED! => {\"changed\": true, \"rc\": 1, \"return_code\": 1, \"stderr\": \"stat: cannot statx '/data/backup/tower': No such file or directory\n\", \"stderr_lines\": [\"stat: cannot statx '/data/backup/tower': No such file or directory\"], \"stdout\": \"\", \"stdout_lines\": []}\n\r\nPLAY RECAP *****\r\nlocalhost : ok=11 changed=2 unreachable=0 failed=1 skipped=12 rescued=0 ignored=0 \n","job":"1906987945759275168","name":"awxrestore-2023-12-03","namespace":"awx","error":"exit status 2","stacktrace":"github.com/operator-framework/operator-sdk/internal/ansible/runner.(*runner).Run.func1\n\t/workspace/internal/ansible/runner/runner.go:269"}----- Ansible Task Status Event StdOut (awx.ansible.com/v1beta1, Kind=AWXRestore, awxrestore-2023-12-03/awx) -----
PLAY RECAP ***** localhost : ok=11 changed=2 unreachable=0 failed=1 skipped=12 rescued=0 ignored=0
{"level":"error","ts":"2023-12-06T10:27:43Z","msg":"Reconciler error","controller":"awxrestore-controller","object":{"name":"awxrestore-2023-12-03","namespace":"awx"},"namespace":"awx","name":"awxrestore-2023-12-03","reconcileID":"c19ead42-f6f6-49b3-a0b4-71fdbb2bab0a","error":"event runner on failed","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/controller.(Controller).reconcileHandler\n\t/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.14.5/pkg/internal/controller/controller.go:329\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(Controller).processNextWorkItem\n\t/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.14.5/pkg/internal/controller/controller.go:274\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.14.5/pkg/internal/controller/controller.go:235"}
Files