Open jakecoppinger opened 1 year ago
The last master commit I was on is c93eae35
which seems to run fine.
(git checkout c93eae35 && ansible-playbook -i inventory/hosts setup.yml --tags=setup-all,start
)
Do you know of a command which would fail quick and not affect the server state? Happy to run a git bisect
!
No need to bisect.
Update your Ansible and make sure you not only have ansible-core
(or whatever your distro calls it), but also the full Ansible distribution (including community collections). If you're installing via pip, you need to install the ansible
package.
Upgraded ansible to 2.13.7
:
sudo add-apt-repository --yes --update ppa:ansible/ansible
sudo apt install ansible -y
ansible --version
Ansible collection location is /home/ubuntu/.ansible/collections:/usr/share/ansible/collections.
Pulling latest master and running ansible-playbook -i inventory/hosts setup.yml --tags=setup-all,start
continues until I reach these errors:
TASK [galaxy/com.devture.ansible.role.systemd_service_manager : Fail if service isn't detected to be running] **********************************************************************************************************************************************************************
failed: [matrix.DOMAIN] (item=matrix-mautrix-instagram.service) => changed=false
ansible_loop_var: item
item: matrix-mautrix-instagram.service
msg: matrix-mautrix-instagram.service was not detected to be running. It's possible that there's a configuration problem or another service on your server interferes with it (uses the same ports, etc.). Try running `systemctl status matrix-mautrix-instagram.service` and `jo
urnalctl -fu matrix-mautrix-instagram.service` on the server to investigate. If you're on a slow or overloaded server, it may be that services take a longer time to start and that this error is a false-positive. You can consider raising the value of the `devture_systemd_servi
ce_manager_up_verification_delay_seconds` variable. See `/home/ubuntu/repos/matrix-docker-ansible-deploy/roles/galaxy/com.devture.ansible.role.systemd_service_manager/defaults/main.yml` for more details about that.
failed: [matrix.DOMAIN] (item=matrix-mautrix-signal.service) => changed=false
ansible_loop_var: item
item: matrix-mautrix-signal.service
msg: matrix-mautrix-signal.service was not detected to be running. It's possible that there's a configuration problem or another service on your server interferes with it (uses the same ports, etc.). Try running `systemctl status matrix-mautrix-signal.service` and `journalc
tl -fu matrix-mautrix-signal.service` on the server to investigate. If you're on a slow or overloaded server, it may be that services take a longer time to start and that this error is a false-positive. You can consider raising the value of the `devture_systemd_service_manage
r_up_verification_delay_seconds` variable. See `/home/ubuntu/repos/matrix-docker-ansible-deploy/roles/galaxy/com.devture.ansible.role.systemd_service_manager/defaults/main.yml` for more details about that.
skipping: [matrix.DOMAIN] => (item=matrix-mautrix-signal-daemon.service)
skipping: [matrix.DOMAIN] => (item=matrix-client-element.service)
skipping: [matrix.DOMAIN] => (item=matrix-synapse.service)
skipping: [matrix.DOMAIN] => (item=matrix-coturn.service)
skipping: [matrix.DOMAIN] => (item=matrix-coturn-reload.timer)
skipping: [matrix.DOMAIN] => (item=matrix-grafana.service)
skipping: [matrix.DOMAIN] => (item=matrix-mailer.service)
skipping: [matrix.DOMAIN] => (item=matrix-nginx-proxy.service)
skipping: [matrix.DOMAIN] => (item=matrix-ssl-lets-encrypt-certificates-renew.timer)
skipping: [matrix.DOMAIN] => (item=matrix-ssl-nginx-proxy-reload.timer)
skipping: [matrix.DOMAIN] => (item=matrix-postgres.service)
skipping: [matrix.DOMAIN] => (item=matrix-prometheus.service)
skipping: [matrix.DOMAIN] => (item=matrix-prometheus-node-exporter.service)
failed: [matrix.DOMAIN] (item=matrix-goofys.service) => changed=false
ansible_loop_var: item
item: matrix-goofys.service
msg: matrix-goofys.service was not detected to be running. It's possible that there's a configuration problem or another service on your server interferes with it (uses the same ports, etc.). Try running `systemctl status matrix-goofys.service` and `journalctl -fu matrix-go
ofys.service` on the server to investigate. If you're on a slow or overloaded server, it may be that services take a longer time to start and that this error is a false-positive. You can consider raising the value of the `devture_systemd_service_manager_up_verification_delay_
seconds` variable. See `/home/ubuntu/repos/matrix-docker-ansible-deploy/roles/galaxy/com.devture.ansible.role.systemd_service_manager/defaults/main.yml` for more details about that.
skipping: [matrix.DOMAIN] => (item=matrix-synapse-reverse-proxy-companion.service)
Inspecting with journalctl -fu matrix-synapse.service
synapse is in a restart loop, quitting with code 137.
I think this is an out of memory error but I have plenty of RAM and swap.
Any ideas? I'll try running on another matrix server I'm running to see if this is specific to this instance.
Upgraded ansible to
2.13.7
:sudo add-apt-repository --yes --update ppa:ansible/ansible sudo apt install ansible -y ansible --version
Ansible collection location is /home/ubuntu/.ansible/collections:/usr/share/ansible/collections.
Pulling latest master and running
ansible-playbook -i inventory/hosts setup.yml --tags=setup-all,start
continues until I reach these errors:TASK [galaxy/com.devture.ansible.role.systemd_service_manager : Fail if service isn't detected to be running] ********************************************************************************************************************************************************************** failed: [matrix.DOMAIN] (item=matrix-mautrix-instagram.service) => changed=false ansible_loop_var: item item: matrix-mautrix-instagram.service msg: matrix-mautrix-instagram.service was not detected to be running. It's possible that there's a configuration problem or another service on your server interferes with it (uses the same ports, etc.). Try running `systemctl status matrix-mautrix-instagram.service` and `jo urnalctl -fu matrix-mautrix-instagram.service` on the server to investigate. If you're on a slow or overloaded server, it may be that services take a longer time to start and that this error is a false-positive. You can consider raising the value of the `devture_systemd_servi ce_manager_up_verification_delay_seconds` variable. See `/home/ubuntu/repos/matrix-docker-ansible-deploy/roles/galaxy/com.devture.ansible.role.systemd_service_manager/defaults/main.yml` for more details about that. failed: [matrix.DOMAIN] (item=matrix-mautrix-signal.service) => changed=false ansible_loop_var: item item: matrix-mautrix-signal.service msg: matrix-mautrix-signal.service was not detected to be running. It's possible that there's a configuration problem or another service on your server interferes with it (uses the same ports, etc.). Try running `systemctl status matrix-mautrix-signal.service` and `journalc tl -fu matrix-mautrix-signal.service` on the server to investigate. If you're on a slow or overloaded server, it may be that services take a longer time to start and that this error is a false-positive. You can consider raising the value of the `devture_systemd_service_manage r_up_verification_delay_seconds` variable. See `/home/ubuntu/repos/matrix-docker-ansible-deploy/roles/galaxy/com.devture.ansible.role.systemd_service_manager/defaults/main.yml` for more details about that. skipping: [matrix.DOMAIN] => (item=matrix-mautrix-signal-daemon.service) skipping: [matrix.DOMAIN] => (item=matrix-client-element.service) skipping: [matrix.DOMAIN] => (item=matrix-synapse.service) skipping: [matrix.DOMAIN] => (item=matrix-coturn.service) skipping: [matrix.DOMAIN] => (item=matrix-coturn-reload.timer) skipping: [matrix.DOMAIN] => (item=matrix-grafana.service) skipping: [matrix.DOMAIN] => (item=matrix-mailer.service) skipping: [matrix.DOMAIN] => (item=matrix-nginx-proxy.service) skipping: [matrix.DOMAIN] => (item=matrix-ssl-lets-encrypt-certificates-renew.timer) skipping: [matrix.DOMAIN] => (item=matrix-ssl-nginx-proxy-reload.timer) skipping: [matrix.DOMAIN] => (item=matrix-postgres.service) skipping: [matrix.DOMAIN] => (item=matrix-prometheus.service) skipping: [matrix.DOMAIN] => (item=matrix-prometheus-node-exporter.service) failed: [matrix.DOMAIN] (item=matrix-goofys.service) => changed=false ansible_loop_var: item item: matrix-goofys.service msg: matrix-goofys.service was not detected to be running. It's possible that there's a configuration problem or another service on your server interferes with it (uses the same ports, etc.). Try running `systemctl status matrix-goofys.service` and `journalctl -fu matrix-go ofys.service` on the server to investigate. If you're on a slow or overloaded server, it may be that services take a longer time to start and that this error is a false-positive. You can consider raising the value of the `devture_systemd_service_manager_up_verification_delay_ seconds` variable. See `/home/ubuntu/repos/matrix-docker-ansible-deploy/roles/galaxy/com.devture.ansible.role.systemd_service_manager/defaults/main.yml` for more details about that. skipping: [matrix.DOMAIN] => (item=matrix-synapse-reverse-proxy-companion.service)
Inspecting with
journalctl -fu matrix-synapse.service
synapse is in a restart loop, quitting with code 137.I think this is an out of memory error but I have plenty of RAM and swap.
Any ideas? I'll try running on another matrix server I'm running to see if this is specific to this instance.
Have you figured anything out here? I'm running into the same issue, also upgraded ansible
fully and restarted my machine.
I'm getting the same error, except I can't even upgrade ansible to 2.13. Retrying with pypi.
If anyone is still looking, I got the same problem. Fixed by uninstalling ansible
and ansible-core
and reinstall according to https://docs.ansible.com/ansible/latest/installation_guide/installation_distros.html#installing-ansible-on-ubuntu
This should give you the latest ansible
Running
ansible-playbook -i inventory/hosts setup.yml --tags=setup-all,start
on the latest playbook (master
), I got the errorI assumed the solution was to run
make roles
, which executed successfully.I now get the error:
Is there a bug in the playbook?
Thanks for the amazing work, I've been using this repo nearly flawlessly for a few years now :)
Expected behavior Server is updated and services started.
Matrix Server:
Ansible: If your problem appears to be with Ansible, tell us: