Closed drelephant closed 2 years ago
That Ansible version is probably too old still. Try upgrading to a newer one or running Ansible in a container (see docs/ansible.md
)
Hmm, I had to use python3.7
, which was the most recent on my system.
python3.7 -m pip install --user ansible
:
Collecting ansible
Using cached ansible-4.10.0.tar.gz (36.8 MB)
Preparing metadata (setup.py) ... done
Collecting ansible-core~=2.11.7
Using cached ansible-core-2.11.12.tar.gz (7.1 MB)
Preparing metadata (setup.py) ... done
Requirement already satisfied: PyYAML in /usr/lib/python3/dist-packages (from ansible-core~=2.11.7->ansible) (3.12)
Requirement already satisfied: cryptography in /usr/lib/python3/dist-packages (from ansible-core~=2.11.7->ansible) (2.1.4)
Collecting jinja2
Downloading Jinja2-3.1.2-py3-none-any.whl (133 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 133.1/133.1 kB 5.0 MB/s eta 0:00:00
Collecting packaging
Downloading packaging-21.3-py3-none-any.whl (40 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 40.8/40.8 kB 10.5 MB/s eta 0:00:00
Collecting resolvelib<0.6.0,>=0.5.3
Downloading resolvelib-0.5.4-py2.py3-none-any.whl (12 kB)
Collecting MarkupSafe>=2.0
Downloading MarkupSafe-2.1.1-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (25 kB)
Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in /usr/lib/python3/dist-packages (from packaging->ansible-core~=2.11.7->ansible) (2.2.0)
Building wheels for collected packages: ansible, ansible-core
Building wheel for ansible (setup.py) ... done
Created wheel for ansible: filename=ansible-4.10.0-py3-none-any.whl size=60559848 sha256=4eeb2a58922ee93f3bca748681fa7be5a7b27b7e6fdc38756168396d60bee19e
Stored in directory: /home/redacted/.cache/pip/wheels/0d/c0/47/09bce237cdfaaa8428538fabcd18a8b6da351ad852c8b6a872
Building wheel for ansible-core (setup.py) ... done
Created wheel for ansible-core: filename=ansible_core-2.11.12-py3-none-any.whl size=1952066 sha256=0fa4051cf79697a8d0199f8ebe33673084e3db405146a00e7c7c77df8f302568
Stored in directory: /home/redacted/.cache/pip/wheels/9d/56/09/d8b4ccae450bb6703b67c4ea03bbc1c0120416631fbf8d8b60
Successfully built ansible ansible-core
Installing collected packages: resolvelib, packaging, MarkupSafe, jinja2, ansible-core, ansible
Successfully installed MarkupSafe-2.1.1 ansible-4.10.0 ansible-core-2.11.12 jinja2-3.1.2 packaging-21.3 resolvelib-0.5.4
So it looked like it ran correctly, but then:
ansible --version
:
ansible 2.9.27
config file = /etc/ansible/ansible.cfg
configured module search path = [u'/home/redacted/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
ansible python module location = /usr/lib/python2.7/dist-packages/ansible
executable location = /usr/bin/ansible
python version = 2.7.17 (default, Jul 1 2022, 15:56:32) [GCC 7.5.0]
How do I specify what python to use with Ansible? I have no idea what I'm doing.
Take a look at your inventory/hosts
file, which is based on the examples/hosts
example file.
I changed the end of the line in inventory/hosts
to ansible_python_interpreter=/usr/bin/python3.7
, but it still comes up with the same error when I run ansible-playbook --ask-pass -i inventory/hosts setup.yml --tags=setup-all,ensure-matrix-users-created,start
:
ERROR! couldn't resolve module/action 'community.docker.docker_network'. This often indicates a misspelling, missing collection, or incorrect module path.
The error appears to be in '/home/redacted/matrix-docker-ansible-deploy/roles/matrix-base/tasks/setup_matrix_base.yml': line 22, column 3, but may
be elsewhere in the file depending on the exact syntax problem.
The offending line appears to be:
- name: Ensure Matrix network is created in Docker
^ here
I'm really keen to try out matrix on my own server and escape the monopolies...
Try a newer version of Ansible or see if you need to install some ansible-collections
package on your Linux system separately. Sometimes Ansible is split into ansible-core
and ansible-collections
.
Sorry to be a pain, but how do I do that?
I looked here and did python3.7 -m pip install --upgrade --user ansible
(python3 gave some error about upgrading pip but python3.7 seemed to work?
Requirement already satisfied: ansible in ./.local/lib/python3.7/site-packages (4.10.0)
Requirement already satisfied: ansible-core~=2.11.7 in ./.local/lib/python3.7/site-packages (from ansible) (2.11.12)
Requirement already satisfied: packaging in ./.local/lib/python3.7/site-packages (from ansible-core~=2.11.7->ansible) (21.3)
Requirement already satisfied: resolvelib<0.6.0,>=0.5.3 in ./.local/lib/python3.7/site-packages (from ansible-core~=2.11.7->ansible) (0.5.4)
Requirement already satisfied: jinja2 in ./.local/lib/python3.7/site-packages (from ansible-core~=2.11.7->ansible) (3.1.2)
Requirement already satisfied: PyYAML in /usr/lib/python3/dist-packages (from ansible-core~=2.11.7->ansible) (3.12)
Requirement already satisfied: cryptography in /usr/lib/python3/dist-packages (from ansible-core~=2.11.7->ansible) (2.1.4)
Requirement already satisfied: MarkupSafe>=2.0 in ./.local/lib/python3.7/site-packages (from jinja2->ansible-core~=2.11.7->ansible) (2.1.1)
Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in /usr/lib/python3/dist-packages (from packaging->ansible-core~=2.11.7->ansible) (2.2.0)
It's really hard to navigate getting the mystical correct versions of python, pip and ansible. I thought this was supposed to work from Ubuntu 18.04 and up...
I did ansible-galaxy collection install community.docker
and that seemed to get me to the next error - after running ansible-playbook -i inventory/hosts setup.yml --tags=setup-all,ensure-matrix-users-created,start --ask-pass
again:
SSH password:
ERROR! couldn't resolve module/action 'community.general.pacman'. This often indicates a misspelling, missing collection, or incorrect module path.
The error appears to be in '/home/redacted/matrix-docker-ansible-deploy/roles/matrix-base/tasks/util/ensure_openssl_installed.yml': line 18, column 3, but may
be elsewhere in the file depending on the exact syntax problem.
The offending line appears to be:
- name: Ensure openssl installed (Archlinux)
^ here
, but unfortunately this one I couldn't fix with ansible-galaxy collection install community.general
:
Starting galaxy collection install process
Nothing to do. All requested collections are already installed. If you want to reinstall them, consider using `--force`.
But I still get the same error when trying to run the ansible playbook.
Hmm, maybe I have already upgraded ansible? ansible --version
now shows:
[DEPRECATION WARNING]: Ansible will require Python 3.8 or newer on the controller starting with Ansible 2.12. Current
version: 3.7.15 (default, Oct 12 2022, 19:14:55) [GCC 7.5.0]. This feature will be removed from ansible-core in
version 2.12. Deprecation warnings can be disabled by setting deprecation_warnings=False in ansible.cfg.
ansible [core 2.11.12]
config file = /home/redacted/matrix-docker-ansible-deploy/ansible.cfg
configured module search path = ['/home/redacted/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
ansible python module location = /home/redacted/.local/lib/python3.7/site-packages/ansible
ansible collection location = /home/redacted/.ansible/collections:/usr/share/ansible/collections
executable location = /home/redacted/.local/bin/ansible
python version = 3.7.15 (default, Oct 12 2022, 19:14:55) [GCC 7.5.0]
jinja version = 3.1.2
libyaml = False
ansible-galaxy collection list
:
# /home/redacted/.local/lib/python3.7/site-packages/ansible_collections
Collection Version
----------------------------- -------
community.general 3.8.3
<snip>
# /home/redacted/.ansible/collections/ansible_collections
Collection Version
----------------- -------
community.docker 3.2.0
community.general 5.8.0
Looks like I've got an old version of community.general there? How can I remove it?
edit: I deleted the folder /home/redacted/.local/lib/python3.7/site-packages/ansible_collections/community/general
, now the old version doesn't show up in the list, but I'm still getting the same error when I try to run ansible-playbook. Now I'm really out of ideas!!
Sorry, I really don't have a lot of idea how this all fits together. It's just googling and throwing sh*t at the wall to see what sticks.
Why would it be failing on Archlinux-specific tasks? Are you testing on Archlinux now?
If you can't get get the playbook (and all collection dependencies) running on your distro, you can always run Ansible in a container - see docs/ansible.md
for instructions.
Why would it be failing on Archlinux-specific tasks? Are you testing on Archlinux now?
No idea, you're the expert! I'm still on Ubuntu.
I created a folder, "test_ansible", then created a file, "hosts":
[mymachine]
localhost
and a file, "playbook.yaml":
- name: echo stuff
hosts: mymachine
tasks:
- name: Print ansible_distribution
ansible.builtin.debug:
msg: ansible_distribution == {{ ansible_distribution }}
- name: Print ansible_os_family
ansible.builtin.debug:
msg: ansible_os_family == {{ ansible_os_family }}
then ran ansible-playbook -i hosts playbook.yaml --ask-pass
:
[DEPRECATION WARNING]: Ansible will require Python 3.8 or newer on the controller starting with Ansible 2.12. Current
version: 3.7.15 (default, Oct 12 2022, 19:14:55) [GCC 7.5.0]. This feature will be removed from ansible-core in
version 2.12. Deprecation warnings can be disabled by setting deprecation_warnings=False in ansible.cfg.
SSH password:
[WARNING]: Skipping plugin (/home/redacted/.local/lib/python3.7/site-packages/ansible/plugins/connection/winrm.py) as it
seems to be invalid: invalid syntax (spawnbase.py, line 224)
PLAY [echo stuff] ****************************************************************************************************
TASK [Gathering Facts] ***********************************************************************************************
[DEPRECATION WARNING]: Distribution Ubuntu 18.04 on host localhost should use /usr/bin/python3, but is using
/usr/bin/python for backward compatibility with prior Ansible releases. A future Ansible release will default to
using the discovered platform python for this host. See https://docs.ansible.com/ansible-
core/2.11/reference_appendices/interpreter_discovery.html for more information. This feature will be removed in
version 2.12. Deprecation warnings can be disabled by setting deprecation_warnings=False in ansible.cfg.
ok: [localhost]
TASK [Print ansible_distribution] ************************************************************************************
ok: [localhost] => {
"msg": "ansible_distribution == Ubuntu"
}
TASK [Print ansible_os_family] ***************************************************************************************
ok: [localhost] => {
"msg": "ansible_os_family == Debian"
}
PLAY RECAP ***********************************************************************************************************
localhost : ok=3 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
So I don't understand why it's going to the Archlinux in ensure_openssl_installed.yml
?
I had the suspicion that maybe Ansible fails when it can't find a module (like community.general.pacman
), even if said module is not really necessary (because the when
statement means it's not going to get used).
I replicated it with the following task:
- something.something.missing:
msg: Hello
when: false | bool == true | bool
Despite the when
condition saying this task wouldn't run, Ansible still complains about the something.something.missing
module not being available.
Actually, you can't trick it this way too:
- ansible.builtin.import_tasks: "{{ role_path }}/tasks/missing.yml"
when: false | bool == true | bool
with missing.yml
containing:
---
- something.something.missing:
msg: Hello
However, changing from import_tasks
to include_tasks
helps though.
-- ansible.builtin.import_tasks: "{{ role_path }}/tasks/missing.yml"
+- ansible.builtin.include_tasks: "{{ role_path }}/tasks/missing.yml"
when: false | bool == true | bool
I've changed openssl
and fuse
installation to use this include_tasks
trick in c1c152f7acb93102, which hopefully allows you to go further without having to install another Ansible collection for things you don't need (such as community.general.pacman
, etc.)
Thanks @spantaleev.
Now on to the next error... :-)
ansible-playbook -i inventory/hosts setup.yml --tags=setup-all,ensure-matrix-users-created,start --ask-pass
SSH password:
ERROR! the role 'galaxy/com.devture.ansible.role.playbook_help' was not found in /home/redacted/matrix-docker-ansible-deploy/roles:/home/redacted/.ansible/roles:/usr/share/ansible/roles:/etc/ansible/roles:/home/redacted/matrix-docker-ansible-deploy
The error appears to be in '/home/redacted/matrix-docker-ansible-deploy/setup.yml': line 8, column 7, but may
be elsewhere in the file depending on the exact syntax problem.
The offending line appears to be:
# Most of the roles below are not distributed with the playbook, but downloaded separately using `ansible-galaxy` via the `make roles` command (see `Makefile`).
- role: galaxy/com.devture.ansible.role.playbook_help
^ here
Have you read this error message? Or the changelog file? Have you executed make roles
?
OK, sorry, like I said I have hardly any idea how ansible fits together. I ran ansible-galaxy install -r requirements.yml -p roles/galaxy/ --force
that I found in the Makefile, which worked...
now... ansible-playbook -i inventory/hosts setup.yml --tags=setup-all,ensure-matrix-users-created,start --ask-pass
, I'm back at the same Archlinux error:
SSH password:
ERROR! couldn't resolve module/action 'community.general.pacman'. This often indicates a misspelling, missing collection, or incorrect module path.
The error appears to be in '/home/redacted/matrix-docker-ansible-deploy/roles/custom/matrix-synapse/tasks/ext/synapse-simple-antispam/setup_install.yml': line 24, column 3, but may
be elsewhere in the file depending on the exact syntax problem.
The offending line appears to be:
- name: Ensure git installed (Archlinux)
^ here
Edit: hang on, I think I haven't got all your changes from that commit. I'm pretty crap with git too... Trying to work out how to pull that commit.
I think I need you to do your magic on roles/custom/matrix-synapse/tasks/ext/synapse-simple-antispam/setup_install.yml
as well:
- name: Ensure git installed (Archlinux)
community.general.pacman:
name:
- git
state: present
update_cache: false
when: "ansible_distribution == 'Archlinux'"
5ef70015cbaaa should fix this particular problem as well.
Were you enabling this antispam module, by the way? matrix_synapse_ext_spam_checker_synapse_simple_antispam_enabled: true
?
Were you enabling this antispam module, by the way?
I don't think so. All I want is Synapse and Element.
I literally went through the installation doc doing everything it said. I didn't enable any extra modules or anything.
I had to make the same changes you have to these files because it kept coming up with those pacman
errors in all of them, basically doing the same thing you did and making tasks/util/ensure_<package>_installed_<distro>.yml
files:
modified: roles/custom/matrix-ma1sd/tasks/setup_install.yml
modified: roles/custom/matrix-synapse/tasks/ext/mjolnir-antispam/setup_install.yml
modified: roles/custom/matrix-synapse/tasks/ext/synapse-simple-antispam/setup_install.yml
Also I had to do ansible-galaxy collection install ansible.posix
.
After doing that, I eventually got to a new error, but now I feel I'm close...
ansible-playbook -i inventory/hosts setup.yml --tags=setup-all,ensure-matrix-users-created,start --ask-pass
SSH password:
PLAY [Set up a Matrix server] ****************************************************************************************************************************************************************************************************************
TASK [Gathering Facts] ***********************************************************************************************************************************************************************************************************************
fatal: [matrix.redacted.org]: FAILED! => changed=false
ansible_facts: {}
failed_modules:
setup:
failed: true
module_stderr: |-
sudo: a password is required
module_stdout: ''
msg: |-
MODULE FAILURE
See stdout/stderr for the exact error
rc: 1
msg: |-
The following modules failed to execute: setup
PLAY RECAP ***********************************************************************************************************************************************************************************************************************************
matrix.redacted.org : ok=0 changed=0 unreachable=0 failed=1 skipped=0 rescued=0 ignored=0
I don't know why it's complaining about a password now? I entered the password. It's ubuntu with no root password, I have to use sudo to escalate.
How do I view the stdout/stderr to see the full error?
When I'm up to date with master, I get:
ansible-playbook -i inventory/hosts setup.yml --tags=setup-all,ensure-matrix-users-created,start --ask-pass
SSH password:
ERROR! couldn't resolve module/action 'community.general.pacman'. This often indicates a misspelling, missing collection, or incorrect module path.
The error appears to be in '/home/redacted/spant_matrix-docker-ansible-deploy/roles/custom/matrix-ma1sd/tasks/setup_install.yml': line 77, column 7, but may
be elsewhere in the file depending on the exact syntax problem.
The offending line appears to be:
- name: Ensure gradle is installed for self-building (Archlinux)
^ here
But I would still really like to know about the password error above if you know, I feel like I'm very close!!
I'm running into the exact same issue, looking at it now.
couldn't resolve module/action 'community.general.pacman'
I just removed this from everywhere and now it's running, thx for your comments!
OK, I worked the password thing out, getting closer!!
Needed to add --ask-become-pass
to the command:
ansible-playbook -i inventory/hosts setup.yml --tags=setup-all,ensure-matrix-users-created,start --ask-become-pass --ask-pass
...and it almost worked:
SSH password:
BECOME password[defaults to SSH password]:
PLAY [Set up a Matrix server] ***********************************************************************************************************************************************************************************************************************************************************************
TASK [Gathering Facts] ******************************************************************************************************************************************************************************************************************************************************************************ok: [matrix.redacted.org]
TASK [custom/matrix_playbook_migration : (Deprecation) Catch and report renamed Matrix playbook settings] *******************************************************************************************************************************************************************************************
skipping: [matrix.redacted.org] => (item={u'new': u'devture_playbook_state_preserver_vars_preservation_enabled', u'old': u'matrix_vars_yml_snapshotting_enabled'})
skipping: [matrix.redacted.org] => (item={u'new': u'devture_playbook_state_preserver_vars_preservation_src', u'old': u'matrix_vars_yml_snapshotting_src'})
skipping: [matrix.redacted.org] => (item={u'new': u'devture_playbook_state_preserver_commit_hash_preservation_enabled', u'old': u'matrix_playbook_commit_hash_preservation_enabled'})
skipping: [matrix.redacted.org] => (item={u'new': u'devture_timesync_ntpd_package', u'old': u'matrix_ntpd_package'})
skipping: [matrix.redacted.org] => (item={u'new': u'devture_timesync_ntpd_service', u'old': u'matrix_ntpd_service'})
skipping: [matrix.redacted.org] => (item={u'new': u'devture_systemd_docker_base_systemd_unit_home_path', u'old': u'matrix_systemd_unit_home_path'})
skipping: [matrix.redacted.org] => (item={u'new': u'devture_systemd_docker_base_systemd_path', u'old': u'matrix_systemd_path'})
skipping: [matrix.redacted.org] => (item={u'new': u'devture_systemd_docker_base_host_command_docker', u'old': u'matrix_host_command_docker'})
skipping: [matrix.redacted.org] => (item={u'new': u'devture_systemd_docker_base_host_command_sh', u'old': u'matrix_host_command_sh'})
skipping: [matrix.redacted.org] => (item={u'new': u'devture_systemd_docker_base_host_command_systemctl', u'old': u'matrix_host_command_systemctl'})
skipping: [matrix.redacted.org] => (item={u'new': u'devture_playbook_help_container_retries_count', u'old': u'matrix_container_retries_count'})
skipping: [matrix.redacted.org] => (item={u'new': u'devture_playbook_help_container_retries_delay', u'old': u'matrix_container_retries_delay'})
skipping: [matrix.redacted.org] => (item={u'new': u'devture_playbook_help_geturl_retries_count', u'old': u'matrix_geturl_retries_count'})
skipping: [matrix.redacted.org] => (item={u'new': u'devture_playbook_help_geturl_retries_delay', u'old': u'matrix_geturl_retries_delay'})
TASK [galaxy/com.devture.ansible.role.timesync : ansible.builtin.include_tasks] *********************************************************************************************************************************************************************************************************************
skipping: [matrix.redacted.org]
TASK [galaxy/com.devture.ansible.role.timesync : ansible.builtin.include_tasks] *********************************************************************************************************************************************************************************************************************
included: /home/mouse/matrix-docker-ansible-deploy/roles/galaxy/com.devture.ansible.role.timesync/tasks/setup_debian.yml for matrix.redacted.org
TASK [galaxy/com.devture.ansible.role.timesync : Ensure APT packages are installed] *****************************************************************************************************************************************************************************************************************
[WARNING]: Updating cache and auto-installing missing dependency: python3-apt
fatal: [matrix.redacted.org]: FAILED! => changed=false
msg: 'Could not import python modules: apt, apt_pkg. Please install python3-apt package.'
PLAY RECAP ******************************************************************************************************************************************************************************************************************************************************************************************
matrix.redacted.org : ok=2 changed=0 unreachable=0 failed=1 skipped=2 rescued=0 ignored=0
Anyone know how I can get through this (hopefully the last) obstacle?
The timesync
role tries to install ntp
/ ntpd
.
This failure to install python3-apt
is the apt
module itself complaining I suppose.
This may be helpful: https://askubuntu.com/questions/480908/problem-with-update-manager-no-module-named-apt-pkg-in-ubuntu-13-10-having-i
I'm starting to think that moving away from this old Ubuntu would be more than ideal though.
I got it working!!!
I don't know exactly what fixed it, but these are the commands that I ran:
sudo apt-get update
sudo apt-get install --reinstall python3-apt
sudo python3.7 -m pip install apt
python3.7 -m pip install --user apt
python3.7 -m pip install --upgrade --user ansible
python3.7 -m pip install --upgrade --user apt_inst
python3.7 -m pip install --user apt_inst
env ANSIBLE_DEBUG=1 ansible-playbook -vvv -i inventory/hosts setup.yml --tags=setup-all,ensure-matrix-users-created,start --ask-pass --ask-become-pass
I tried to run ansible-playbook
without the -vvv later, and it didn't work, so ¯\(ツ)/¯
matrix.redacted.org : ok=282 changed=30 unreachable=0 failed=0 skipped=2051 rescued=0 ignored=0
Anyway I'm very happy!! Thanks for all the help!
I'm completely new to Ansible... Be gentle!
Playbook Configuration:
My
vars.yml
file looks like this (minus commented lines):Matrix Server:
Ansible:
I run Ansible on the Matrix server itself
I tried to update Ansible because of a previous error that I didn't understand by following info found here,
sudo add-apt-repository ppa:ansible/ansible
- that installed correctly.ansible --version
:I do have python3 installed, not sure why it's showing 2.7.17 there.
apt-cache policy python3
:grep -v ^# inventory/host_vars/matrix.redacted.org/vars.yml
:grep -v ^# inventory/hosts
:Problem description:
When I run
ansible-playbook -i inventory/hosts setup.yml --tags=setup-all,ensure-matrix-users-created,start
, I get the following:I also tried
ansible-playbook --ask-pass -i inventory/hosts setup.yml --tags=setup-all,ensure-matrix-users-created,start
, with the same result.I'm trying to install Synapse/Element. This is the first time I've ever used Ansible, and I have no idea how to proceed.
Any help much appreciated!