Closed xdkreij closed 7 months ago
This seems to be related to our user/group management package. The obol module should have actually already installed python3-ldap.
Below is the list of packages which should have been installed through ansible. obol_packages:
My first guess is that obol is using a different version of python then the one which was installed through the os.
This seems to be related to our user/group management package. The obol module should have actually already installed python3-ldap.
Below is the list of packages which should have been installed through ansible. obol_packages:
* python3-pip * python3-ldap * openldap-devel
My first guess is that obol is using a different version of python then the one which was installed through the os.
I thought so to, but since the manual installation of python3-ldap breaks due to a missing yum package python39-devel, which ansible does not install...
Looking at different python versions
[ansible@d-vcpu-srv-01 site]$ pip list | grep python
python-ldap 3.4.4 <---------- manually installed
[ansible@d-vcpu-srv-01 site]$ pip3.6 list | grep python
dbus-python 1.2.4
python-dateutil 2.6.1
python-dmidecode 3.12.2
python-ldap 3.3.1
python-linux-procfs 0.7.0
python-qpid-proton 0.37.0
python-ldap seems to be installed at 3.6;
Ansible however looks at
/ansible-playbook --version
ansible-playbook [core 2.15.8]
config file = /opt/trinityx_git/site/ansible.cfg
configured module search path = ['/root/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
ansible python module location = /usr/local/lib/python3.9/site-packages/ansible
ansible collection location = /root/.ansible/collections:/usr/share/ansible/collections
executable location = /usr/local/bin/ansible-playbook
python version = 3.9.7 (default, Sep 13 2021, 08:18:39) [GCC 8.5.0 20210514 (Red Hat 8.5.0-3)] (/usr/bin/python3.9)
jinja version = 3.1.2
libyaml = True
So the obol role looks at the wrong python version i would guess..
side note: if a role is dependent on a different role, it ain't a good role imho :P If sssd requires specific packages, it should not be dependent on a different role to install these packages
Every role should be able to be used as a standalone package.
doing source /trinity/local/python/bin/activate
and then run ansible playbook will trigger the above listed problem as well.
In that case, logout first, login and then re-run the playbook.
source /trinity/local/python/bin/activate
[ansible@d-vcpu-srv-01 site]$ source /trinity/local/python/bin/activate
(python) [ansible@d-vcpu-srv-01 site]$ python --version
Python 3.6.8
(python) [ansible@d-vcpu-srv-01 site]$
(python) [ansible@d-vcpu-srv-01 site]$
(python) [ansible@d-vcpu-srv-01 site]$ pip list
Package Version
----------------------- --------------
pip 21.3.1
setuptools 59.6.0
trinityx-config-manager 2.0.1700240422
wheel 0.37.1
indeed. so please do not do source /trinity/local/python/bin/activate
.
indeed. so please do not do
source /trinity/local/python/bin/activate
.
I am not :-) I run ansible directly outside the venv that has been set up by running controller.yml
can you run it as such?: ansible-playbook controller.yml --tags=openldap,sssd
or even this: ansible-playbook controller.yml --tags=openldap,obol,sssd
or even this:
ansible-playbook controller.yml --tags=openldap,obol,sssd
Sure, but what would you like to see? I've manually installed python39-devel and the pip package, so the run will be successful now.
PLAY RECAP ********************************************************************************************************************************************************************
d-vcpu-srv-01 : ok=53 changed=1 unreachable=0 failed=0 skipped=26 rescued=0 ignored=0
I'll delete both of the packages, but i'm not sure if the devel package will be installed by these roles, and re-run the lot
I've manually installed all packages in the following runs
1st run - without python39-devel, without pip3.9 python-ldap, with pip3.6 python-ldap
TASK [trinity/sssd : Adding access controler groups to the system] ************************************************************************************************************
failed: [d-vcpu-srv-01] (item=admins) => {"ansible_loop_var": "item", "changed": true, "cmd": "/usr/local/sbin/obol group list | grep admins || /usr/local/sbin/obol group add admins", "delta": "0:00:00.096354", "end": "2024-01-04 14:29:51.323975", "item": "admins", "msg": "non-zero return code", "rc": 1, "start": "2024-01-04 14:29:51.227621", "stderr": "Traceback (most recent call last):\n
File \"/usr/local/sbin/obol\", line 42, in <module>\n import ldap\nModuleNotFoundError: No module named 'ldap'
2nd run - with python39-devel, without pip3.9 python-ldap, with pip3.6 python-ldap
import ldap\nModuleNotFoundError: No module named 'ldap'\n
3rd run - with python39-devel, without pip3.9 python-ldap, without pip3.6 python-ldap
import ldap\nModuleNotFoundError: No module named 'ldap'\n
4th run - with python39-devel, with pip3.9 python-ldap, without pip3.6 python-ldap
d-vcpu-srv-01 : ok=53 changed=1 unreachable=0 failed=0 skipped=26 rescued=0 ignored=0
So basically everything breaks unless I install the yum package python39-devel
and pip package python-ldap
manually
The strangest thing...... we assume a clean O/S installation which should (and does) install the whole playbook in one go without issues. What did you install or reconfigure in terms of python on your headnode?
The strangest thing...... we assume a clean O/S installation which should (and does) install the whole playbook in one go without issues. What did you install or reconfigure in terms of python on your headnode?
0 configurations has been done other than what has been stated at the docs on docs.clustervision.com (it's a clean OS install on the cloud using terraform), So... mostly group_vars/all, hosts and network configuration(s)
And all executed on RHEL 8.6; Which resulted in multiple issues of which most of them has been reported here :-)
The strangest thing...... we assume a clean O/S installation which should (and does) install the whole playbook in one go without issues. What did you install or reconfigure in terms of python on your headnode?
Just did a clean install on a new VM with the same OS version (RHEL 8.6), besides some satellite and manual python and ansible setup (as prepare.sh does not work for installing ansible, this issue still applies)
I've applied the partitioning workaround as well, which worked fine so far - other than that, installation seems solid up until this sssd task.
Install python-ldap
[root@d-vcpu-srv-01 site]# pip --version
pip 23.3.2 from /root/.local/lib/python3.9/site-packages/pip (python 3.9)
[root@d-vcpu-srv-01 site]# pip install python-ldap
Collecting python-ldap
Downloading python-ldap-3.4.4.tar.gz (377 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 377.9/377.9 kB 6.6 MB/s eta 0:00:00
Installing build dependencies ... done
Getting requirements to build wheel ... done
Preparing metadata (pyproject.toml) ... done
Collecting pyasn1>=0.3.7 (from python-ldap)
Downloading pyasn1-0.5.1-py2.py3-none-any.whl.metadata (8.6 kB)
Collecting pyasn1-modules>=0.1.5 (from python-ldap)
Downloading pyasn1_modules-0.3.0-py2.py3-none-any.whl (181 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 181.3/181.3 kB 17.3 MB/s eta 0:00:00
Downloading pyasn1-0.5.1-py2.py3-none-any.whl (84 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 84.9/84.9 kB 8.6 MB/s eta 0:00:00
Building wheels for collected packages: python-ldap
Building wheel for python-ldap (pyproject.toml) ... error
error: subprocess-exited-with-error
× Building wheel for python-ldap (pyproject.toml) did not run successfully.
│ exit code: 1
╰─> [83 lines of output]
running bdist_wheel
running build
running build_py
creating build
creating build/lib.linux-x86_64-cpython-39
copying Lib/ldapurl.py -> build/lib.linux-x86_64-cpython-39
copying Lib/ldif.py -> build/lib.linux-x86_64-cpython-39
creating build/lib.linux-x86_64-cpython-39/ldap
copying Lib/ldap/__init__.py -> build/lib.linux-x86_64-cpython-39/ldap
copying Lib/ldap/async.py -> build/lib.linux-x86_64-cpython-39/ldap
copying Lib/ldap/asyncsearch.py -> build/lib.linux-x86_64-cpython-39/ldap
copying Lib/ldap/cidict.py -> build/lib.linux-x86_64-cpython-39/ldap
copying Lib/ldap/compat.py -> build/lib.linux-x86_64-cpython-39/ldap
copying Lib/ldap/constants.py -> build/lib.linux-x86_64-cpython-39/ldap
copying Lib/ldap/dn.py -> build/lib.linux-x86_64-cpython-39/ldap
copying Lib/ldap/filter.py -> build/lib.linux-x86_64-cpython-39/ldap
copying Lib/ldap/functions.py -> build/lib.linux-x86_64-cpython-39/ldap
copying Lib/ldap/ldapobject.py -> build/lib.linux-x86_64-cpython-39/ldap
copying Lib/ldap/logger.py -> build/lib.linux-x86_64-cpython-39/ldap
copying Lib/ldap/modlist.py -> build/lib.linux-x86_64-cpython-39/ldap
copying Lib/ldap/pkginfo.py -> build/lib.linux-x86_64-cpython-39/ldap
copying Lib/ldap/resiter.py -> build/lib.linux-x86_64-cpython-39/ldap
copying Lib/ldap/sasl.py -> build/lib.linux-x86_64-cpython-39/ldap
copying Lib/ldap/syncrepl.py -> build/lib.linux-x86_64-cpython-39/ldap
creating build/lib.linux-x86_64-cpython-39/ldap/controls
copying Lib/ldap/controls/__init__.py -> build/lib.linux-x86_64-cpython-39/ldap/controls
copying Lib/ldap/controls/deref.py -> build/lib.linux-x86_64-cpython-39/ldap/controls
copying Lib/ldap/controls/libldap.py -> build/lib.linux-x86_64-cpython-39/ldap/controls
copying Lib/ldap/controls/openldap.py -> build/lib.linux-x86_64-cpython-39/ldap/controls
copying Lib/ldap/controls/pagedresults.py -> build/lib.linux-x86_64-cpython-39/ldap/controls
copying Lib/ldap/controls/ppolicy.py -> build/lib.linux-x86_64-cpython-39/ldap/controls
copying Lib/ldap/controls/psearch.py -> build/lib.linux-x86_64-cpython-39/ldap/controls
copying Lib/ldap/controls/pwdpolicy.py -> build/lib.linux-x86_64-cpython-39/ldap/controls
copying Lib/ldap/controls/readentry.py -> build/lib.linux-x86_64-cpython-39/ldap/controls
copying Lib/ldap/controls/sessiontrack.py -> build/lib.linux-x86_64-cpython-39/ldap/controls
copying Lib/ldap/controls/simple.py -> build/lib.linux-x86_64-cpython-39/ldap/controls
copying Lib/ldap/controls/sss.py -> build/lib.linux-x86_64-cpython-39/ldap/controls
copying Lib/ldap/controls/vlv.py -> build/lib.linux-x86_64-cpython-39/ldap/controls
creating build/lib.linux-x86_64-cpython-39/ldap/extop
copying Lib/ldap/extop/__init__.py -> build/lib.linux-x86_64-cpython-39/ldap/extop
copying Lib/ldap/extop/dds.py -> build/lib.linux-x86_64-cpython-39/ldap/extop
copying Lib/ldap/extop/passwd.py -> build/lib.linux-x86_64-cpython-39/ldap/extop
creating build/lib.linux-x86_64-cpython-39/ldap/schema
copying Lib/ldap/schema/__init__.py -> build/lib.linux-x86_64-cpython-39/ldap/schema
copying Lib/ldap/schema/models.py -> build/lib.linux-x86_64-cpython-39/ldap/schema
copying Lib/ldap/schema/subentry.py -> build/lib.linux-x86_64-cpython-39/ldap/schema
copying Lib/ldap/schema/tokenizer.py -> build/lib.linux-x86_64-cpython-39/ldap/schema
creating build/lib.linux-x86_64-cpython-39/slapdtest
copying Lib/slapdtest/__init__.py -> build/lib.linux-x86_64-cpython-39/slapdtest
copying Lib/slapdtest/_slapdtest.py -> build/lib.linux-x86_64-cpython-39/slapdtest
running egg_info
writing Lib/python_ldap.egg-info/PKG-INFO
writing dependency_links to Lib/python_ldap.egg-info/dependency_links.txt
writing requirements to Lib/python_ldap.egg-info/requires.txt
writing top-level names to Lib/python_ldap.egg-info/top_level.txt
reading manifest file 'Lib/python_ldap.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
no previously-included directories found matching 'Doc/.build'
adding license file 'LICENCE'
writing manifest file 'Lib/python_ldap.egg-info/SOURCES.txt'
creating build/lib.linux-x86_64-cpython-39/slapdtest/certs
copying Lib/slapdtest/certs/README -> build/lib.linux-x86_64-cpython-39/slapdtest/certs
copying Lib/slapdtest/certs/ca.conf -> build/lib.linux-x86_64-cpython-39/slapdtest/certs
copying Lib/slapdtest/certs/ca.pem -> build/lib.linux-x86_64-cpython-39/slapdtest/certs
copying Lib/slapdtest/certs/client.conf -> build/lib.linux-x86_64-cpython-39/slapdtest/certs
copying Lib/slapdtest/certs/client.key -> build/lib.linux-x86_64-cpython-39/slapdtest/certs
copying Lib/slapdtest/certs/client.pem -> build/lib.linux-x86_64-cpython-39/slapdtest/certs
copying Lib/slapdtest/certs/gencerts.sh -> build/lib.linux-x86_64-cpython-39/slapdtest/certs
copying Lib/slapdtest/certs/gennssdb.sh -> build/lib.linux-x86_64-cpython-39/slapdtest/certs
copying Lib/slapdtest/certs/server.conf -> build/lib.linux-x86_64-cpython-39/slapdtest/certs
copying Lib/slapdtest/certs/server.key -> build/lib.linux-x86_64-cpython-39/slapdtest/certs
copying Lib/slapdtest/certs/server.pem -> build/lib.linux-x86_64-cpython-39/slapdtest/certs
running build_ext
building '_ldap' extension
creating build/temp.linux-x86_64-cpython-39
creating build/temp.linux-x86_64-cpython-39/Modules
gcc -pthread -Wno-unused-result -Wsign-compare -DDYNAMIC_ANNOTATIONS_ENABLED=1 -DNDEBUG -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong -grecord-gcc-switches -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -D_GNU_SOURCE -fPIC -fwrapv -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong -grecord-gcc-switches -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -D_GNU_SOURCE -fPIC -fwrapv -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong -grecord-gcc-switches -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -D_GNU_SOURCE -fPIC -fwrapv -fPIC -DHAVE_SASL -DHAVE_TLS -DLDAPMODULE_VERSION=3.4.4 "-DLDAPMODULE_AUTHOR=python-ldap project" "-DLDAPMODULE_LICENSE=Python style" -IModules -I/usr/include/python3.9 -c Modules/LDAPObject.c -o build/temp.linux-x86_64-cpython-39/Modules/LDAPObject.o
In file included from Modules/LDAPObject.c:3:
Modules/common.h:9:10: fatal error: Python.h: No such file or directory
#include "Python.h"
^~~~~~~~~~
compilation terminated.
error: command '/bin/gcc' failed with exit code 1
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed building wheel for python-ldap
Failed to build python-ldap
ERROR: Could not build wheels for python-ldap, which is required to install pyproject.toml-based projects
install python3-devel
=========================================================================================================================================
Package Architecture Version Repository Size
=========================================================================================================================================
Installing:
python39-devel x86_64 3.9.7-1.module+el8.6.0+12577+62b8d0a1 rhel-8-for-x86_64-appstream-rpms 232 k
Transaction Summary
=========================================================================================================================================
Install 1 Package
trying pip install python-ldap
again
[root@d-vcpu-srv-01 site]# pip install python-ldap
Collecting python-ldap
Using cached python-ldap-3.4.4.tar.gz (377 kB)
Installing build dependencies ... done
Getting requirements to build wheel ... done
Preparing metadata (pyproject.toml) ... done
Collecting pyasn1>=0.3.7 (from python-ldap)
Using cached pyasn1-0.5.1-py2.py3-none-any.whl.metadata (8.6 kB)
Collecting pyasn1-modules>=0.1.5 (from python-ldap)
Using cached pyasn1_modules-0.3.0-py2.py3-none-any.whl (181 kB)
Using cached pyasn1-0.5.1-py2.py3-none-any.whl (84 kB)
Building wheels for collected packages: python-ldap
Building wheel for python-ldap (pyproject.toml) ... done
Created wheel for python-ldap: filename=python_ldap-3.4.4-cp39-cp39-linux_x86_64.whl size=330846 sha256=38dda04c063404056101aff1edbfa78088b9b6bc5c014dfd0f2470db5a93d39f
Stored in directory: /root/.cache/pip/wheels/41/a8/91/0074f21a2da44dbf94955c3f9b72e4713bc74199172d1721c4
Successfully built python-ldap
Installing collected packages: pyasn1, pyasn1-modules, python-ldap
Successfully installed pyasn1-0.5.1 pyasn1-modules-0.3.0 python-ldap-3.4.4
Kicking off controller.yml
ansible-playbook -i hosts controller.yml -v --tags sssd
<snip>
TASK [trinity/sssd : Adding access controler groups to the system] **********************************************************************
changed: [d-vcpu-srv-01<mydomain.nl>] => (item=admins) => {"ansible_loop_var": "item", "changed": true, "cmd": "/usr/local/sbin/obol group list | grep admins || /usr/local/sbin/obol group add admins", "delta": "0:00:00.275013", "end": "2024-01-10 13:49:54.682298", "item": "admins", "msg": "", "rc": 0, "start": "2024-01-10 13:49:54.407285", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []}
TASK [trinity/sssd : Setting up the system to use sssd for authentication] **************************************************************
ok: [d-vcpu-srv-01.<mydomain.nl>] => {"changed": false, "cmd": ["authconfig", "--enablemkhomedir", "--enablesssd", "--enablesssdauth", "--update"], "delta": "0:00:00.980007", "end": "2024-01-10 13:49:55.886621", "msg": "", "rc": 0, "start": "2024-01-10 13:49:54.906614", "stderr": "", "stderr_lines": [], "stdout": "Running authconfig compatibility tool.\nThe purpose of this tool is to enable authentication against chosen services with authselect and minimum configuration. It does not provide all capabilities of authconfig.\n\nIMPORTANT: authconfig is replaced by authselect, please update your scripts.\nSee man authselect-migration(7) to help you with migration to authselect\n\nExecuting: /usr/bin/authselect check\nExecuting: /usr/bin/authselect select sssd with-mkhomedir --force\nExecuting: /usr/bin/systemctl enable sssd.service\nExecuting: /usr/bin/systemctl stop sssd.service\nExecuting: /usr/bin/systemctl start sssd.service\nExecuting: /usr/bin/systemctl enable oddjobd.service\nExecuting: /usr/bin/systemctl stop oddjobd.service\nExecuting: /usr/bin/systemctl start oddjobd.service", "stdout_lines": ["Running authconfig compatibility tool.", "The purpose of this tool is to enable authentication against chosen services with authselect and minimum configuration. It does not provide all capabilities of authconfig.", "", "IMPORTANT: authconfig is replaced by authselect, please update your scripts.", "See man authselect-migration(7) to help you with migration to authselect", "", "Executing: /usr/bin/authselect check", "Executing: /usr/bin/authselect select sssd with-mkhomedir --force", "Executing: /usr/bin/systemctl enable sssd.service", "Executing: /usr/bin/systemctl stop sssd.service", "Executing: /usr/bin/systemctl start sssd.service", "Executing: /usr/bin/systemctl enable oddjobd.service", "Executing: /usr/bin/systemctl stop oddjobd.service", "Executing: /usr/bin/systemctl start oddjobd.service"]}
TASK [trinity/sssd : Configure nsswitch.conf] *******************************************************************************************
skipping: [d-vcpu-srv-01.<mydomain.nl>] => {"changed": false, "false_condition": "ansible_facts['os_family'] == \"Debian\"", "skip_reason": "Conditional result was False"}
TASK [trinity/sssd : Configure pam to use sss] ******************************************************************************************
skipping: [d-vcpu-srv-01.<mydomain.nl>] => {"changed": false, "false_condition": "ansible_facts['os_family'] == \"Debian\"", "skip_reason": "Conditional result was False"}
TASK [trinity/sssd : Configure pam to create homedir on first logon] ********************************************************************
skipping: [d-vcpu-srv-01.<mydomain.nl>] => {"changed": false, "false_condition": "ansible_facts['os_family'] == \"Debian\"", "skip_reason": "Conditional result was False"}
PLAY RECAP ******************************************************************************************************************************
d-vcpu-srv-01.<mydomain.nl> : ok=25 changed=1 unreachable=0 failed=0 skipped=22 rescued=0 ignored=0
I'll give the current release on RHEL 8.6 a go and check for the above.
We have tested against RHEL 8.9 and RHEL 9.3 quite extensively and we were not able to reproduce the problem. If the problem still persists for your release, would it be possible to upgrade to RHEL 8.9 or inform me after pulling the latest TrinityX and try a rerun of the ansible-playbook?
Expected results
Ansible execution error
**Results after installing missing package***
sudo pip install python-ldap
sudo yum install python39-devel
<--- another requirement that wasn't installed by ansiblesudo pip install python-ldap
<--- 2nd attemptDisco
(Yes Mr. Python, a am currently running sudo for all... it's a test :-))
Lets try ansible run again...
sudo /usr/local/bin/ansible-playbook controller.yml -v
And we are good to go