search

ansible-collections / community.crypto

The community.crypto collection for Ansible.
https://galaxy.ansible.com/ui/repo/published/community/crypto/
Other
94 stars 86 forks source link

cryptography python module not found when using community.crypto #762

Open kwikmr2 opened 1 month ago

kwikmr2 commented 1 month ago
SUMMARY

Built AWX EE from https://github.com/Frewx/awx-ee-builder.git. Defined "quay.io/ansible/awx-ee:latest" in execution-environment.yml for base_image. Included "cryptography" in requirements.txt. Included "community.general" (latest) and "community.crypto" (latest) in requirements.yml.

When executing a simple task via AWX, the following error occurs: "Cannot detect any of the required Python libraries cryptography (>= 1.6)"

ISSUE TYPE
COMPONENT NAME

community.crypto.x509_certificate_info

ANSIBLE VERSION
ansible [core 2.15.12]
  config file = None
  configured module search path = [‘/runner/.ansible/plugins/modules’, ‘/usr/share/ansible/plugins/modules’]
  ansible python module location = /usr/local/lib/python3.9/site-packages/ansible
  ansible collection location = /runner/.ansible/collections:/usr/share/ansible/collections
  executable location = /usr/local/bin/ansible
  python version = 3.9.18 (main, Jan 24 2024, 00:00:00) [GCC 11.4.1 20231218 (Red Hat 11.4.1-3)] (/usr/bin/python3)
  jinja version = 3.1.4
  libyaml = True
COLLECTION VERSION
# /usr/local/lib/python3.9/site-packages/ansible_collections
Collection       Version
---------------- -------
community.crypto 2.16.1
# /usr/share/ansible/collections/ansible_collections
Collection       Version
---------------- -------
community.crypto 2.20.0
CONFIGURATION
CONFIG_FILE() = None
OS / ENVIRONMENT

quay.io/ansible/awx-ee:latest based on CentOS Stream release 9

STEPS TO REPRODUCE

Deploy EE based on the settings posted above. Setup EE in AWX (24.3.1) and execute simple playbook against target host.

---
  - name: Get information on generated certificate
    community.crypto.x509_certificate_info:
      path: /data/path/test-ca.crt
    register: result

  - name: Dump certificate information
    ansible.builtin.debug:
      var: result
EXPECTED RESULTS

That the information about a certificate would return

ACTUAL RESULTS
TASK [Get information on generated certificate] ********************************
task path: /runner/project/tasks/ssl-main.yml:2
<192.168.57.104> ESTABLISH SSH CONNECTION FOR USER: ansible_svc
<192.168.57.104> SSH: EXEC sshpass -d12 ssh -C -o ControlMaster=auto -o ControlPersist=60s -o StrictHostKeyChecking=no -o 'User="ansible_svc"' -o ConnectTimeout=10 -o 'ControlPath="/runner/cp/30094d2be6"' 192.168.57.104 '/bin/sh -c '"'"'echo ~ansible_svc && sleep 0'"'"''
<192.168.57.104> (0, b'/home/ansible_svc\\n', b"Warning: Permanently added '192.168.57.104' (ECDSA) to the list of known hosts.\\r\\n<redacted>.\\n")
<192.168.57.104> ESTABLISH SSH CONNECTION FOR USER: ansible_svc
<192.168.57.104> SSH: EXEC sshpass -d12 ssh -C -o ControlMaster=auto -o ControlPersist=60s -o StrictHostKeyChecking=no -o 'User="ansible_svc"' -o ConnectTimeout=10 -o 'ControlPath="/runner/cp/30094d2be6"' 192.168.57.104 '/bin/sh -c '"'"'( umask 77 && mkdir -p "` echo /home/ansible_svc/.ansible/tmp `"&& mkdir "` echo /home/ansible_svc/.ansible/tmp/ansible-tmp-1717007346.3548055-27-128022922135843 `" && echo ansible-tmp-1717007346.3548055-27-128022922135843="` echo /home/ansible_svc/.ansible/tmp/ansible-tmp-1717007346.3548055-27-128022922135843 `" ) && sleep 0'"'"''
<192.168.57.104> (0, b'ansible-tmp-1717007346.3548055-27-128022922135843=/home/ansible_svc/.ansible/tmp/ansible-tmp-1717007346.3548055-27-128022922135843\\n', b'')
<labymrepo01> Attempting python interpreter discovery
<192.168.57.104> ESTABLISH SSH CONNECTION FOR USER: ansible_svc
<192.168.57.104> SSH: EXEC sshpass -d12 ssh -C -o ControlMaster=auto -o ControlPersist=60s -o StrictHostKeyChecking=no -o 'User="ansible_svc"' -o ConnectTimeout=10 -o 'ControlPath="/runner/cp/30094d2be6"' 192.168.57.104 '/bin/sh -c '"'"'echo PLATFORM; uname; echo FOUND; command -v '"'"'"'"'"'"'"'"'python3.12'"'"'"'"'"'"'"'"'; command -v '"'"'"'"'"'"'"'"'python3.11'"'"'"'"'"'"'"'"'; command -v '"'"'"'"'"'"'"'"'python3.10'"'"'"'"'"'"'"'"'; command -v '"'"'"'"'"'"'"'"'python3.9'"'"'"'"'"'"'"'"'; command -v '"'"'"'"'"'"'"'"'python3.8'"'"'"'"'"'"'"'"'; command -v '"'"'"'"'"'"'"'"'python3.7'"'"'"'"'"'"'"'"'; command -v '"'"'"'"'"'"'"'"'python3.6'"'"'"'"'"'"'"'"'; command -v '"'"'"'"'"'"'"'"'/usr/bin/python3'"'"'"'"'"'"'"'"'; command -v '"'"'"'"'"'"'"'"'/usr/libexec/platform-python'"'"'"'"'"'"'"'"'; command -v '"'"'"'"'"'"'"'"'python2.7'"'"'"'"'"'"'"'"'; command -v '"'"'"'"'"'"'"'"'/usr/bin/python'"'"'"'"'"'"'"'"'; command -v '"'"'"'"'"'"'"'"'python'"'"'"'"'"'"'"'"'; echo ENDFOUND && sleep 0'"'"''
<192.168.57.104> (0, b'PLATFORM\\nLinux\\nFOUND\\n/usr/libexec/platform-python\\n/usr/bin/python2.7\\n/usr/bin/python\\n/usr/bin/python\\nENDFOUND\\n', b'')
<192.168.57.104> ESTABLISH SSH CONNECTION FOR USER: ansible_svc
<192.168.57.104> SSH: EXEC sshpass -d12 ssh -C -o ControlMaster=auto -o ControlPersist=60s -o StrictHostKeyChecking=no -o 'User="ansible_svc"' -o ConnectTimeout=10 -o 'ControlPath="/runner/cp/30094d2be6"' 192.168.57.104 '/bin/sh -c '"'"'/usr/libexec/platform-python && sleep 0'"'"''
<192.168.57.104> (0, b'{"osrelease_content": "NAME=\\\\"CentOS Linux\\\\"\\\\nVERSION=\\\\"7 (Core)\\\\"\\\\nID=\\\\"centos\\\\"\\\\nID_LIKE=\\\\"rhel fedora\\\\"\\\\nVERSION_ID=\\\\"7\\\\"\\\\nPRETTY_NAME=\\\\"CentOS Linux 7 (Core)\\\\"\\\\nANSI_COLOR=\\\\"0;31\\\\"\\\\nCPE_NAME=\\\\"cpe:/o:centos:centos:7\\\\"\\\\nHOME_URL=\\\\"https://www.centos.org/\\\\"\\\\nBUG_REPORT_URL=\\\\"https://bugs.centos.org/\\\\"\\\\n\\\\nCENTOS_MANTISBT_PROJECT=\\\\"CentOS-7\\\\"\\\\nCENTOS_MANTISBT_PROJECT_VERSION=\\\\"7\\\\"\\\\nREDHAT_SUPPORT_PRODUCT=\\\\"centos\\\\"\\\\nREDHAT_SUPPORT_PRODUCT_VERSION=\\\\"7\\\\"\\\\n\\\\n", "platform_dist_result": ["centos", "7.9.2009", "Core"]}\\n', b'')
Using module file /usr/share/ansible/collections/ansible_collections/community/crypto/plugins/modules/x509_certificate_info.py
<192.168.57.104> PUT /runner/.ansible/tmp/ansible-local-22yu_u7i7f/tmp5su6kh3e TO /home/ansible_svc/.ansible/tmp/ansible-tmp-1717007346.3548055-27-128022922135843/AnsiballZ_x509_certificate_info.py
<192.168.57.104> SSH: EXEC sshpass -d12 sftp -o BatchMode=no -b - -C -o ControlMaster=auto -o ControlPersist=60s -o StrictHostKeyChecking=no -o 'User="ansible_svc"' -o ConnectTimeout=10 -o 'ControlPath="/runner/cp/30094d2be6"' '[192.168.57.104]'
<192.168.57.104> (0, b'sftp> put /runner/.ansible/tmp/ansible-local-22yu_u7i7f/tmp5su6kh3e /home/ansible_svc/.ansible/tmp/ansible-tmp-1717007346.3548055-27-128022922135843/AnsiballZ_x509_certificate_info.py\\n', b'')
<192.168.57.104> ESTABLISH SSH CONNECTION FOR USER: ansible_svc
<192.168.57.104> SSH: EXEC sshpass -d12 ssh -C -o ControlMaster=auto -o ControlPersist=60s -o StrictHostKeyChecking=no -o 'User="ansible_svc"' -o ConnectTimeout=10 -o 'ControlPath="/runner/cp/30094d2be6"' 192.168.57.104 '/bin/sh -c '"'"'chmod u+x /home/ansible_svc/.ansible/tmp/ansible-tmp-1717007346.3548055-27-128022922135843/ /home/ansible_svc/.ansible/tmp/ansible-tmp-1717007346.3548055-27-128022922135843/AnsiballZ_x509_certificate_info.py && sleep 0'"'"''
<192.168.57.104> (0, b'', b'')
<192.168.57.104> ESTABLISH SSH CONNECTION FOR USER: ansible_svc
<192.168.57.104> SSH: EXEC sshpass -d12 ssh -C -o ControlMaster=auto -o ControlPersist=60s -o StrictHostKeyChecking=no -o 'User="ansible_svc"' -o ConnectTimeout=10 -o 'ControlPath="/runner/cp/30094d2be6"' -tt 192.168.57.104 '/bin/sh -c '"'"'sudo -H -S -p "[sudo via ansible, key=etjodrossjnhiiejjjipkloplcelzydr] password:" -u root /bin/sh -c '"'"'"'"'"'"'"'"'echo BECOME-SUCCESS-etjodrossjnhiiejjjipkloplcelzydr ; /usr/bin/python /home/ansible_svc/.ansible/tmp/ansible-tmp-1717007346.3548055-27-128022922135843/AnsiballZ_x509_certificate_info.py'"'"'"'"'"'"'"'"' && sleep 0'"'"''
Escalation succeeded
<192.168.57.104> (1, b'\\r\\n{"msg": "Cannot detect any of the required Python libraries cryptography (>= 1.6)", "failed": true, "invocation": {"module_args": {"content": null, "select_crypto_backend": "auto", "name_encoding": "ignore", "valid_at": null, "path": "/data/path/test-ca.crt"}}}\\r\\n', b'Shared connection to 192.168.57.104 closed.\\r\\n')
<192.168.57.104> Failed to connect to the host via ssh: Shared connection to 192.168.57.104 closed.
<192.168.57.104> ESTABLISH SSH CONNECTION FOR USER: ansible_svc
<192.168.57.104> SSH: EXEC sshpass -d12 ssh -C -o ControlMaster=auto -o ControlPersist=60s -o StrictHostKeyChecking=no -o 'User="ansible_svc"' -o ConnectTimeout=10 -o 'ControlPath="/runner/cp/30094d2be6"' 192.168.57.104 '/bin/sh -c '"'"'rm -f -r /home/ansible_svc/.ansible/tmp/ansible-tmp-1717007346.3548055-27-128022922135843/ > /dev/null 2>&1 && sleep 0'"'"''
<192.168.57.104> (0, b'', b'')
fatal: [labymrepo01]: FAILED! => {
    "ansible_facts": {
        "discovered_interpreter_python": "/usr/bin/python"
    },
    "changed": false,
    "invocation": {
        "module_args": {
            "content": null,
            "name_encoding": "ignore",
            "path": "/data/path/test-ca.crt",
            "select_crypto_backend": "auto",
            "valid_at": null
        }
    },
    "msg": "Cannot detect any of the required Python libraries cryptography (>= 1.6)"
}
felixfontein commented 1 month ago

You need to install the cryptography dependency for the Python where the module is executed. If you execute the module on a target node, you have to make sure it's also installed there. Ansible, AWX, the collection, requirements.txt, ... won't do that automatically for you.

kwikmr2 commented 1 month ago

I just noticed in the debug output that Python2.7 is being used: "04> (0, b'PLATFORM\nLinux\nFOUND\n/usr/libexec/platform-python\n/usr/bin/python2.7\n/usr/bin/python\n/usr/bin/python\nENDFOUND\n', b'')"

How is this possible when the EE is CentOS Stream 9 with Python3.9 installed? The whole purpose of the EE is to execute within that environment.

felixfontein commented 1 month ago

If you ask Ansible to run the task on a remote target, then it won't be run inside the EE, but on the remote target.

If you want to run the module in the EE, you have to use hosts: localhost or delegate_to: localhost.

kwikmr2 commented 1 month ago

I made the adjustments and now it fails because the path to the certificate is not on the localhost (the EE)...this is seems to be a paradox.

---
  - name: Get information on generated certificate
    community.crypto.x509_certificate_info:
      path: /data/path/test-ca.crt
    register: result
    delegate_to: localhost

Error:

The full traceback is:
  File "/tmp/ansible_community.crypto.x509_certificate_info_payload_lcdbcu61/ansible_community.crypto.x509_certificate_info_payload.zip/ansible_collections/community/crypto/plugins/modules/x509_certificate_info.py", line 444, in main
fatal: [labymrepo01 -> localhost]: FAILED! => {
    "changed": false,
    "invocation": {
        "module_args": {
            "content": null,
            "name_encoding": "ignore",
            "path": "/data/path/test-ca.crt",
            "select_crypto_backend": "auto",
            "valid_at": null
        }
    },
    "msg": "Error while reading certificate file from disk: [Errno 2] No such file or directory: '/data/path/test-ca.crt'"
}
felixfontein commented 1 month ago

Well, if you want to operate on files on the remote, you either first have to fetch them to the controller, or you have to run the module on the target - but then you have to make sure that the module's requirements are available.

kwikmr2 commented 1 month ago

Okay, then assuming it is not feasible to install the cryptography python module on every single remote host in the inventory AND using delegate_to: localhost breaks since the certificate to be inspected is not on the EE...that would make this in a way only usable with CLI Ansible and not AWX/Tower.

felixfontein commented 1 month ago

I don't see what the difference between CLI Ansible and AWX/Tower is. In both cases, installing on the controller is easier than installing on all remotes, and you can determine yourself whether to run on the controller or on the targets.