Closed AlanCoding closed 3 years ago
We did get some help with this previously, and to document I'm going to have to quote some people here, hope you don't mind.
<something something about ansible-core packaging, python3.6 vs 3.8>
@shanemcd asked, on what I believe is this issue:
@sivel what does that mean for the dnf module / stuff that requires selinux bindings for things in 2.11?
Then we got a reply
@shanemcd you would likely want to define a localhost ansible_connection=local in inventory to avoid using implicit localhost, and relying on interpreter discovery
Maybe this either isn't the same issue (rpm bindings, not selinux bindings), or there's another gap where the bindings need to be installed for the desired interpreter, because my testing with recent images yields:
$ podman run --rm --tty --interactive quay.io/ansible/awx-ee:latest python3.8 -c "import rpm"
Traceback (most recent call last):
File "<string>", line 1, in <module>
ModuleNotFoundError: No module named 'rpm'
Recap:
rpm
in /usr/lib64/python3.6/site-packages
quay.io/ansible/ansible-runner:devel
we do not have rpm
in either /usr/lib/python3.8/site-packages
or /usr/lib/python3.6/site-packages
quay.io/ansible/awx-ee:latest
for either interpreterThat is the bug.
I tried to track down some candidates that might have provided this through dnf list installed
in both images. The most likely I saw were:
"python3-rpm-macros.noarch",
"python-srpm-macros.noarch",
"python3-rpm-generators.noarch",
"python36-devel.x86_64",
This question looks pretty illustrative of the problem:
https://stackoverflow.com/questions/15412482/installing-rpm-module-for-non-system-python
Advice here is "don't do that". Don't use a non-system python and expect to get the rpm bindings.
@wenottingham how much do you care about compatibility and ongoing support for https://github.com/ansible/awx-facts-playbooks
I think https://github.com/ansible/awx-ee/issues/25#issuecomment-755349051 is going down the wrong road. We don't need any of this in the EE image - the errors from the scan playbook would be on the managed hosts, and that's where the import error is coming from.
What sorts of hosts are you running the playbook against?
This is surfaced as a part of testing Insights - because a host's insights id is discovered through fact gathering. To do this testing, a host is created in AWX API with properties:
{
"name": "registered_host",
"inventory": "<host inventory>",
"variables": "{\"ansible_host\": \"127.0.0.1\", \"ansible_connection\": \"local\"}"
}
(with much from awxkit defaults)
Then a job template is created with properties:
{
"name": "<some name>",
"inventory": "<inventory with that host>",
"project": "<project for https://github.com/ansible/awx-facts-playbooks>",
"playbook": "scan_facts.yml",
"limit": "registered_host",
"use_fact_cache": true
}
The scan_facts.yml playbook here targets hosts: all
. With the JT limit, it targets "registered_host", and runs locally.
In terms of test infrastructure, I don't know that we use an independent managed host these days, much less a managed host that is configured for our pytest-ansible actions to be directed towards. Speaking extremely speculatively, maybe we could replace "ansible_connection": "local"
with "ansible_host": "<server url>"
. We target the server URL for testing the awx.awx inventory plugin (I assume is working but could be wrong), but this is not the same. We need SSH credentials, which are locally present in the test executor somehow, but would have to be moved around in some novel ways. This would take work. There's a reason we never did it before.
Putting those challenges aside, what you're proposing isn't just having this playbook not work on hosts set to local connection - this will make the playbook fail whenever a host in the inventory sets local connection. That doesn't sound reasonable, and I'd think you would prefer to change the host pattern or skip those hosts in scan_facts.yml
.
Avoiding local connection in that playbook may be pretty trivial, but that would still leave the insights testing problem unsolved.
I should mention that @pabelanger proposed having both pythons installed in the EE, and then we can use ansible_python_interpreter
to use the system python (3.6) which can import rpm
, but have everything else work in the installed python (3.8). I have to agree this would work, but it would create a lot of user confusion with EEs, or maybe just this EE in specific.
I mention that because I find it about equally distasteful as the other options I mentioned here.
We could also solve the "get the insights id" by just using https://github.com/ansible/awx/pull/8650 instead.
That can maybe put an insights id into the hostvars, not into the ansible facts for the hosts, which is how that integration works.
My understanding is that the insights id in facts is used to map "whatever" hosts to what insights thinks they are in the playbook. If we just say that for insights remdediations you pull the inventory from insights, it sidesteps that process entirely and should just work. cc @chrismeyersfsu
solved using /usr/libexec/platform-python
but needs docs for some of the things said here
The import here is breaking:
https://github.com/ansible/awx-facts-playbooks/blob/c5b4693061097fea5cf852e7d0792b1a9e75248c/library/scan_packages.py#L57
These are still supported, and will need to work in AWX-EE. They didn't used to fail, but some recent changes here are causing it to fail.
I can confirm that this is present in the AWX images before execution environments. It is a global package:
I have not yet fully determined where this comes from, but the source appears to be part of "rpm" itself. This matches with the content in the AWX image site-packages, plus some
*.so
files.https://github.com/rpm-software-management/rpm/tree/master/python/rpm