Use _AnsibleCollectionFinder for lookup plugin loading

[jr64]

SUMMARY

This pull request fixes a problem in the turbo server where lookup plugins are not loaded correctly if Ansible's collection_finder is not initialized yet.

I encountered this bug when trying to deploy VMs with the vmware.vmware_rest collection which uses cloud.common to speed up operations. See ansible-collections/vmware.vmware_rest#421 for a bug report by another uses who experienced the same issue.

ISSUE TYPE

• Bugfix Pull Request

COMPONENT NAME

• turbo server

ADDITIONAL INFORMATION

Reproducing the bug:

Environment:

ansible --version                                                                                
ansible [core 2.15.2]
  config file = None
  configured module search path = ['/home/jr/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /home/jr/.cache/pypoetry/virtualenvs/cloud-common-bug-repro-C0FdZm2j-py3.10/lib/python3.10/site-packages/ansible
  ansible collection location = /home/jr/.ansible/collections:/usr/share/ansible/collections
  executable location = /home/jr/.cache/pypoetry/virtualenvs/cloud-common-bug-repro-C0FdZm2j-py3.10/bin/ansible
  python version = 3.10.6 (main, May 29 2023, 11:10:38) [GCC 11.3.0] (/home/jr/.cache/pypoetry/virtualenvs/cloud-common-bug-repro-C0FdZm2j-py3.10/bin/python)
  jinja version = 3.1.2
  libyaml = True

Create a test collection with the following minimal lookup plugin:

__metaclass__ = type

DOCUMENTATION = r"""
name: testlookup
short_description: test
description:
    - test
author:
    - test
"""

from ansible_collections.cloud.common.plugins.plugin_utils.turbo.lookup import TurboLookupBase

class LookupModule(TurboLookupBase):
    async def _run(self, terms, variables, **kwargs):
        return ["Testresult"]

    run = TurboLookupBase.run_on_daemon

Test playbook:

---
- hosts: localhost
  tasks:
    - debug:
        msg: "{{ lookup('mytest.testcollection.testlookup') }}"

When running, this will fail with the following error message:

ansible-playbook test-playbook.yml       
[WARNING]: No inventory was parsed, only implicit localhost is available
[WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all'

PLAY [localhost] ****************************************************************************************************************************************************************************************************************************************************************

TASK [Gathering Facts] **********************************************************************************************************************************************************************************************************************************************************
ok: [localhost]

TASK [debug] ********************************************************************************************************************************************************************************************************************************************************************
fatal: [localhost]: FAILED! => {"msg": "An unhandled exception occurred while running the lookup plugin 'mytest.testcollection.testlookup'. Error was a <class 'ansible_collections.cloud.common.plugins.module_utils.turbo.exceptions.EmbeddedModuleUnexpectedFailure'>, original message: No _run() found. No _run() found"}

PLAY RECAP ******************************************************************************************************************

Root cause

The problem is in the function run_as_lookup_plugin in server.py:

https://github.com/ansible-collections/cloud.common/blob/6036c79da7d824918b3872db49ce09adb02ba945/plugins/module_utils/turbo/server.py#L248C1-L250C10

instance will be None if the _AnsibleCollectionFinder is not initialized. I'm not an expert on Ansible internals but from what I have gathered, calling _install() will install hooks for importlib.import_module to be able to load Python code from folders that do not contain a __init__.py.

See: https://github.com/ansible/ansible/blob/a5ccc0124f4677eb55a90a1e2e53b6984b2b140d/lib/ansible/utils/collection_loader/_collection_finder.py#L294C1-L300C57

If AnsibleCollectionFinder hooks are enabled, import_module will return the following object (note that the path ends in __synthetic__ instead of __init__.py):

<module 'ansible_collections.mytest.testcollection.plugins.lookup' from '/home/jr/.ansible/collections/ansible_collections/mytest/testcollection/plugins/lookup/__synthetic__'>

Without the hooks, import_module will return a namespace instead because normally, Python 3 treats folders without __init__.py as namespaces.

To understand exactly what happens, we have to dig into the Ansible plugin_loader:

https://github.com/ansible/ansible/blob/a5ccc0124f4677eb55a90a1e2e53b6984b2b140d/lib/ansible/plugins/loader.py#L854C1-L865C99

plugin_loader.lookup_loader.get() calls get_with_context which in turn calls find_plugin_with_context which calls _resolve_plugin_step which calls _find_fq_plugin https://github.com/ansible/ansible/blob/a5ccc0124f4677eb55a90a1e2e53b6984b2b140d/lib/ansible/plugins/loader.py#L535C1-L543C49

Here, sys.modules.get is called which returns a namespace if the hooks are not installed. The path is then set to pkg.__file__ which is None for a namespace, causing the loading of the lookup plugin to fail.

[softwarefactory-project-zuul[bot]]

Build failed. https://ansible.softwarefactory-project.io/zuul/buildset/59c9cd83fd3b4645b153962cf41083e1

:x: ansible-test-cloud-integration-vmware-rest NODE_FAILURE Node request 200-0006230140 failed in 0s :heavy_check_mark: build-ansible-collection SUCCESS in 10m 49s :heavy_check_mark: ansible-test-splitter SUCCESS in 4m 45s :heavy_check_mark: integration-kubernetes.core-with-turbo-1 SUCCESS in 19m 00s :heavy_check_mark: integration-kubernetes.core-with-turbo-2 SUCCESS in 27m 19s :heavy_check_mark: integration-kubernetes.core-with-turbo-3 SUCCESS in 35m 29s :heavy_check_mark: ansible-galaxy-importer SUCCESS in 5m 08s

[gravesm]

@jr64 Could you try https://github.com/ansible-collections/cloud.common/pull/130 to see if that fixes the problem you're having?

[jr64]

@gravesm I can confirm #130 also addresses my issue. Thank you, I'm closing this pull request.