ValueError: this loader cannot find files for ansible.plugins, only ansible_collections.ansible.builtin.plugins

[gtirloni]

ansible-core 2.13.0 ansible 6.0.0b1 ansible-compat 2.1.0 mitogen from Git master branch ( 89c0cc94d16218e2647bb8bb32b011231def0fd7)

Controller OS: Fedora 36, Python 3.10 Host OS: CentOS 7, Python 3.6

Works without mitogen strategy. Error happens with Python 2 and 3 in the host (set through ansible_python_interpreter)

[mux  49590] 18:58:01.502846 D mitogen.responder: sending ansible.module_utils (0.50 KiB) to ssh.server
[mux  49590] 18:58:01.502918 D mitogen.io: Router(Broker(1900))._async_route(Message(2, 0, 0, 107, None, b'\x80\x02(X\x14\x00\x00\x00ansible.module_utilsq\x00]q\x01(X\x05\x00\x00\x00_textq\x02X\x0f\x00\x00'..515), None)
[mux  49590] 18:58:01.502990 D mitogen.io: MitogenProtocol(ssh.server)._send(Message(2, 0, 0, 107, None, b'\x80\x02(X\x14\x00\x00\x00ansible.module_utilsq\x00]q\x01(X\x05\x00\x00\x00_textq\x02X\x0f\x00\x00'..515))
[mux  49590] 18:58:01.549921 D mitogen: PkgutilMethod(): <_frozen_importlib_external.SourceFileLoader object at 0x7fb2db7937c0>.get_file_name('ansible.module_utils.distro') failed: ImportError('loader for distro cannot handle ansible.module_utils.distro')
[mux  49590] 18:58:01.550039 D mitogen: SysModulesMethod(): <module 'distro' from '/home/user/tmp/ansible-2.13/lib/python3.10/site-packages/distro.py'>.__name__ is 'distro', aborting
[mux  49590] 18:58:01.550287 D mitogen: ParentEnumerationMethod(): 'ansible.module_utils.distro' is PKG_DIRECTORY: '/home/user/tmp/ansible-2.13/lib/python3.10/site-packages/ansible/module_utils/distro/__init__.py'
[mux  49590] 18:58:01.585593 D mitogen: While importing 'ansible.module_utils.basic'
Traceback (most recent call last):
  File "/home/user/code/mitogen-master/mitogen/master.py", line 1134, in _send_module_and_related
    tup = self._build_tuple(fullname)
  File "/home/user/code/mitogen-master/mitogen/master.py", line 1087, in _build_tuple
    for name in self._finder.find_related(fullname)
  File "/home/user/code/mitogen-master/mitogen/master.py", line 959, in find_related
    names = self.find_related_imports(name)
  File "/home/user/code/mitogen-master/mitogen/master.py", line 910, in find_related_imports
    modpath, src, _ = self.get_module_source(fullname)
  File "/home/user/code/mitogen-master/mitogen/master.py", line 858, in get_module_source
    tup = method.find(fullname)
  File "/home/user/code/mitogen-master/mitogen/master.py", line 538, in find
    path = loader.get_filename(fullname)
  File "/home/user/tmp/ansible-2.13/lib/python3.10/site-packages/ansible/utils/collection_loader/_collection_finder.py", line 516, in get_filename
    raise ValueError('this loader cannot find files for {0}, only {1}'.format(fullname, self._fullname))
ValueError: this loader cannot find files for ansible.plugins, only ansible_collections.ansible.builtin.plugins
[mux  49590] 18:58:01.586228 D mitogen.io: Waker(fd=6/8).defer() [immediate]

Which leads to this error:

The full traceback is:
Traceback (most recent call last):
  File "/home/user/tmp/ansible-2.13/lib/python3.10/site-packages/ansible/executor/task_executor.py", line 157, in run
    res = self._execute()
  File "/home/user/tmp/ansible-2.13/lib/python3.10/site-packages/ansible/executor/task_executor.py", line 630, in _execute
    result = self._handler.run(task_vars=vars_copy)
  File "/home/user/code/mitogen-master/ansible_mitogen/mixins.py", line 146, in run
    return super(ActionModuleMixin, self).run(tmp, task_vars)
  File "/home/user/tmp/ansible-2.13/lib/python3.10/site-packages/ansible/plugins/action/gather_facts.py", line 100, in run
    res = self._execute_module(module_name=fact_module, module_args=mod_args, task_vars=task_vars, wrap_async=False)
  File "/home/user/code/mitogen-master/ansible_mitogen/mixins.py", line 376, in _execute_module
    self._set_temp_file_args(module_args, wrap_async)
  File "/home/user/code/mitogen-master/ansible_mitogen/mixins.py", line 355, in _set_temp_file_args
    self._connection.get_good_temp_dir()
  File "/home/user/code/mitogen-master/ansible_mitogen/connection.py", line 832, in get_good_temp_dir
    self._connect()
  File "/home/user/code/mitogen-master/ansible_mitogen/connection.py", line 854, in _connect
    self._connect_stack(stack)
  File "/home/user/code/mitogen-master/ansible_mitogen/connection.py", line 801, in _connect_stack
    dct = mitogen.service.call(
  File "/home/user/code/mitogen-master/mitogen/service.py", line 126, in call
    return call_context.call_service(service_name, method_name, **kwargs)
  File "/home/user/code/mitogen-master/mitogen/core.py", line 2314, in call_service
    return recv.get().unpickle()
  File "/home/user/code/mitogen-master/mitogen/core.py", line 977, in unpickle
    raise obj
mitogen.core.CallError: builtins.ModuleNotFoundError: The Mitogen master process was unable to serve 'ansible_mitogen.target'. It may be a native Python extension, or it may be missing entirely. Check the importer debug logs on the master for more information.
  File "<stdin>", line 3698, in _dispatch_one
  File "<stdin>", line 3685, in _parse_request
  File "<stdin>", line 682, in import_module
  File "<stdin>", line 1521, in load_module

fatal: [server]: FAILED! => {}

MSG:

Unexpected failure during module execution.

[gtirloni]

This will likely be fixed by https://github.com/mitogen-hq/mitogen/pull/933 (since I see similar errors in the CI logs).

[moreati]

Yiou shouldn't have been able to even attempt it with ansible-core 2.13, using that revision. It should have been prevented by this code https://github.com/mitogen-hq/mitogen/blob/89c0cc94d16218e2647bb8bb32b011231def0fd7/ansible_mitogen/loaders.py#L50-L83.

Did you make any modifications to you copy of mitogen?

[gtirloni]

Did you make any modifications to you copy of mitogen?

Yes

[moreati]

duplicate of #933