wait_for_connection to localhost explicitly specified in the inventory and ansible_python_interpreter set to ansible_playbook_python fails with 'timed out waiting for ping module test: 'int' object has no attribute 'template'"'

Reproduction steps:

$ docker run --rm -it rockylinux:8.9.20231119 bash
$ yum install -y python38
$ pip3 install ansible==6.7.0
$ pip3 install mitogen=0.3.7
$ ANSIBLE_STRATEGY_PLUGINS=/usr/local/lib/python3.8/site-packages/ansible_mitogen/plugins/strategy ANSIBLE_STRATEGY=mitogen_linear ansible -i /tmp/hosts.yml localhost -m ansible.builtin.wait_for_connection -vvv

where

# $ cat /tmp/hosts.yml 
all:
        hosts:
                localhost:
                        ansible_connection: local
                        ansible_python_interpreter: "{{ ansible_playbook_python }}"

Result:

localhost | FAILED! => {
    "changed": false,
    "elapsed": 600,
    "msg": "timed out waiting for ping module test: 'int' object has no attribute 'template'"
}

Suprisingly, with implicit inventory it works fine:

$ ANSIBLE_STRATEGY_PLUGINS=/usr/local/lib/python3.8/site-packages/ansible_mitogen/plugins/strategy ANSIBLE_STRATEGY=mitogen_linear ansible localhost -m ansible.builtin.wait_for_connection
[WARNING]: No inventory was parsed, only implicit localhost is available
localhost | SUCCESS => {
    "changed": false,
    "elapsed": 0
}

Which version of Ansible are you running?

$ ansible --version
ansible [core 2.13.13]
(...)
python version = 3.8.17 (default, Aug 10 2023, 18:03:39) [GCC 8.5.0 20210514 (Red Hat 8.5.0-20)]
jinja version = 3.1.3
libyaml = True

Is your version of Ansible patched in any way? No
Are you running with any custom modules, or module_utils loaded? No
Have you tried the latest master version from Git? Yes
Do you have some idea of what the underlying problem may be? https://mitogen.networkgenomics.com/ansible_detailed.html#common-problems has instructions to help figure out the likely cause and how to gather relevant logs. No
Mention your host and target OS and versions Rocky 8.8
Mention your host and target Python versions Python 3.8
If reporting a performance issue, mention the number of targets and a rough description of your workload (lots of copies, lots of tiny file edits, etc.)
If reporting a crash or hang in Ansible, please rerun with -vvv and include 200 lines of output around the point of the error, along with a full copy of any traceback or error text in the log. Beware "-vvv" may include secret data! Edit as necessary before posting.
If reporting any kind of problem with Ansible, please include the Ansible version along with output of "ansible-config dump --only-changed". Empty

EDIT: 2024-10-07 @moreati Syntax highlighting

Now I've done a lot of digging around in ansible_mitogen/transport_config.py and ansible_mitogen/connection.py I can probably diagnose this. Prime suspect is

https://github.com/mitogen-hq/mitogen/blob/8e64459bbfbdc6f89cc4f275cb82142bea7bf776/ansible_mitogen/connection.py#L944

and https://github.com/mitogen-hq/mitogen/commit/833e2845e9d25fac94b5aab6fd0b613a3bd97007 probably contains the building blocks to fix it.

Still occurs with master,

➜  mitogen git:(master) ✗ ANSIBLE_STRATEGY=mitogen_linear ansible -i issue1079_inv.yml localhost -m ansible.builtin.wait_for_connection -a "timeout=10"     
localhost | FAILED! => {
    "changed": false,
    "elapsed": 10,
    "msg": "timed out waiting for ping module test: 'int' object has no attribute 'template'"
}

same -vvv

```console ➜ mitogen git:(master) ✗ ANSIBLE_STRATEGY=mitogen_linear ansible -i issue1079_inv.yml localhost -m ansible.builtin.wait_for_connection -a "timeout=10" -vvv ansible [core 2.17.5] config file = /Users/alex/src/mitogen/ansible.cfg configured module search path = ['/Users/alex/src/mitogen/tests/ansible/lib/modules'] ansible python module location = /Users/alex/.local/share/uv/tools/ansible-core/lib/python3.13/site-packages/ansible ansible collection location = /Users/alex/.ansible/collections:/usr/share/ansible/collections executable location = /Users/alex/.local/bin/ansible python version = 3.13.0 (main, Oct 16 2024, 08:05:40) [Clang 18.1.8 ] (/Users/alex/.local/share/uv/tools/ansible-core/bin/python) jinja version = 3.1.4 libyaml = True Using /Users/alex/src/mitogen/ansible.cfg as config file host_list declined parsing /Users/alex/src/mitogen/issue1079_inv.yml as it did not pass its verify_file() method script declined parsing /Users/alex/src/mitogen/issue1079_inv.yml as it did not pass its verify_file() method Parsed /Users/alex/src/mitogen/issue1079_inv.yml inventory source with yaml plugin Skipping callback 'default', as we already have a stdout callback. Skipping callback 'minimal', as we already have a stdout callback. Skipping callback 'oneline', as we already have a stdout callback. [top 88106] 16:20:03.219759 D ansible_mitogen.process: inherited open file limits: soft=1048575 hard=(infinity) [top 88106] 16:20:03.219842 D ansible_mitogen.process: max open files already set to hard limit: 524288 [mux 88108] 16:20:03.224877 D mitogen.service: Pool(81a0, size=32, th='MainThread'): initialized [mux 88108] 16:20:03.225766 D ansible_mitogen.process: Service pool configured: size=32 wait_for_connection: attempting ping module test ... wait_for_connection: attempting ping module test [task 88109] 16:20:13.271395 D ansible_mitogen.mixins: _remove_tmp_path(None) [task 88109] 16:20:13.273145 D ansible_mitogen.mixins: _remove_tmp_path(None) localhost | FAILED! => { "changed": false, "elapsed": 10, "msg": "timed out waiting for ping module test: 'int' object has no attribute 'template'" } [mux 88108] 16:20:13.280671 D mitogen: Waker(fd=36/38): disconnecting [mux 88108] 16:20:13.282092 D mitogen.service: thread mitogen.Pool.81a0.0 exiting gracefully [mux 88108] 16:20:13.283423 D mitogen.service: thread mitogen.Pool.81a0.1 exiting gracefully [mux 88108] 16:20:13.284447 D mitogen.service: thread mitogen.Pool.81a0.5 exiting gracefully [mux 88108] 16:20:13.285470 D mitogen: Router(Broker(ee40)): stats: 0 module requests in 0 ms, 0 sent (0 ms minify time), 0 negative responses. Sent 0.0 kb total, 0.0 kb avg. [mux 88108] 16:20:13.286389 D mitogen.service: thread mitogen.Pool.81a0.6 exiting gracefully ... [mux 88108] 16:20:13.304953 D mitogen.service: thread mitogen.Pool.81a0.30 exiting gracefully [mux 88108] 16:20:13.305462 D mitogen.service: thread mitogen.Pool.81a0.31 exiting gracefully [mux 88108] 16:20:13.306406 D mitogen.service: FileService().on_shutdown() [top 88106] 16:20:13.308218 D ansible_mitogen.process: multiplexer 0 PID 88108 exited with return code 0 ```

doesn't occur with vanilla Ansible

➜  mitogen git:(master) ✗ ANSIBLE_STRATEGY=linear ansible -i issue1079_inv.yml localhost -m ansible.builtin.wait_for_connection -a "timeout=10"     
[WARNING]: Reset is not implemented for this connection
localhost | SUCCESS => {
    "changed": false,
    "elapsed": 0
}

all:
  hosts:
    localhost:
      ansible_connection: local
      ansible_python_interpreter: "{{ ansible_playbook_python }}"

Untemplated ansible_interpreter_python makes the problem go away

➜  mitogen git:(master) ✗ ANSIBLE_STRATEGY=mitogen_linear ansible -i issue1079_inv.yml localhost -m ansible.builtin.wait_for_connection -a "timeout=20"                               
localhost | SUCCESS => {
    "changed": false,
    "elapsed": 0
}

all:
  hosts:
    localhost:
      ansible_connection: local
      ansible_python_interpreter: python3

I need to ponder this some more. The templar made available in '_mitogen.smuggled.reset_connection' is only available during a meta: reset_connection task.

Note to self: There's already some special handling for wait_for_connection

➜  mitogen git:(issue1079-templated-python-interpreter) ✗ ag wait_for_connection   
ansible_mitogen/mixins.py
367:        # wait_for_connection, the `ping` test from Ansible won't pass because we lost connection
369:        # see https://github.com/dw/mitogen/issues/655 and Ansible's `wait_for_connection` module for more info
370:        if module_name == 'ansible.legacy.ping' and type(self).__name__ == 'wait_for_connection':

ansible_mitogen/services.py
162:        # and then a `wait_for_connection` right afterwards

ansible_mitogen/connection.py
865:        Ansible 2.4 wait_for_connection plug-in.
984:    # Compatibility with Ansible 2.4 wait_for_connection plug-in.

tests/ansible/integration/context_service/reconnection.yml
24:    - wait_for_connection:

tests/ansible/regression/issue_655__wait_for_connection_error.yml
2:# Spins up a Centos8 container and runs the wait_for_connection test inside of it
7:- name: regression/issue_655__wait_for_connection_error.yml
85:              wait_for_connection:

tests/ansible/regression/all.yml
15:- import_playbook: issue_655__wait_for_connection_error.yml

docs/changelog.rst
242:* :gh:issue:`655` wait_for_connection gives errors

mitogen-hq / mitogen

wait_for_connection to localhost explicitly specified in the inventory and ansible_python_interpreter set to ansible_playbook_python fails with 'timed out waiting for ping module test: 'int' object has no attribute 'template'"' #1079