search

mitogen-hq / mitogen

Distributed self-replicating programs in Python
https://mitogen.networkgenomics.com/
BSD 3-Clause "New" or "Revised" License
2.35k stars 199 forks source link

ansible 10 with mitogen 0.3.9: SyntaxError: future feature annotations is not defined and interpreter discovery warning #1105

Open ifalatiksetlog opened 3 months ago

ifalatiksetlog commented 3 months ago

After upgrading to ansible v10.3.0 and mitogen 0.3.9 I now receive the following error before the first role of my playbook is executed:

ERROR! [mux  62115] 13:32:36.287729 E mitogen.[ssh.<ansible_host>]: while importing u'ansible.module_utils.json_utils'
Traceback (most recent call last):
  File "<stdin>", line 1710, in load_module
SyntaxError: future feature annotations is not defined (json_utils.py, line 27)

I also receive this warning, before the first task is executed:

[WARNING]: Platform linux on host <inventory_hostname> is using the discovered Python interpreter at /usr/bin/python3.9, but future installation of another Python interpreter could change the meaning of that path. See https://docs.ansible.com/ansible-
core/2.17/reference_appendices/interpreter_discovery.html for more information.

These didn't occur previously, don't occur without mitogen strategies and don't occur when setting ansible_python_interpreter=/usr/bin/python3.9 explicitely.

Host information

Target information

Verbose ansible:

(... -vvv 2>&1 | grep 'future feature annotations' -C50)

[mux  64290] 13:41:20.732448 D mitogen.importer.[ssh.<ansible_host>]: received ansible.utils.vars
[mux  64290] 13:41:20.732463 D mitogen.importer.[ssh.<ansible_host>]: received ansible.vars
[mux  64290] 13:41:20.732478 D mitogen.importer.[ssh.<ansible_host>]: received ansible.vars.clean
[mux  64290] 13:41:20.732493 D mitogen.importer.[ssh.<ansible_host>]: received ansible.vars.fact_cache
[mux  64290] 13:41:20.732507 D mitogen.importer.[ssh.<ansible_host>]: received ansible.vars.hostvars
[mux  64290] 13:41:20.732522 D mitogen.importer.[ssh.<ansible_host>]: received ansible.vars.manager
[mux  64290] 13:41:20.732537 D mitogen.importer.[ssh.<ansible_host>]: received ansible.vars.plugins
[mux  64290] 13:41:20.732552 D mitogen.importer.[ssh.<ansible_host>]: received ansible.vars.reserved
[mux  64290] 13:41:20.732586 D mitogen.importer.[ssh.<ansible_host>]: received ansible.module_utils.basic
[mux  64290] 13:41:20.732606 D mitogen.importer.[ssh.<ansible_host>]: received ansible.module_utils.json_utils
[mux  64290] 13:41:20.732622 D mitogen.importer.[ssh.<ansible_host>]: received ansible_mitogen
[mux  64290] 13:41:20.732637 D mitogen.importer.[ssh.<ansible_host>]: received ansible_mitogen.target
[mux  64290] 13:41:20.732652 D mitogen.importer.[ssh.<ansible_host>]: received mitogen.parent
[mux  64290] 13:41:20.732668 D mitogen.importer.[ssh.<ansible_host>]: received mitogen.select
[mux  64290] 13:41:20.736710 D mitogen.importer.[ssh.<ansible_host>]: received mitogen.service
[mux  64290] 13:41:20.736824 D mitogen.importer.[ssh.<ansible_host>]: received ansible_mitogen.runner
[mux  64290] 13:41:20.738165 D mitogen.importer.[ssh.<ansible_host>]: received mitogen.fork
[mux  64290] 13:41:20.753090 D mitogen.[ssh.<ansible_host>]: Dispatcher: dispatching (None, u'ansible_mitogen.target', None, u'init_child', (), Kwargs({u'candidate_temp_dirs': [u'~/.ansible/tmp', u'/var/tmp', u'/tmp'], u'log_level': 10}))
[mux  64290] 13:41:20.757734 D mitogen.importer.[ssh.<ansible_host>]: requesting ansible_mitogen
[mux  64290] 13:41:20.758436 D mitogen.importer.[ssh.<ansible_host>]: requesting ansible_mitogen.target
[mux  64290] 13:41:20.758517 D mitogen.importer.[ssh.<ansible_host>]: json.decoder is submodule of a locally loaded package
[mux  64290] 13:41:20.758550 D mitogen.importer.[ssh.<ansible_host>]: json.re is submodule of a locally loaded package
[mux  64290] 13:41:20.758574 D mitogen.importer.[ssh.<ansible_host>]: json.sys is submodule of a locally loaded package
[mux  64290] 13:41:20.759542 D mitogen.importer.[ssh.<ansible_host>]: json.struct is submodule of a locally loaded package
[mux  64290] 13:41:20.759604 D mitogen.importer.[ssh.<ansible_host>]: json.json is submodule of a locally loaded package
[mux  64290] 13:41:20.761796 D mitogen.importer.[ssh.<ansible_host>]: json.scanner is submodule of a locally loaded package
[mux  64290] 13:41:20.761864 D mitogen.importer.[ssh.<ansible_host>]: json._json is submodule of a locally loaded package
[mux  64290] 13:41:20.766810 D mitogen.importer.[ssh.<ansible_host>]: json.encoder is submodule of a locally loaded package
[mux  64290] 13:41:20.779616 D mitogen.importer.[ssh.<ansible_host>]: requesting mitogen.parent
[mux  64290] 13:41:20.807945 D mitogen.importer.[ssh.<ansible_host>]: mitogen has no submodule binascii
[mux  64290] 13:41:20.808105 D mitogen.importer.[ssh.<ansible_host>]: mitogen has no submodule errno
[mux  64290] 13:41:20.808154 D mitogen.importer.[ssh.<ansible_host>]: mitogen has no submodule fcntl
[mux  64290] 13:41:20.808193 D mitogen.importer.[ssh.<ansible_host>]: mitogen has no submodule getpass
[mux  64290] 13:41:20.808211 D mitogen.importer.[ssh.<ansible_host>]: mitogen has no submodule heapq
[mux  64290] 13:41:20.808228 D mitogen.importer.[ssh.<ansible_host>]: mitogen has no submodule inspect
[mux  64290] 13:41:20.827589 D mitogen.importer.[ssh.<ansible_host>]: mitogen has no submodule logging
[mux  64290] 13:41:20.827743 D mitogen.importer.[ssh.<ansible_host>]: mitogen has no submodule os
[mux  64290] 13:41:20.827791 D mitogen.importer.[ssh.<ansible_host>]: mitogen has no submodule re
[mux  64290] 13:41:20.827827 D mitogen.importer.[ssh.<ansible_host>]: mitogen has no submodule signal
[mux  64290] 13:41:20.828349 D mitogen.importer.[ssh.<ansible_host>]: mitogen has no submodule socket
[mux  64290] 13:41:20.828462 D mitogen.importer.[ssh.<ansible_host>]: mitogen has no submodule struct
[mux  64290] 13:41:20.829709 D mitogen.importer.[ssh.<ansible_host>]: mitogen has no submodule subprocess
[mux  64290] 13:41:20.829767 D mitogen.importer.[ssh.<ansible_host>]: mitogen has no submodule sys
[mux  64290] 13:41:20.829799 D mitogen.importer.[ssh.<ansible_host>]: mitogen has no submodule termios
[mux  64290] 13:41:20.829821 D mitogen.importer.[ssh.<ansible_host>]: mitogen has no submodule textwrap
[mux  64290] 13:41:20.831820 D mitogen.importer.[ssh.<ansible_host>]: mitogen has no submodule threading
[mux  64290] 13:41:20.831917 D mitogen.importer.[ssh.<ansible_host>]: mitogen has no submodule zlib
ERROR! [mux  64290] 13:41:20.847786 E mitogen.[ssh.<ansible_host>]: while importing u'ansible.module_utils.json_utils'
Traceback (most recent call last):
  File "<stdin>", line 1710, in load_module
SyntaxError: future feature annotations is not defined (json_utils.py, line 27)

[WARNING]: 
[mux  64290] 13:41:20.848992 W mitogen.service: Pool(9a00, size=32, th='mitogen.Pool.9a00.0'): call error: Message(0, 1, 0, 110, 1000, b"\x80\x02X'\x00\x00\x00ansible_mitogen.services.ContextServiceq\x00X\x03"..802): exceptions.SyntaxError: future feature annotations is not defined (json_utils.py, line 27)
  File "<stdin>", line 3860, in _dispatch_one
  File "<stdin>", line 3847, in _parse_request
  File "<stdin>", line 714, in import_module
  File "<stdin>", line 1718, in load_module
  File "<string>", line 1, in <module>
  File "master:/usr/share/ansible/plugins/strategy/mitogen-0.3.9/ansible_mitogen/target.py", line 82, in <module>
    import ansible.module_utils.json_utils
  File "<stdin>", line 1710, in load_module

[WARNING]: 
[task 64324] 13:41:20.850140 W ansible_mitogen.connection: Connection failed; stack configuration was:
({'kwargs': {'check_host_keys': 'enforce',
             'compression': True,
             'connect_timeout': 10,
             'hostname': '<ansible_host>',
             'identities_only': False,
             'identity_file': None,
             'keepalive_count': 10,
             'keepalive_interval': 30,
             'password': None,
             'port': None,
             'python_path': ['/usr/bin/python'],
             'remote_name': None,
             'ssh_args': ['-o',
                          'ControlMaster=auto',
                          '-o',
                          'ControlPersist=1200'],
             'ssh_debug_level': None,
             'ssh_path': 'ssh',
             'username': 'ifalatik'},
  'method': 'ssh'},
 {'enable_lru': True,
  'kwargs': {'connect_timeout': 10,
             'password': None,
             'python_path': ['/usr/bin/python'],
             'remote_name': None,
             'sudo_args': ['-H', '-S', '-n'],
             'sudo_path': 'sudo',
             'username': 'root'},
  'method': 'sudo'})
[mux  64290] 13:41:20.832042 D mitogen.importer.[ssh.<ansible_host>]: mitogen has no submodule thread
[mux  64290] 13:41:20.833506 D mitogen.importer.[ssh.<ansible_host>]: mitogen has no submodule mitogen
[mux  64290] 13:41:20.833595 D mitogen.importer.[ssh.<ansible_host>]: requesting mitogen.service
[mux  64290] 13:41:20.843951 D mitogen.importer.[ssh.<ansible_host>]: mitogen has no submodule grp
[mux  64290] 13:41:20.846127 D mitogen.importer.[ssh.<ansible_host>]: mitogen has no submodule pprint
[mux  64290] 13:41:20.846192 D mitogen.importer.[ssh.<ansible_host>]: mitogen has no submodule pwd
[mux  64290] 13:41:20.846221 D mitogen.importer.[ssh.<ansible_host>]: mitogen has no submodule stat
[mux  64290] 13:41:20.846243 D mitogen.importer.[ssh.<ansible_host>]: requesting mitogen.select
[mux  64290] 13:41:20.846261 D mitogen.importer.[ssh.<ansible_host>]: requesting ansible
[mux  64290] 13:41:20.847635 D mitogen.importer.[ssh.<ansible_host>]: requesting ansible.module_utils
[mux  64290] 13:41:20.847737 D mitogen.importer.[ssh.<ansible_host>]: requesting ansible.module_utils.json_utils
[mux  64290] 13:41:20.848799 D mitogen.[ssh.<ansible_host>]: Dispatcher: Message(2, 0, 0, 101, 1000, '\x80\x02(NX\x16\x00\x00\x00ansible_mitogen.targetq\x00NX\n\x00\x00\x00init_childq'..186) -> CallError(u'exceptions.SyntaxError: future feature annotations is not defined (json_utils.py, line 27)\n  File "<stdin>", line 3860, in _dispatch_one\n  File "<stdin>", line 3847, in _parse_request\n  File "<stdin>", line 714, in import_module\n  File "<stdin>", line 1718, in load_module\n  File "<string>", line 1, in <module>\n  File "master:/usr/share/ansible/plugins/strategy/mitogen-0.3.9/ansible_mitogen/target.py", line 82, in <module>\n    import ansible.module_utils.json_utils\n  File "<stdin>", line 1710, in load_module\n',)
[mux  64290] 13:41:20.855652 D mitogen.parent: creating connection to context 3 using mitogen.ssh
[mux  64290] 13:41:20.856554 D mitogen.parent: command line for Connection(None): ssh -o "LogLevel ERROR" -l ifalatik -o "Compression yes" -o "ServerAliveInterval 30" -o "ServerAliveCountMax 10" -o "BatchMode yes" -o "StrictHostKeyChecking yes" -o ControlMaster=auto -o ControlPersist=1200 <ansible_host> python3 -c "'import sys;sys.path=[p for p in sys.path if p];import binascii,os,zlib;exec(zlib.decompress(binascii.a2b_base64(\"eNqVkl1PwjAYhe/5Fd69bVxGWw3I4hIJ4EciH1lQYpCQfXTaCO3SAhN/vR0zMPDCeNen55y+J9sbOBNfGTcTGUe4pp28QiI9s5Aq/YGwVyvOyTpjiDiUEHzgwKmStiotOV4ow1FQBV2FSRVyC+XAtQyXdvyUzHwfuqHOhYRQJlWJzaYem10Dbe0UszUu/+TxehVGC25T9bXR9UjIerZdvSsJ3onj1MDcJvx7vL1nhNF9gw3XRig59S6KZPGkZzNcboS2tzB6Gd8PB/PH9tOgc9/rzm+DYX8+CdqjUS+AmQ8Udl9j728Hd8/ECsfVS4/FBToWnGM8B7QUK/XGpbcxsUr4TSMNOYloI4yT1GtcshbBgGv2tVyLFUfUgf7DeEgIeZVgWxQZhHGt438tROQmPFbLTHNjULETicq4tJsAOgLsah4miDavWswG0sw/OCYO5BEUC5JmP4M6u3P500/c+V/u323pcdv9OjH8Da3K5uY=\")))'"
[mux  64290] 13:41:20.863631 D mitogen.parent: child for Connection(None) started: pid:64333 stdin:84 stdout:84 stderr:87
[mux  64290] 13:41:21.293237 D mitogen.parent: BootstrapProtocol(ssh.<ansible_host>): first stage started succcessfully
[mux  64290] 13:41:21.382564 D mitogen.parent: BootstrapProtocol(ssh.<ansible_host>): first stage received mitogen.core source
[mux  64290] 13:41:21.392854 D mitogen.parent: BootstrapProtocol(ssh.<ansible_host>): new child booted successfully
[mux  64290] 13:41:21.393153 D mitogen.[ssh.<ansible_host>]: Python version is 3.9.2 (default, Feb 28 2021, 17:03:44) 
[GCC 10.2.1 20210110]
[mux  64290] 13:41:21.393223 D mitogen.[ssh.<ansible_host>]: Parent is context 0 (master); my ID is 3
[mux  64290] 13:41:21.394018 D mitogen.[ssh.<ansible_host>]: pid:246508 ppid:246507 uid:2031/2031, gid:2031/2031 host:'<hostname>'
[mux  64290] 13:41:21.394071 D mitogen.[ssh.<ansible_host>]: Recovered sys.executable: '/usr/bin/python3'
[mux  64290] 13:41:21.394240 D mitogen.parent: starting function call to ssh.<ansible_host>: ansible_mitogen.target.init_child(log_level=10, candidate_temp_dirs=['~/.ansible/tmp', '/var/tmp', '/tmp'])
[mux  64290] 13:41:21.394494 D mitogen.responder: sending ansible (0.55 KiB) to ssh.<ansible_host>
[mux  64290] 13:41:21.394609 D mitogen.responder: sending ansible.module_utils (0.50 KiB) to ssh.<ansible_host>
[mux  64290] 13:41:21.394797 D mitogen.responder: sending ansible.cli (16.41 KiB) to ssh.<ansible_host>
[mux  64290] 13:41:21.394954 D mitogen.responder: sending ansible.cli.arguments (4.53 KiB) to ssh.<ansible_host>
[mux  64290] 13:41:21.395085 D mitogen.responder: sending ansible.cli.arguments.option_helpers (11.94 KiB) to ssh.<ansible_host>
[mux  64290] 13:41:21.395233 D mitogen.responder: sending ansible.cli.playbook (9.03 KiB) to ssh.<ansible_host>
[mux  64290] 13:41:21.395314 D mitogen.responder: sending ansible.config (0.23 KiB) to ssh.<ansible_host>
[mux  64290] 13:41:21.395388 D mitogen.responder: sending ansible.config.manager (10.47 KiB) to ssh.<ansible_host>
[mux  64290] 13:41:21.395453 D mitogen.responder: sending ansible.constants (5.38 KiB) to ssh.<ansible_host>
[mux  64290] 13:41:21.395497 D mitogen.responder: sending ansible.context (1.88 KiB) to ssh.<ansible_host>
[mux  64290] 13:41:21.395553 D mitogen.responder: sending ansible.errors (6.63 KiB) to ssh.<ansible_host>
[mux  64290] 13:41:21.395621 D mitogen.responder: sending ansible.errors.yaml_strings (2.80 KiB) to ssh.<ansible_host>
[mux  64290] 13:41:21.395666 D mitogen.responder: sending ansible.executor (1.14 KiB) to ssh.<ansible_host>
[mux  64290] 13:41:21.395735 D mitogen.responder: sending ansible.executor.play_iterator (13.05 KiB) to ssh.<ansible_host>
[mux  64290] 13:41:21.395815 D mitogen.responder: sending ansible.executor.playbook_executor (10.43 KiB) to ssh.<ansible_host>
[mux  64290] 13:41:21.395867 D mitogen.responder: sending ansible.executor.stats (2.81 KiB) to ssh.<ansible_host>
[mux  64290] 13:41:21.395933 D mitogen.responder: sending ansible.executor.task_queue_manager (12.54 KiB) to ssh.<ansible_host>
[mux  64290] 13:41:21.395990 D mitogen.responder: sending ansible.executor.task_result (7.15 KiB) to ssh.<ansible_host>
[mux  64290] 13:41:21.396048 D mitogen.responder: sending ansible.inventory (0.29 KiB) to ssh.<ansible_host>
[mux  64290] 13:41:21.396192 D mitogen.responder: sending ansible.inventory.data (8.68 KiB) to ssh.<ansible_host>
[mux  64290] 13:41:21.396353 D mitogen.responder: sending ansible.inventory.group (8.73 KiB) to ssh.<ansible_host>
[mux  64290] 13:41:21.396427 D mitogen.responder: sending ansible.inventory.helpers (1.98 KiB) to ssh.<ansible_host>
[mux  64290] 13:41:21.396485 D mitogen.responder: sending ansible.inventory.host (6.66 KiB) to ssh.<ansible_host>
[mux  64290] 13:41:21.396587 D mitogen.responder: sending ansible.inventory.manager (16.50 KiB) to ssh.<ansible_host>
[mux  64290] 13:41:21.396728 D mitogen.responder: sending ansible.module_utils._text (0.89 KiB) to ssh.<ansible_host>
[mux  64290] 13:41:21.396786 D mitogen.responder: sending ansible.module_utils.common (0.53 KiB) to ssh.<ansible_host>
[mux  64290] 13:41:21.396835 D mitogen.responder: sending ansible.module_utils.common._utils (1.32 KiB) to ssh.<ansible_host>
[mux  64290] 13:41:21.396886 D mitogen.responder: sending ansible.module_utils.common.arg_spec (4.84 KiB) to ssh.<ansible_host>
[mux  64290] 13:41:21.396925 D mitogen.responder: sending ansible.module_utils.common.collections (2.41 KiB) to ssh.<ansible_host>
[mux  64290] 13:41:21.397021 D mitogen.responder: sending ansible.module_utils.common.file (2.30 KiB) to ssh.<ansible_host>
[mux  64290] 13:41:21.397088 D mitogen.responder: sending ansible.module_utils.common.json (2.12 KiB) to ssh.<ansible_host>
[mux  64290] 13:41:21.397160 D mitogen.responder: sending ansible.module_utils.common.locale (1.86 KiB) to ssh.<ansible_host>
[mux  64290] 13:41:21.397303 D mitogen.responder: sending ansible.module_utils.common.parameters (13.28 KiB) to ssh.<ansible_host>
[mux  64290] 13:41:21.397426 D mitogen.responder: sending ansible.module_utils.common.process (1.77 KiB) to ssh.<ansible_host>
[mux  64290] 13:41:21.397530 D mitogen.responder: sending ansible.module_utils.common.sys_info (3.05 KiB) to ssh.<ansible_host>
[mux  64290] 13:41:21.397596 D mitogen.responder: sending ansible.module_utils.common.text (0.35 KiB) to ssh.<ansible_host>
[mux  64290] 13:41:21.397649 D mitogen.responder: sending ansible.module_utils.common.text.converters (5.14 KiB) to ssh.<ansible_host>
[mux  64290] 13:41:21.397700 D mitogen.responder: sending ansible.module_utils.common.text.formatters (2.58 KiB) to ssh.<ansible_host>

Ansible config:

ANSIBLE_FORCE_COLOR(/workspaces/workspace/ansible/ansible.cfg) = True
CONFIG_FILE() = /workspaces/workspace/ansible/ansible.cfg
DEFAULT_CALLBACK_PLUGIN_PATH(env: ANSIBLE_CALLBACK_PLUGINS) = ['/workspaces/workspace/.venv/lib/python3.12/site-packages/ara/plugins/callback']
DEFAULT_GATHERING(/workspaces/workspace/ansible/ansible.cfg) = smart
DEFAULT_HOST_LIST(/workspaces/workspace/ansible/ansible.cfg) = ['/workspaces/workspace/ansible/inventory']
DEFAULT_STDOUT_CALLBACK(/workspaces/workspace/ansible/ansible.cfg) = debug
DEFAULT_STRATEGY(/workspaces/workspace/ansible/ansible.cfg) = mitogen_linear
DEFAULT_STRATEGY_PLUGIN_PATH(env: ANSIBLE_STRATEGY_PLUGINS) = ['/usr/share/ansible/plugins/strategy/mitogen-0.3.9/ansible_mitogen/plugins/strategy']
DISPLAY_SKIPPED_HOSTS(/workspaces/workspace/ansible/ansible.cfg) = False
MAX_FILE_SIZE_FOR_DIFF(/workspaces/workspace/ansible/ansible.cfg) = 208896
PAGER(env: PAGER) = less
RETRY_FILES_ENABLED(/workspaces/workspace/ansible/ansible.cfg) = False
moreati commented 3 months ago

Using Python 3.12 on the controller (macOS 14.6, venv) and a Debian 11 target (Proxmox LXC guest) I couldn't reproduce this.

➜  tmp python3.12 -mvenv v312
➜  tmp v312/bin/pip install ansible==10.3.0 mitogen
...
Successfully installed MarkupSafe-2.1.5 PyYAML-6.0.2 ansible-10.3.0 ansible-core-2.17.3 cffi-1.17.0 cryptography-43.0.0 jinja2-3.1.4 mitogen-0.3.9 packaging-24.1 pycparser-2.22 resolvelib-1.0.1
➜  tmp ANSIBLE_STRATEGY=mitogen_linear ANSIBLE_STRATEGY_PLUGINS=v312/lib/python3.12/site-packages/ansible_mitogen/plugins/strategy v312/bin/ansible -mping d11.mynet
[WARNING]: Platform linux on host d11.mynet is using the discovered Python interpreter at /usr/bin/python3.9, but future installation of another Python interpreter could change the meaning of that path. See
https://docs.ansible.com/ansible-core/2.17/reference_appendices/interpreter_discovery.html for more information.
d11.mynet | SUCCESS => {
    "ansible_facts": {
        "discovered_interpreter_python": "/usr/bin/python3.9"
    },
    "changed": false,
    "ping": "pong"
}
➜  tmp v312/bin/python --version; v312/bin/pip list
Python 3.12.5
Package      Version
------------ -------
ansible      10.3.0
ansible-core 2.17.3
cffi         1.17.0
cryptography 43.0.0
Jinja2       3.1.4
MarkupSafe   2.1.5
mitogen      0.3.9
packaging    24.1
pip          24.2
pycparser    2.22
PyYAML       6.0.2
resolvelib   1.0.1
➜  tmp ssh d11.mynet "ls /usr/bin/python*"
/usr/bin/python3
/usr/bin/python3.9
➜  tmp ssh d11.mynet /usr/bin/python3 --version; ssh d11.mynet /usr/bin/python3.9 --version
Python 3.9.2
Python 3.9.2
moreati commented 3 months ago

Could you provide a minimal reproduction without ARA (e.g. single task playbook or /usr/bin/ansible invocation)? I don't think ARA would make a difference, but it's hard for me to rule out completely.

ifalatiksetlog commented 3 months ago

I believe the issue is with the target hosts still having python2 installed:

ssh $hostname "ls -lsh /usr/bin/python*"
   0 lrwxrwxrwx 1 root root    7 Mar  2  2021 /usr/bin/python -> python2
   0 lrwxrwxrwx 1 root root    9 Jul 28  2021 /usr/bin/python2 -> python2.7
3.4M -rwxr-xr-x 1 root root 3.4M Sep 19  2023 /usr/bin/python2.7
   0 lrwxrwxrwx 1 root root    9 Apr  5  2021 /usr/bin/python3 -> python3.9
5.3M -rwxr-xr-x 1 root root 5.3M Feb 28  2021 /usr/bin/python3.9

Minimal reproduction using Vagrant: Vagrantfile:

Vagrant.configure("2") do |config|
    config.vm.box = "debian/bullseye64"
    config.vm.hostname = "debian.local"
    config.vm.provider "virtualbox" do |vb|
        vb.memory = "4096"
        vb.cpus = "4"
    end
    config.vm.synced_folder '.', '/vagrant', disabled: true
    config.vm.network :forwarded_port, guest: 22, host: 2522, auto_correct: false, id: "ssh"
        auto_config = false

    config.vm.provision "shell", inline: <<-SHELL
        apt-get update
        apt-get upgrade -y
        apt-get install -y python2
        ln -s python2 /usr/bin/python
    SHELL

    config.vm.provision "ansible" do |ansible|
        ansible.playbook = "playbook.yml"
        ansible.compatibility_mode = "2.0"
    end
end

ansible.cfg:

[defaults]
inventory = ./hosts.yml
strategy = mitogen_linear
strategy_plugins = /home/ifalatik/projects/python-venvs/ansible/lib/python3.12/site-packages/ansible_mitogen/plugins/strategy

playboook.yml:

---

- hosts: all
  tasks:
    - name: Ping
      ping:

results in:

PLAY [all] *********************************************************************

TASK [Gathering Facts] *********************************************************
ERROR! [mux  67580] 15:33:46.413472 E mitogen.[ssh.172.22.160.1:2522]: while importing u'ansible.module_utils.json_utils'
Traceback (most recent call last):
  File "<stdin>", line 1710, in load_module
SyntaxError: future feature annotations is not defined (json_utils.py, line 27)
ok: [default]
[WARNING]: Platform linux on host default is using the discovered Python
interpreter at /usr/bin/python3.9, but future installation of another Python
interpreter could change the meaning of that path. See
https://docs.ansible.com/ansible-
core/2.17/reference_appendices/interpreter_discovery.html for more information.

TASK [Ping] ********************************************************************
ok: [default]

PLAY RECAP *********************************************************************
default                    : ok=2    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0
ifalatiksetlog commented 2 months ago

Hi there, is there any work being done on this? Do you need more information?

Frazew commented 2 months ago

I also believe this to be an issue with interpreter discovery with mitogen using python2 on systems where python points to it, explicitly setting ansible_python_interpreter=/usr/bin/python3 resolves this issue. I think https://github.com/mitogen-hq/mitogen/issues/1097 cannot work when using mitogen as a strategy plugin before Ansible has had chance to perform interpreter discovery on the remote machine.

It also seems that mitogen fails to consider the global ansible python interpreter configuration? While setting ansible_python_interpreter=/usr/bin/python3 works, having the following in ansible.cfg does not:

[defaults]
interpreter_python = /usr/bin/python3

Feel free to tell me if that would need a separate issue

ifalatiksetlog commented 1 month ago

still broken in 3.11

savchenko commented 3 weeks ago

I am seeing the above-mentioned warning with the following:

Setting interpreter_python and/or ansible_python_interpreter has no effect.

pyproject.toml

[tool.poetry.dependencies]
python = "^3.11"
ansible-core = "^2.17.5"
ansible-lint = { version = "^24.9.2", markers = "platform_system != 'Windows'" }
passlib = "^1.7.4"
jmespath = "^1.0.1"