Error when trying to unpickle data (cannot unpickle 'ansible.utils.unsafe_proxy'/'AnsibleUnsafeText')

vdanjean commented 9 months ago

Main error: mitogen.core.CallError: mitogen.core.StreamError: cannot unpickle 'ansible.utils.unsafe_proxy'/'AnsibleUnsafeText'

Which version of Ansible are you running? Debian unstable packages (for ansible and mitogen) python3-ansible-compat: Installed: 4.1.10-1 ansible-lint: Installed: 6.17.2-1 ansible: Installed: 7.7.0+dfsg-3 ansible-core: Installed: 2.14.13-1 ansible-mitogen: Installed: 0.3.4-2 python3-mitogen: Installed: 0.3.4-2 ansible-mitogen: Installed: 0.3.4-2
Is your version of Ansible patched in any way? Not by me (the +dfsg in Debian version says that Debian remove non-free parts)
Are you running with any custom modules, or module_utils loaded? No
Have you tried the latest master version from Git? No
Do you have some idea of what the underlying problem may be? No
Mention your host and target OS and versions Linux Debian sid (host) and Linux Debian bullseye (target)
Mention your host and target Python versions Host: Python 3.11.7 Target: Python 3.9.2
If reporting a crash or hang in Ansible, please rerun with -vvv and include 200 lines of output around the point of the error, along with a full copy of any traceback or error text in the log. Beware "-vvv" may include secret data! Edit as necessary before posting. Provided in ansible-mitogen-error.txt
If reporting any kind of problem with Ansible, please include the Ansible version along with output of "ansible-config dump --only-changed". Provided in ansible-config-changed.txt file

ansible-mitogen-error.txt ansible-config-changed.txt

vdanjean commented 9 months ago

Note that removing 'strategy = mitogen_linear' in the config makes it working. Here is the log: ansible-plain-no-error.txt

vdanjean commented 9 months ago

Trying to manually unpickle data seems to work, both on host and target (for the later, I needed to copy mitogen and ansible python packages from host into a separate directory on target and add it to PYTHONPATH) Here is the simple script (txt extension for upload): python-test.txt

O host:

$ python3 /tmp/python-test.txt 
('eyak-759992-7ffff7c7f040-5a92dd14c9',
 'ansible_mitogen.target',
 None,
 'run_module',
 (),
 Kwargs({'kwargs': {'runner_name': 'NewStyleRunner', 'module': 'ansible.legacy.systemd', 'path': '/usr/lib/python3/dist-packages/ansible/modules/systemd.py', 'json_args': '{"name": "ntp", "state": "restarted", "_ansible_check_mode": false, "_ansible_no_log": false, "_ansible_debug": false, "_ansible_diff": false, "_ansible_verbosity": 3, "_ansible_version": "2.14.13", "_ansible_module_name": "ansible.legacy.systemd", "_ansible_syslog_facility": "LOG_USER", "_ansible_selinux_special_fs": ["fuse", "nfs", "vboxsf", "ramfs", "9p", "vfat"], "_ansible_string_conversion_action": "warn", "_ansible_socket": null, "_ansible_shell_executable": "/bin/sh", "_ansible_keep_remote_files": false, "_ansible_tmpdir": null, "_ansible_remote_tmp": "/root/.ansible/tmp"}', 'env': {}, 'interpreter_fragment': None, 'is_python': None, 'module_map': {'builtin': ['ansible.module_utils._text', 'ansible.module_utils.basic', 'ansible.module_utils.common', 'ansible.module_utils.common._collections_compat', 'ansible.module_utils.common._json_compat', 'ansible.module_utils.common._utils', 'ansible.module_utils.common.arg_spec', 'ansible.module_utils.common.collections', 'ansible.module_utils.common.file', 'ansible.module_utils.common.locale', 'ansible.module_utils.common.parameters', 'ansible.module_utils.common.process', 'ansible.module_utils.common.sys_info', 'ansible.module_utils.common.text', 'ansible.module_utils.common.text.converters', 'ansible.module_utils.common.text.formatters', 'ansible.module_utils.common.validation', 'ansible.module_utils.common.warnings', 'ansible.module_utils.compat', 'ansible.module_utils.compat._selectors2', 'ansible.module_utils.compat.selectors', 'ansible.module_utils.compat.selinux', 'ansible.module_utils.compat.typing', 'ansible.module_utils.compat.version', 'ansible.module_utils.distro', 'ansible.module_utils.distro._distro', 'ansible.module_utils.errors', 'ansible.module_utils.facts', 'ansible.module_utils.facts.ansible_collector', 'ansible.module_utils.facts.collector', 'ansible.module_utils.facts.compat', 'ansible.module_utils.facts.default_collectors', 'ansible.module_utils.facts.hardware', 'ansible.module_utils.facts.hardware.aix', 'ansible.module_utils.facts.hardware.base', 'ansible.module_utils.facts.hardware.darwin', 'ansible.module_utils.facts.hardware.dragonfly', 'ansible.module_utils.facts.hardware.freebsd', 'ansible.module_utils.facts.hardware.hpux', 'ansible.module_utils.facts.hardware.hurd', 'ansible.module_utils.facts.hardware.linux', 'ansible.module_utils.facts.hardware.netbsd', 'ansible.module_utils.facts.hardware.openbsd', 'ansible.module_utils.facts.hardware.sunos', 'ansible.module_utils.facts.namespace', 'ansible.module_utils.facts.network', 'ansible.module_utils.facts.network.aix', 'ansible.module_utils.facts.network.base', 'ansible.module_utils.facts.network.darwin', 'ansible.module_utils.facts.network.dragonfly', 'ansible.module_utils.facts.network.fc_wwn', 'ansible.module_utils.facts.network.freebsd', 'ansible.module_utils.facts.network.generic_bsd', 'ansible.module_utils.facts.network.hpux', 'ansible.module_utils.facts.network.hurd', 'ansible.module_utils.facts.network.iscsi', 'ansible.module_utils.facts.network.linux', 'ansible.module_utils.facts.network.netbsd', 'ansible.module_utils.facts.network.nvme', 'ansible.module_utils.facts.network.openbsd', 'ansible.module_utils.facts.network.sunos', 'ansible.module_utils.facts.other', 'ansible.module_utils.facts.other.facter', 'ansible.module_utils.facts.other.ohai', 'ansible.module_utils.facts.sysctl', 'ansible.module_utils.facts.system', 'ansible.module_utils.facts.system.apparmor', 'ansible.module_utils.facts.system.caps', 'ansible.module_utils.facts.system.chroot', 'ansible.module_utils.facts.system.cmdline', 'ansible.module_utils.facts.system.date_time', 'ansible.module_utils.facts.system.distribution', 'ansible.module_utils.facts.system.dns', 'ansible.module_utils.facts.system.env', 'ansible.module_utils.facts.system.fips', 'ansible.module_utils.facts.system.loadavg', 'ansible.module_utils.facts.system.local', 'ansible.module_utils.facts.system.lsb', 'ansible.module_utils.facts.system.pkg_mgr', 'ansible.module_utils.facts.system.platform', 'ansible.module_utils.facts.system.python', 'ansible.module_utils.facts.system.selinux', 'ansible.module_utils.facts.system.service_mgr', 'ansible.module_utils.facts.system.ssh_pub_keys', 'ansible.module_utils.facts.system.user', 'ansible.module_utils.facts.timeout', 'ansible.module_utils.facts.utils', 'ansible.module_utils.facts.virtual', 'ansible.module_utils.facts.virtual.base', 'ansible.module_utils.facts.virtual.dragonfly', 'ansible.module_utils.facts.virtual.freebsd', 'ansible.module_utils.facts.virtual.hpux', 'ansible.module_utils.facts.virtual.linux', 'ansible.module_utils.facts.virtual.netbsd', 'ansible.module_utils.facts.virtual.openbsd', 'ansible.module_utils.facts.virtual.sunos', 'ansible.module_utils.facts.virtual.sysctl', 'ansible.module_utils.parsing', 'ansible.module_utils.parsing.convert_bool', 'ansible.module_utils.pycompat24', 'ansible.module_utils.service', 'ansible.module_utils.six'], 'custom': []}, 'py_module_name': 'ansible.modules.systemd', 'good_temp_dir': '/root/.ansible/tmp', 'cwd': None, 'extra_env': None, 'emulate_tty': True, 'service_context': Context(0, None)}}))

On target:

$ PYTHONPATH=/tmp/python python3 /tmp/python-test 
('eyak-759992-7ffff7c7f040-5a92dd14c9',
'ansible_mitogen.target',
None,
'run_module',
(),
{'kwargs': {'cwd': None,
         'emulate_tty': True,
         'env': {},
         'extra_env': None,
         'good_temp_dir': '/root/.ansible/tmp',
         'interpreter_fragment': None,
         'is_python': None,
         'json_args': '{"name": "ntp", "state": "restarted", '
                      '"_ansible_check_mode": false, "_ansible_no_log": '
                      'false, "_ansible_debug": false, "_ansible_diff": '
                      'false, "_ansible_verbosity": 3, "_ansible_version": '
                      '"2.14.13", "_ansible_module_name": '
                      '"ansible.legacy.systemd", '
                      '"_ansible_syslog_facility": "LOG_USER", '
                      '"_ansible_selinux_special_fs": ["fuse", "nfs", '
                      '"vboxsf", "ramfs", "9p", "vfat"], '
                      '"_ansible_string_conversion_action": "warn", '
                      '"_ansible_socket": null, '
                      '"_ansible_shell_executable": "/bin/sh", '
                      '"_ansible_keep_remote_files": false, '
                      '"_ansible_tmpdir": null, "_ansible_remote_tmp": '
                      '"/root/.ansible/tmp"}',
         'module': 'ansible.legacy.systemd',
         'module_map': {'builtin': ['ansible.module_utils._text',
                                    'ansible.module_utils.basic',
                                    'ansible.module_utils.common',
                                    'ansible.module_utils.common._collections_compat',
                                    'ansible.module_utils.common._json_compat',
                                    'ansible.module_utils.common._utils',
                                    'ansible.module_utils.common.arg_spec',
                                    'ansible.module_utils.common.collections',
                                    'ansible.module_utils.common.file',
                                    'ansible.module_utils.common.locale',
                                    'ansible.module_utils.common.parameters',
                                    'ansible.module_utils.common.process',
                                    'ansible.module_utils.common.sys_info',
                                    'ansible.module_utils.common.text',
                                    'ansible.module_utils.common.text.converters',
                                    'ansible.module_utils.common.text.formatters',
                                    'ansible.module_utils.common.validation',
                                    'ansible.module_utils.common.warnings',
                                    'ansible.module_utils.compat',
                                    'ansible.module_utils.compat._selectors2',
                                    'ansible.module_utils.compat.selectors',
                                    'ansible.module_utils.compat.selinux',
                                    'ansible.module_utils.compat.typing',
                                    'ansible.module_utils.compat.version',
                                    'ansible.module_utils.distro',
                                    'ansible.module_utils.distro._distro',
                                    'ansible.module_utils.errors',
                                    'ansible.module_utils.facts',
                                    'ansible.module_utils.facts.ansible_collector',
                                    'ansible.module_utils.facts.collector',
                                    'ansible.module_utils.facts.compat',
                                    'ansible.module_utils.facts.default_collectors',
                                    'ansible.module_utils.facts.hardware',
                                    'ansible.module_utils.facts.hardware.aix',
                                    'ansible.module_utils.facts.hardware.base',
                                    'ansible.module_utils.facts.hardware.darwin',
                                    'ansible.module_utils.facts.hardware.dragonfly',
                                    'ansible.module_utils.facts.hardware.freebsd',
                                    'ansible.module_utils.facts.hardware.hpux',
                                    'ansible.module_utils.facts.hardware.hurd',
                                    'ansible.module_utils.facts.hardware.linux',
                                    'ansible.module_utils.facts.hardware.netbsd',
                                    'ansible.module_utils.facts.hardware.openbsd',
                                    'ansible.module_utils.facts.hardware.sunos',
                                    'ansible.module_utils.facts.namespace',
                                    'ansible.module_utils.facts.network',
                                    'ansible.module_utils.facts.network.aix',
                                    'ansible.module_utils.facts.network.base',
                                    'ansible.module_utils.facts.network.darwin',
                                    'ansible.module_utils.facts.network.dragonfly',
                                    'ansible.module_utils.facts.network.fc_wwn',
                                    'ansible.module_utils.facts.network.freebsd',
                                    'ansible.module_utils.facts.network.generic_bsd',
                                    'ansible.module_utils.facts.network.hpux',
                                    'ansible.module_utils.facts.network.hurd',
                                    'ansible.module_utils.facts.network.iscsi',
                                    'ansible.module_utils.facts.network.linux',
                                    'ansible.module_utils.facts.network.netbsd',
                                    'ansible.module_utils.facts.network.nvme',
                                    'ansible.module_utils.facts.network.openbsd',
                                    'ansible.module_utils.facts.network.sunos',
                                    'ansible.module_utils.facts.other',
                                    'ansible.module_utils.facts.other.facter',
                                    'ansible.module_utils.facts.other.ohai',
                                    'ansible.module_utils.facts.sysctl',
                                    'ansible.module_utils.facts.system',
                                    'ansible.module_utils.facts.system.apparmor',
                                    'ansible.module_utils.facts.system.caps',
                                    'ansible.module_utils.facts.system.chroot',
                                    'ansible.module_utils.facts.system.cmdline',
                                    'ansible.module_utils.facts.system.date_time',
                                    'ansible.module_utils.facts.system.distribution',
                                    'ansible.module_utils.facts.system.dns',
                                    'ansible.module_utils.facts.system.env',
                                    'ansible.module_utils.facts.system.fips',
                                    'ansible.module_utils.facts.system.loadavg',
                                    'ansible.module_utils.facts.system.local',
                                    'ansible.module_utils.facts.system.lsb',
                                    'ansible.module_utils.facts.system.pkg_mgr',
                                    'ansible.module_utils.facts.system.platform',
                                    'ansible.module_utils.facts.system.python',
                                    'ansible.module_utils.facts.system.selinux',
                                    'ansible.module_utils.facts.system.service_mgr',
                                    'ansible.module_utils.facts.system.ssh_pub_keys',
                                    'ansible.module_utils.facts.system.user',
                                    'ansible.module_utils.facts.timeout',
                                    'ansible.module_utils.facts.utils',
                                    'ansible.module_utils.facts.virtual',
                                    'ansible.module_utils.facts.virtual.base',
                                    'ansible.module_utils.facts.virtual.dragonfly',
                                    'ansible.module_utils.facts.virtual.freebsd',
                                    'ansible.module_utils.facts.virtual.hpux',
                                    'ansible.module_utils.facts.virtual.linux',
                                    'ansible.module_utils.facts.virtual.netbsd',
                                    'ansible.module_utils.facts.virtual.openbsd',
                                    'ansible.module_utils.facts.virtual.sunos',
                                    'ansible.module_utils.facts.virtual.sysctl',
                                    'ansible.module_utils.parsing',
                                    'ansible.module_utils.parsing.convert_bool',
                                    'ansible.module_utils.pycompat24',
                                    'ansible.module_utils.service',
                                    'ansible.module_utils.six'],
                        'custom': []},
         'path': '/usr/lib/python3/dist-packages/ansible/modules/systemd.py',
         'py_module_name': 'ansible.modules.systemd',
         'runner_name': 'NewStyleRunner',
         'service_context': Context(0, None)}})

Not sure why the layout is not similar on host and target.

vdanjean commented 9 months ago

Using the command module (calling systemd ntp restart) instead of the service module (that seems to delegate to the systemd module) works! Here is the log. Using the systemd_service module does not work (same pickle error, log can be provided if needed) ansible-mitogen-no-error.txt

upekkha commented 9 months ago

Possibly a duplicate of #1034, where you can also find possible work-arounds.

moreati commented 7 months ago

duplicate of #1034

mitogen-hq / mitogen

Error when trying to unpickle data (cannot unpickle 'ansible.utils.unsafe_proxy'/'AnsibleUnsafeText') #1035