dsgnr commented 3 years ago

When using the package module with the latest RC (as well as latest commit on master branch) of Mitogen for Ansible 2.10.5, I am receiving an exception;

fatal: [EL8]: FAILED! => {
    ansible_facts":
        "discovered_interpreter_python": "/usr/libexec/platform-python"
    "changed": false
    "module_stderr":
        Traceback (most recent call last):
          File "master:/Users/dsgnr/.virtualenvs/ansible/lib/python3.7/site-packages/ansible_mitogen/runner.py", line 975, in _run
            self._run_code(code, mod)
          File "master:/Users/dsgnr/.virtualenvs/ansible/lib/python3.7/site-packages/ansible_mitogen/runner.py", line 939, in _run_code
            exec(code, vars(mod))
          File "master:/Users/dsgnr/.virtualenvs/py3/lib/python3.7/site-packages/ansible/modules/dnf.py", line 1330, in <module>
          File "master:/Users/dsgnr/.virtualenvs/py3/lib/python3.7/site-packages/ansible/modules/dnf.py", line 1319, in main
          File "master:/Users/dsgnr/.virtualenvs/py3/lib/python3.7/site-packages/ansible/modules/dnf.py", line 1288, in run
          File "master:/Users/dsgnr/.virtualenvs/py3/lib/python3.7/site-packages/ansible/modules/dnf.py", line 620, in _base
          File "/usr/lib/python3.6/site-packages/dnf/base.py", line 300, in init_plugins
            self._plugins._load(self.conf, disabled_glob, enable_plugins)
          File "/usr/lib/python3.6/site-packages/dnf/plugin.py", line 131, in _load
            raise RuntimeError("load_plugins() called twice")
        RuntimeError: load_plugins() called twice
    "module_stdout": ""
    "msg":
        MODULE FAILURE
        See stdout/stderr for the exact error
    "rc": 1
}

+- name: Install package
+  package:
+    name: "{{ package }}"
+    state: present

However, using the individual modules works fine;

+- name: Install package for Ubuntu
+  apt:
+    name: "{{ package }}"
+    state: present
+  when: ansible_os_family == "Ubuntu"
+
+- name: Install package for RHEL up to version 7
+  yum:
+    name: "{{ package }}"
+    state: present
+  when: ansible_os_family == "RedHat" and ansible_distribution_major_version|int <= 7
+
+- name: Install package for RHEL 8
+  dnf:
+    name: "{{ package }}"
+    state: present
+  when: ansible_os_family == "RedHat" and ansible_distribution_major_version|int == 8

Which version of Ansible are you running? Ansible 2.10.5
Is your version of Ansible patched in any way?
No
Are you running with any custom modules, or module_utils loaded? No
Have you tried the latest master version from Git? Running v0.3.0-rc.0
Do you have some idea of what the underlying problem may be?
Mention your host and target OS and versions Red Hat Enterprise Linux release 8.3 (Ootpa)

Mention your host and target Python versions

# /usr/libexec/platform-python
Python 3.6.8 (default, Aug 18 2020, 08:33:21)
[GCC 8.3.1 20191121 (Red Hat 8.3.1-5)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>>

EDIT (AW): Formatted error and traceback

wouterhummelink commented 3 years ago

This also occurs if the dnf/yum module is used more than once in the same playbook. I think because mitogen always keeps the process alive that state from a previous call to libdnf is left around, causing subsequent attempts to initialize libdnf to fail. Some code would need te added to clean up after the dnf module.

ansible_facts: {}
  module_stderr: |-
    Traceback (most recent call last):
      File "master:/home/wouterhummelink/.local/lib/python3.9/site-packages/ansible_mitogen/runner.py", line 975, in _run
        self._run_code(code, mod)
      File "master:/home/wouterhummelink/.local/lib/python3.9/site-packages/ansible_mitogen/runner.py", line 939, in _run_code
        exec(code, vars(mod))
      File "master:/home/wouterhummelink/.local/lib/python3.9/site-packages/ansible/modules/dnf.py", line 1330, in <module>
      File "master:/home/wouterhummelink/.local/lib/python3.9/site-packages/ansible/modules/dnf.py", line 1319, in main
      File "master:/home/wouterhummelink/.local/lib/python3.9/site-packages/ansible/modules/dnf.py", line 1288, in run
      File "master:/home/wouterhummelink/.local/lib/python3.9/site-packages/ansible/modules/dnf.py", line 620, in _base
      File "/usr/lib/python3.6/site-packages/dnf/base.py", line 300, in init_plugins
        self._plugins._load(self.conf, disabled_glob, enable_plugins)
      File "/usr/lib/python3.6/site-packages/dnf/plugin.py", line 131, in _load
        raise RuntimeError("load_plugins() called twice")
    RuntimeError: load_plugins() called twice
  module_stdout: ''
  msg: |-
    MODULE FAILURE
    See stdout/stderr for the exact error
  rc: 1

wouterhummelink commented 3 years ago

This could be solved by calling into dnf.plugin.Plugins._unload() but I'm not quite sure where such cleanup should go... ansible or mitogen

wouterhummelink commented 3 years ago

I've locally patched dnf.py in ansible to call _unload() just before the call to init_plugins() that appears to successfully work around this issue.

wouterhummelink commented 3 years ago

@moreati The ansible project appears to be unwilling to fix this on their end.

moreati commented 3 years ago

thank you, I don't think this will make it into the next release candidate. Possibly the one after.

fauust commented 3 years ago

Hi, I can confirm this on 3 centos8 and a rhel8. Any help you might need in debugging this, just ping me (I am very happy that mitogen finally is usable with ansible 2.10)!

Aethylred commented 3 years ago

Another confirmation for CentOS8

Madic- commented 3 years ago

As workaround I added the following to the task:

  vars:
    mitogen_task_isolation: fork

Example:

- name: Podman | Installing podman....
  package:
    name:
      - podman
    state: present
  vars:
    mitogen_task_isolation: fork

fauust commented 3 years ago

@Madic- workaround works for me too.

JiffsMaverick commented 3 years ago

I can confirm this bug for Oracle Linux 8. Mitogen 0.3.0-rc1 and ansible-base 2.10.5.

moreati commented 3 years ago

816 attempting to reproduce as part of #816

moreati commented 3 years ago

I've reproduced this in https://github.com/moreati/mitogen/tree/issue776, after checking out the branch you can use Tox to run these tests, e.g.

ANSIBLE_TAGS=issue_776 tox -e py39-mode_ansible-distros_centos8

annttu commented 3 years ago

Adding ansible.legacy.dnf to ALWAYS_FORK_MODULES list in ansible_mitogen/planner.py seems to fix this issue with yum.

+++ ./ansible_mitogen/planner.py    2021-03-26 14:18:56.000000000 +0200
@@ -321,6 +321,7 @@
     ALWAYS_FORK_MODULES = frozenset([
         'dnf',  # issue #280; py-dnf/hawkey need therapy
         'firewalld',  # issue #570: ansible module_utils caches dbus conn
+        'ansible.legacy.dnf',  # issue #776
     ])

     def should_fork(self):

pun-ky commented 3 years ago

@annttu yep, I could confirm that it works with RHEL8/Python3 and Ansible 2.10 .

thanks for sharing this fix! guys could you somehow release the fix even as an unofficial version?

right now I need to reference fork in my build but it just would be nice to reference your awesome project ;)

btw maybe this ALWAYS_FORK_MODULES param should be a configurable option from ansible.cfg? :)

DaVince commented 3 years ago

Adding ansible.legacy.dnf to ALWAYS_FORK_MODULES list in ansible_mitogen/planner.py seems to fix this issue with yum.

+++ ./ansible_mitogen/planner.py  2021-03-26 14:18:56.000000000 +0200
@@ -321,6 +321,7 @@
     ALWAYS_FORK_MODULES = frozenset([
         'dnf',  # issue #280; py-dnf/hawkey need therapy
         'firewalld',  # issue #570: ansible module_utils caches dbus conn
+        'ansible.legacy.dnf',  # issue #776
     ])

     def should_fork(self):

Can confirm this fixes it on Ansible 2.10.7 on MacOS as well, for both package and dnf.

bradh352 commented 3 years ago

If you want to easily install the PR #845 that contains this fix without manually patching, you can run something like: pip install git+https://github.com/mitogen-hq/mitogen.git@refs/pull/845/merge

Nothing4You commented 3 years ago

if you look at the ALWAYS_FORK_MODULES list you can see that dnf already was in there for a different reason (#280), it just wasn't migrated to the new naming scheme i guess?

momiji commented 3 years ago

Hello, this is a must have! Please release version 0.3.0 in pypi with this patch, it really a required on redhat8. Good job !

mitogen-hq / mitogen

RuntimeError: load_plugins() called twice - Ansible 2.10.5, RHEL 8, package module #776

816 attempting to reproduce as part of #816