Firewalld restart handler issue

[rpreeves]

SUMMARY

Our pipeline is failing at times due to the attempt restart firewalld during Infinispan deployment.

ISSUE TYPE

• Bug Report

ANSIBLE VERSION

ansible [core 2.16.0]
  config file = /etc/ansible/ansible.cfg
  configured module search path = ['/home/rreeves/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/local/lib/python3.11/site-packages/ansible
  ansible collection location = /home/rreeves/.ansible/collections:/usr/share/ansible/collections
  executable location = /usr/local/bin/ansible
  python version = 3.11.5 (main, Sep 22 2023, 15:34:29) [GCC 8.5.0 20210514 (Red Hat 8.5.0-20)] (/usr/bin/python3.11)
  jinja version = 3.1.4
  libyaml = True

COLLECTION VERSION

# /usr/local/lib/python3.11/site-packages/ansible_collections
Collection                    Version
----------------------------- -------
amazon.aws                    7.0.0
ansible.netcommon             5.3.0
ansible.posix                 1.5.4
ansible.utils                 2.11.0
ansible.windows               2.1.0
arista.eos                    6.2.1
awx.awx                       23.3.1
azure.azcollection            1.19.0
check_point.mgmt              5.1.1
chocolatey.chocolatey         1.5.1
cisco.aci                     2.8.0
cisco.asa                     4.0.3
cisco.dnac                    6.7.6
cisco.intersight              2.0.3
cisco.ios                     5.2.0
cisco.iosxr                   6.1.0
cisco.ise                     2.5.16
cisco.meraki                  2.16.14
cisco.mso                     2.5.0
cisco.nxos                    5.2.1
cisco.ucs                     1.10.0
cloud.common                  2.1.4
cloudscale_ch.cloud           2.3.1
community.aws                 7.0.0
community.azure               2.0.0
community.ciscosmb            1.0.7
community.crypto              2.16.0
community.digitalocean        1.24.0
community.dns                 2.6.3
community.docker              3.4.11
community.general             8.0.2
community.grafana             1.6.1
community.hashi_vault         6.0.0
community.hrobot              1.8.2
community.libvirt             1.3.0
community.mongodb             1.6.3
community.mysql               3.8.0
community.network             5.0.2
community.okd                 2.3.0
community.postgresql          3.2.0
community.proxysql            1.5.1
community.rabbitmq            1.2.3
community.routeros            2.10.0
community.sap                 2.0.0
community.sap_libs            1.4.1
community.sops                1.6.7
community.vmware              4.0.0
community.windows             2.0.0
community.zabbix              2.1.0
containers.podman             1.11.0
cyberark.conjur               1.2.2
cyberark.pas                  1.0.23
dellemc.enterprise_sonic      2.2.0
dellemc.openmanage            8.4.0
dellemc.powerflex             2.0.1
dellemc.unity                 1.7.1
f5networks.f5_modules         1.27.0
fortinet.fortimanager         2.3.0
fortinet.fortios              2.3.4
frr.frr                       2.0.2
gluster.gluster               1.0.2
google.cloud                  1.2.0
grafana.grafana               2.2.3
hetzner.hcloud                2.3.0
hpe.nimble                    1.1.4
ibm.qradar                    2.1.0
ibm.spectrum_virtualize       2.0.0
ibm.storage_virtualize        2.1.0
infinidat.infinibox           1.3.12
infoblox.nios_modules         1.5.0
inspur.ispim                  2.1.0
inspur.sm                     2.3.0
junipernetworks.junos         5.3.0
kubernetes.core               2.4.0
lowlydba.sqlserver            2.2.2
microsoft.ad                  1.3.0
netapp.aws                    21.7.1
netapp.azure                  21.10.1
netapp.cloudmanager           21.22.1
netapp.elementsw              21.7.0
netapp.ontap                  22.8.2
netapp.storagegrid            21.11.1
netapp.um_info                21.8.1
netapp_eseries.santricity     1.4.0
netbox.netbox                 3.15.0
ngine_io.cloudstack           2.3.0
ngine_io.exoscale             1.1.0
openstack.cloud               2.1.0
openvswitch.openvswitch       2.1.1
ovirt.ovirt                   3.2.0
purestorage.flasharray        1.22.0
purestorage.flashblade        1.14.0
purestorage.fusion            1.6.0
sensu.sensu_go                1.14.0
splunk.es                     2.1.0
t_systems_mms.icinga_director 2.0.1
telekom_mms.icinga_director   1.34.1
theforeman.foreman            3.14.0
vmware.vmware_rest            2.3.1
vultr.cloud                   1.10.0
vyos.vyos                     4.1.0
wti.remote                    1.0.5

STEPS TO REPRODUCE

1. Setup project
2. Run playbook
3. Observe issue with firewall restart

- name: "Setup Infinispan node"
  include_role:
    name: middleware_automation.infinispan.infinispan
  vars:
    infinispan_jvm_package: "{{ _infinispan_jvm_package }}"
    infinispan_version: "{{ infinispan_release }}"
    infinispan_installation_path: "{{ _infinispan_install_path }}"
    infinispan_supervisor_password: "{{ _infinispan_supervisor_password }}"
    infinispan_users: "{{ _infinispan_users }}"
    infinispan_config_override_template: "templates/infinispan-nxt.xml.j2"
    infinispan_configure_firewalld: true
    infinispan_healthcheck: false

EXPECTED RESULTS

Successful playbook execution, without error.

ACTUAL RESULTS

09:03:15  RUNNING HANDLER [middleware_automation.infinispan.infinispan : Restart firewalld] ***
09:03:42  [WARNING]: Module invocation had junk after the JSON data: polkit-agent-
09:03:42  helper-1: pam_authenticate failed: Authentication failure
09:03:50  fatal: [MTTLVA-CPSVST05]: FAILED! => {"changed": false, "msg": "Unable to restart service firewalld: Failed to restart firewalld.service: Connection timed out\nSee system logs and 'systemctl status firewalld.service' for details.\n"}
09:03:50  fatal: [MTTLVA-CPSVST04]: FAILED! => {"changed": false, "msg": "Unable to restart service firewalld: Failed to restart firewalld.service: Connection timed out\nSee system logs and 'systemctl status firewalld.service' for details.\n"}
09:03:50  fatal: [MTTLVA-CPSVST06]: FAILED! => {"changed": false, "msg": "Unable to restart service firewalld: Failed to restart firewalld.service: Connection timed out\nSee system logs and 'systemctl status firewalld.service' for details.\n"}

[guidograzioli]

Hello thanks for reporting; I tried to reproduce your issue, but I was not able to see this behavior. I do not see either middleware_automation.infinispan nor redhat.data_grid in the installed collection list, which version are you using? Also, on what distribution are you deploying?

[guidograzioli]

Closing with no info; welcome to reopen if error happens again.