search

theforeman / foreman-ansible-modules

Ansible modules for interacting with the Foreman API and various plugin APIs such as Katello
GNU General Public License v3.0
147 stars 163 forks source link

Module theforeman.foreman.host fails when Host Parameters is defined #1336

Closed mrjk closed 2 years ago

mrjk commented 2 years ago
SUMMARY

Module theforeman.foreman.host fails when a parameter is set in Host Parameters in Foreman. It may also impact more modules in different conditions.

ISSUE TYPE
ANSIBLE VERSION
$ ansible --version
ansible [core 2.11.4] 
  config file = REPO/ansible.cfg
  configured module search path = ['USER/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location =REPO/.tmp/lib/python3.10/site-packages/ansible
  ansible collection location = REPO/collections
  executable location = REPO/.tmp/bin/ansible
  python version = 3.10.1 (main, Dec 18 2021, 23:53:45) [GCC 11.1.0]
  jinja version = 3.0.3
  libyaml = True

$ python --version
Python 3.10.1

I explicitly mention this is Python 3.10. I know those module does not support (yet) this version, but it's worth mention it.

COLLECTION VERSION
$ ansible-galaxy collection list

# REPO/collections/ansible_collections
Collection         Version
------------------ -------
ansible.utils      2.4.2  
community.libvirt  1.0.1  
theforeman.foreman 3.1.0  
theforeman.foreman 3.1.0  

# REPO/.tmp/lib/python3.10/site-packages/ansible_collections
Collection                    Version
----------------------------- -------
amazon.aws                    1.5.0  
ansible.netcommon             2.3.0  
ansible.posix                 1.2.0  
ansible.utils                 2.3.1  
ansible.windows               1.7.2  
arista.eos                    2.2.0  
awx.awx                       19.2.2 
azure.azcollection            1.8.0  
check_point.mgmt              2.0.0  
chocolatey.chocolatey         1.1.0  
cisco.aci                     2.0.0  
cisco.asa                     2.0.2  
cisco.intersight              1.0.16 
cisco.ios                     2.3.1  
cisco.iosxr                   2.4.0  
cisco.meraki                  2.4.2  
cisco.mso                     1.2.0  
cisco.nso                     1.0.3  
cisco.nxos                    2.5.0  
cisco.ucs                     1.6.0  
cloudscale_ch.cloud           2.2.0  
community.aws                 1.5.0  
community.azure               1.0.0  
community.crypto              1.8.0  
community.digitalocean        1.8.0  
community.docker              1.9.0  
community.fortios             1.0.0  
community.general             3.5.0  
community.google              1.0.0  
community.grafana             1.2.1  
community.hashi_vault         1.3.2  
community.hrobot              1.1.1  
community.kubernetes          1.2.1  
community.kubevirt            1.0.0  
community.libvirt             1.0.2  
community.mongodb             1.3.0  
community.mysql               2.1.0  
community.network             3.0.0  
community.okd                 1.1.2  
community.postgresql          1.4.0  
community.proxysql            1.1.0  
community.rabbitmq            1.1.0  
community.routeros            1.2.0  
community.skydive             1.0.0  
community.sops                1.1.0  
community.vmware              1.12.0 
community.windows             1.6.0  
community.zabbix              1.4.0  
containers.podman             1.6.2  
cyberark.conjur               1.1.0  
cyberark.pas                  1.0.7  
dellemc.enterprise_sonic      1.1.0  
dellemc.openmanage            3.6.0  
dellemc.os10                  1.1.1  
dellemc.os6                   1.0.7  
dellemc.os9                   1.0.4  
f5networks.f5_modules         1.11.0 
fortinet.fortimanager         2.1.3  
fortinet.fortios              2.1.2  
frr.frr                       1.0.3  
gluster.gluster               1.0.1  
google.cloud                  1.0.2  
hetzner.hcloud                1.4.4  
hpe.nimble                    1.1.3  
ibm.qradar                    1.0.3  
infinidat.infinibox           1.2.4  
inspur.sm                     1.2.0  
junipernetworks.junos         2.4.0  
kubernetes.core               1.2.1  
mellanox.onyx                 1.0.0  
netapp.aws                    21.6.0 
netapp.azure                  21.8.1 
netapp.cloudmanager           21.9.0 
netapp.elementsw              21.6.1 
netapp.ontap                  21.9.0 
netapp.um_info                21.7.0 
netapp_eseries.santricity     1.2.13 
netbox.netbox                 3.1.1  
ngine_io.cloudstack           2.1.0  
ngine_io.exoscale             1.0.0  
ngine_io.vultr                1.1.0  
openstack.cloud               1.5.0  
openvswitch.openvswitch       2.0.0  
ovirt.ovirt                   1.5.4  
purestorage.flasharray        1.10.0 
purestorage.flashblade        1.6.0  
sensu.sensu_go                1.11.1 
servicenow.servicenow         1.0.6  
splunk.es                     1.0.2  
t_systems_mms.icinga_director 1.20.0 
theforeman.foreman            2.1.2  
vyos.vyos                     2.5.0  
wti.remote                    1.0.1
KATELLO/FOREMAN VERSION
# rpm -q tfm-rubygem-katello foreman
tfm-rubygem-katello-3.11.1-1.el7.noarch
foreman-1.21.3-1.el7.noarch

I know this version is quite old.

STEPS TO REPRODUCE

To reproduce this bug, go into any Foreman host in the Web UI, click Edit, in the Parameters tab, there is a Host Parameters section. Ansible Host module will crash if any parameter is set here.

- name: Register host in Foreman
  hosts: "{{ targets|d('site') }}"
  gather_facts: true
  gather_subset: all
  tags:
    - register_foreman

  tasks:

    - name: "Ensure host is correctly setup into Foreman"    
      delegate_to: KAT_HOST            
      #no_log: true    
      theforeman.foreman.host:    
        username: "{{ system_secrets_db.foreman_svc_admin.user }}"    
        password: "{{ system_secrets_db.foreman_svc_admin.password }}"    
        server_url: "https://KAT_HOST/"            
        validate_certs: false    

        state: present    
        name: "{{ ansible_fqdn }}"    
        mac: "{{ ansible_default_ipv4.macaddress }}"    

        environment: "{{ foreman_environment_name }}"    
        organization: "{{ foreman_organization_name }}"    
        location: "{{ foreman_location_name }}"    
        hostgroup: "{{ foreman_hostgroup_title }}"
EXPECTED RESULTS

The module should not crash when a Host Parameter is set.

ACTUAL RESULTS

There is below a formatted trace:

fatal: [TEST_NODE -> KAT_HOST]: FAILED! => {"changed": false, "module_stderr": "/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:1020: InsecureRequestWarning: Unverified HTTPS request is being made to host 'KAT_HOST'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings                                                                                                                       InsecureRequestWarning,    
Traceback (most recent call last):    
  File \"<stdin>\", line 100, in <module>    
  File \"<stdin>\", line 92, in _ansiballz_main                                                                                                                                                                                                                File \"<stdin>\", line 41, in invoke_module    
  File \"/usr/lib64/python2.7/runpy.py\", line 176, in run_module    
    fname, loader, pkg_name)    
  File \"/usr/lib64/python2.7/runpy.py\", line 82, in _run_module_code    
    mod_name, mod_fname, mod_loader, pkg_name)    
  File \"/usr/lib64/python2.7/runpy.py\", line 72, in _run_code    
    exec code in run_globals    
  File \"/tmp/ansible_theforeman.foreman.host_payload_YyqUDz/ansible_theforeman.foreman.host_payload.zip/ansible_collections/theforeman/foreman/plugins/modules/host.py\", line 535, in <module>    
  File \"/tmp/ansible_theforeman.foreman.host_payload_YyqUDz/ansible_theforeman.foreman.host_payload.zip/ansible_collections/theforeman/foreman/plugins/modules/host.py\", line 525, in main    
  File \"/tmp/ansible_theforeman.foreman.host_payload_YyqUDz/ansible_theforeman.foreman.host_payload.zip/ansible_collections/theforeman/foreman/plugins/module_utils/foreman_helper.py\", line 337, in run    
  File \"/tmp/ansible_theforeman.foreman.host_payload_YyqUDz/ansible_theforeman.foreman.host_payload.zip/ansible_collections/theforeman/foreman/plugins/module_utils/foreman_helper.py\", line 214, in run    
  File \"/tmp/ansible_theforeman.foreman.host_payload_YyqUDz/ansible_theforeman.foreman.host_payload.zip/ansible_collections/theforeman/foreman/plugins/module_utils/foreman_helper.py\", line 1741, in parameters_list_to_str_list    
KeyError: 'parameter_type'    
", "module_stdout": "", "msg": "MODULE FAILURE    
See stdout/stderr for the exact error", "rc": 1}
TECHNICAL ANALYSIS

As we can see, it's inforeman_helper.py line 1741, in parameters_list_to_str_list function. It seems that new_param does not have a ['parameter_type'] attribute. 

1736 # Helper for converting lists of parameters
1737 def parameters_list_to_str_list(parameters):
1738     filtered_params = []
1739     for param in parameters:
1740         new_param = {k: v for (k, v) in param.items() if k in parameter_ansible_spec.keys()}
1741         new_param['value'] = parameter_value_to_str(new_param['value'], new_param['parameter_type'])
1742         filtered_params.append(new_param)
1743     return filtered_params

If I monkey patch the 1741 line:

new_param['value'] = parameter_value_to_str(new_param['value'], new_param.get('parameter_type', 'string'))

It solves the issue. However, it's probably not the correct fix to apply, as it should be applied at the object creation.

NOTES

I'm not sure about anything except the technical explanation. I'm not sure about my Foreman instance reliability, the problem may be somewhere else also and impact all module using this function.

evgeni commented 2 years ago

The short version is: your Foreman is too old and we don't support it anymore.

The longer is: Foreman introduced typed parameters in either 1.22 or 1.24 and the API would return the type correctly since. We do inject type=string for parameters coming from a playbook, if nothing else is set, but try not to guess what's in Foreman.

Your hack seems to solve the issue for you, so feel free to use it, but we would prefer not to carry around workarounds for such old Foreman versions.

mrjk commented 2 years ago

Hey @evgeni , thanks for your quick answer.

I'm agree with you in the sense of our instance is way too old to be supported. However, from a coder perspective, I guess it would be also a good idea to have a string as default type when the parameter_type is missing :p

Anyway, thanks for your support, I'll keep my hack until we'll update our Foreman instance.