IBM / ansible-for-i

the tool is to provide several customized modules for Ansible to manage IBM i systems.
GNU General Public License v3.0
56 stars 55 forks source link

ibmi-fix-check.yml - Error 403: Forbidden, but curl works. #153

Closed K2701 closed 1 year ago

K2701 commented 1 year ago

I am running Ansible on an IBMi partition at V7R3 as the central "Control Node". The versions of software on the IBMi partition are:

ibmi_module_version = "1.9.1"

-bash-5.1$ ansible --version ansible 2.9.10 python version = 3.6.15 (default, Dec 17 2021, 09:57:34) [GCC 6.3.0] ===============================================

When running the example IBM playbook named "ibmi-fix-check.yml" in Ansible on the IBMi partition, with the "validate_certs=False" all of the YAML tasks return ("error": "HTTP Error 403: Forbidden",).
However, from the same partition we can use "curl" and successfully access each of the web pages and the data returns as expected.

Code Snippet: =========================

Results from the above: ======================= <127.0.0.1> ESTABLISH LOCAL CONNECTION FOR USER: xxxxxx <127.0.0.1> EXEC /bin/sh -c '/QOpensys/pkgs/bin/python3.6 && sleep 0' ok: [127.0.0.1] => { "certs": false, "changed": false, "count": 0, "elapsed_time": "0:00:03.347611", "end": "2023-06-05 09:11:52.503603", "expanded_requisites": false, "group_info": [], "invocation": { "module_args": { "expanded_requisites": false, "groups": null, "ptfs": [ "SI71691", "SI74112" ], "timeout": 30, "validate_certs": false } }, "ptf_info": [ { "ptf_id": "SI71691", "req_list": [ { "error": "HTTP Error 403: Forbidden", "url": "https://www.ibm.com/support/pages/ptf/SI71691" } ] }, { "ptf_id": "SI74112", "req_list": [ { "error": "HTTP Error 403: Forbidden", "url": "https://www.ibm.com/support/pages/ptf/SI74112" } ] } ], "rc": 0, "start": "2023-06-05 09:11:49.155992", "stderr": "", "stderr_lines": [], "timeout": 30 } =======================

However, if I run the same ibmi-fix-check.yml file with the "validate_certs=True" the results still fail but with a different error. Code Snippet of Results from "validate_certs=True". ================================ <127.0.0.1> ESTABLISH LOCAL CONNECTION FOR USER: enskdr <127.0.0.1> EXEC /bin/sh -c '/QOpensys/pkgs/bin/python3.6 && sleep 0' ok: [127.0.0.1] => { "certs": true, "changed": false, "count": 0, "elapsed_time": "0:00:02.500378", "end": "2023-06-05 09:09:58.616488", "expanded_requisites": false, "group_info": [], "invocation": { "module_args": { "expanded_requisites": false, "groups": null, "ptfs": [ "SI71691", "SI74112" ], "timeout": 10, "validate_certs": true } }, "ptf_info": [ { "ptf_id": "SI74112", "req_list": [ { "error": "'int' object is not callable", "url": "https://www.ibm.com/support/pages/ptf/SI74112" } ] }, { "ptf_id": "SI71691", "req_list": [ { "error": "'int' object is not callable", "url": "https://www.ibm.com/support/pages/ptf/SI71691" } ] } ], "rc": 0, "start": "2023-06-05 09:09:56.116110", "stderr": "", "stderr_lines": [], "timeout": 10 }

================================

As per the documentation I have the ca-certficates-mozilla installed. Since this is running (delegated) to an IBM i server....... ================================= notes:

================================= Installed Packages
Name : ca-certificates-mozilla
Arch : noarch
Version : 2022.2.58
Release : 1
Size : 1.0 M
Repo : installed
From repo : ibmi-base
Summary : The Mozilla CA root certificate bundle
License : MPL-2.0
Description : This package contains the set of CA certificates chosen by the : Mozilla Foundation for use with the Internet PKI.
=================================

That package installs under /QOpenSys/pkgs/share/pki/trust and it is named mozilla.p11-kit. ============================================== Work with Object Links
Directory . . . . : /QOpenSys/pkgs/share/pki/trust
Type options, press Enter.
2=Edit 3=Copy 4=Remove 5=Display 7=Rename 8=Display attributes 11=Change current directory ...

Opt Object link Type Attribute Text
. DIR
.. DIR
anchors DIR
blacklist DIR
mozilla.p11-kit STMF
==============================================

Why with "validate_certs=False" does a Error 403: Forbidden get returned? Why with "validate_certs=True" does the "error": "'int' object is not callable"" get returned?

In the urls.py module the certs path only gets resolved to the following locations ================= /etc/ansible /QOpenSys/etc/ssl/certs ================= So, based on the path that resolves in the code is would "appear" this module is not searched.

Is it possible that the IBM changes to secure the Fix Central servers in August of 2022 is related? (** Fix Central unsecured and anonymous FTP downloads have been disabled as of 31 August 2022.) https://www.ibm.com/support/pages/node/6475697

Perhaps the required CA certificate from IBM for the PSP site is NOT in the certs list or bundles on V7R3 since the restrictions went into place in August 2022, and would this be affecting the results when "validate_certs=True" ?

Thank You, { End }

robgjertsen1 commented 1 year ago

Yes, there is an issue with both fix_check and also group_fix_check modules getting the HTTP 403 error from the IBM service site. I will put out a patch for these files so that the request is not rejected by the HTTP server. The code is using the default python urllib header for the http/https requests and this no longer works with the web server(s) for the IBM service site; I assume an upgrade occurred with web server code that now blocks this default python request header.

I have reproduced the issue myself and also verified an initial fix that resolves the problem.

There may also be an issue with validate_certs set to true that I can look into later where we may need to refresh the certificate.

robgjertsen1 commented 1 year ago

I've placed a patch in the branch gjertsen-patch-fix-check that should resolve the http 403 error with the fix check modules.

2 module files changed: plugins/modules/ibmi_fix_check.py plugins/modules/ibmi_fix_group_check.py

You can copy these two files to the ansible-for-i collection location on the managing host in the plugins/modules subdirectory, e.g.,

/home/gjertsen/.ansible/collections/ansible_collections/ibm/power_ibmi/plugins/modules

Note that this doesn't address the cert error that you mentioned.

K2701 commented 1 year ago

I downloaded both patches and replaced the modules as instructed. I then tested the "plugins/modules/ibmi_fix_check.py" module with the "validate_certs=False" and it worked as expected. Of course, with "validate_certs=True" the process fails with the "error": "'int' object is not callable" message. When accessing the "HTTPCookieProcessor object. ===================================== (Pdb) step TypeError: 'int' object is not callable

/QOpenSys/pkgs/lib/python3.6/urllib/request.py(526)open() -> response = self._open(req, data) (Pdb) p processor <urllib.request.HTTPCookieProcessor object at 0x700000000b85a58> ===================================== Thank You,

K2701 commented 1 year ago

Walking the code in the PDB debugger I found the issue appears to be in the (/QOpenSys/pkgs/lib/python3.6/site-packages/ansible/module_utils/urls.py) module.

This again is the 2.9.10 ansible release level.

When the function (urllib_request.urlopen(urlopen_args) is called the "context" object for the SSLCONTEXT appears it was not passed in the arguments (urlopen_args). ============================================================

/QOpenSys/pkgs/lib/python3.6/site-packages/ansible/module_utils/urls.py(1295)open() -> r = urllib_request.urlopen(*urlopen_args) ============================================================

In that function I saw the context was NONE. You can see the lines of code it skipped and ran below. ================================================

/QOpenSys/pkgs/lib/python3.6/urllib/request.py(140)urlopen() -> def urlopen(url, data=None, timeout=socket._GLOBAL_DEFAULT_TIMEOUT, (Pdb) a url = <ansible.module_utils.urls.RequestWithMethod object at 0x700000000b89160> data = None timeout = 30 (Pdb) next /QOpenSys/pkgs/lib/python3.6/urllib/request.py(200)urlopen() -> if cafile or capath or cadefault: (Pdb) n /QOpenSys/pkgs/lib/python3.6/urllib/request.py(216)urlopen() -> elif context: (Pdb) n /QOpenSys/pkgs/lib/python3.6/urllib/request.py(219)urlopen() -> elif _opener is None: (Pdb) n /QOpenSys/pkgs/lib/python3.6/urllib/request.py(222)urlopen() -> opener = _opener (Pdb) n /QOpenSys/pkgs/lib/python3.6/urllib/request.py(223)urlopen() -> return opener.open(url, data, timeout) (Pdb) n TypeError: 'int' object is not callable /QOpenSys/pkgs/lib/python3.6/urllib/request.py(223)urlopen() -> return opener.open(url, data, timeout) (Pdb) pp context None ================================================

And then it appears to try and create a "Default" context in the make_context() function. Note: HAS_SSLCONTEXT is "TRUE". ================================================

/QOpenSys/pkgs/lib/python3.6/site-packages/ansible/module_utils/urls.py(927)make_context() -> if HAS_SSLCONTEXT: (Pdb) pp cadata None (Pdb) pp HAS_SSLCONTEXT True (Pdb) next /QOpenSys/pkgs/lib/python3.6/site-packages/ansible/module_utils/urls.py(928)make_context() -> context = create_default_context(cafile=cafile) (Pdb) pp cafile None (Pdb) step --Call-- /QOpenSys/pkgs/lib/python3.6/ssl.py(488)create_default_context() -> def create_default_context(purpose=Purpose.SERVER_AUTH, *, cafile=None, (Pdb) (Pdb) a purpose = <Purpose.SERVER_AUTH: _ASN1Object(nid=129, shortname='serverAuth', longname='TLS Web Server Authentication', oid='1.3.6.1.5.5.7.3.1')> (Pdb) ================================================

I examined the most current versions of the URL.PY code on Github and noticed that at Ansible Version 2.13 there was a change for the was the (context) was handled.

I downloaed the following modules from the current "devel" branch section of the github respository. (Note: I had to "add" the compat.typing.py module as it was introduced at Ver 2.13

ansible_ansible_devel_lib_ansible_module_utils_urls.py ansible_ansible_devel_lib_ansible_module_utils_compat_typing.py

Now when i run the ibmi-fix-check.yml file with "validate_certs=true" the process works. ================================================= TASK [display_two_ptf] ** task path: /home/xxxxxx/ibmi-fix-check.yml:23 ok: [127.0.0.1] => { "msg": { "certs": true, "changed": false, "count": 0, "elapsed_time": "0:00:05.048376", "end": "2023-06-09 17:04:39.208992", "expanded_requisites": false, "failed": false, "group_info": [], "http_agent": "ansible/ibm.power_ibmi", "ptf_info": [ { "ptf_id": "SI74112", "req_list": [ { "ptf_id": "SI45537", "req_type": "PRE" }, { "ptf_id": "SI57276", "req_type": "DIST" }, { "ptf_id": "SI57305", "req_type": "DIST" } ] }, { "ptf_id": "SI71691", "req_list": [ { "ptf_id": "SI70931", "req_type": "PRE" } ] } ], "rc": 0, "start": "2023-06-09 17:04:34.160616", "stderr": "", "stderr_lines": [], "timeout": 30 } }

=================================================

And it still works with "validate_certs=False". ==================================================== TASK [display_two_ptf] ** task path: /home/xxxxxx/ibmi-fix-check.yml:23 ok: [127.0.0.1] => { "msg": { "certs": false, "changed": false, "count": 0, "elapsed_time": "0:00:03.880353", "end": "2023-06-09 17:06:52.490738", "expanded_requisites": false, "failed": false, "group_info": [], "http_agent": "ansible/ibm.power_ibmi", "ptf_info": [ { "ptf_id": "SI71691", "req_list": [ { "ptf_id": "SI70931", "req_type": "PRE" } ] }, { "ptf_id": "SI74112", "req_list": [ { "ptf_id": "SI45537", "req_type": "PRE" }, { "ptf_id": "SI57276", "req_type": "DIST" }, { "ptf_id": "SI57305", "req_type": "DIST" } ] } ], "rc": 0, "start": "2023-06-09 17:06:48.610385", "stderr": "", "stderr_lines": [], "timeout": 30 } }

{ End }

robgjertsen1 commented 1 year ago

Thanks for digging into this. Will close now that you opened an OSS issue to track getting an updated ansible version on IBM i.