search

Pure-Storage-Ansible / FlashArray-Collection

Ansible Collection for Pure Storage FlashArray
GNU General Public License v3.0
19 stars 21 forks source link

Module purefa_info error #443

Closed colinet closed 10 months ago

colinet commented 10 months ago

Describe the bug Running below playbook triggers an internal error:

- name: "pure | volumegroup info | collect information"
  purestorage.flasharray.purefa_info:
    fa_url: "{{ url }}"
    api_token: "{{ token }}"
    gather_subset:
      - volumes
  register: _volume_infos

The error reported is: 

File \"/home/xxx/.ansible/tmp/ansible-tmp-1694095155.2924778-663760-240555658022964/AnsiballZ_purefa_info.py\", line 107, in <module>\n    _ansiballz_main()\n  
File \"/home/xxx/.ansible/tmp/ansible-tmp-1694095155.2924778-663760-240555658022964/AnsiballZ_purefa_info.py\", line 99, in _ansiballz_main\n    invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)\n 
File \"/home/xxx/.ansible/tmp/ansible-tmp-1694095155.2924778-663760-240555658022964/AnsiballZ_purefa_info.py\", line 47, in invoke_module\n    runpy.run_module(mod_name='ansible_collections.purestorage.flasharray.plugins.modules.purefa_info', init_globals=dict(_module_fqn='ansible_collections.purestorage.flasharray.plugins.modules.purefa_info', _modlib_path=modlib_path),\n  
File \"/usr/lib64/python3.9/runpy.py\", line 225, in run_module\n    return _run_module_code(code, init_globals, run_name, mod_spec)\n  
File \"/usr/lib64/python3.9/runpy.py\", line 97, in _run_module_code\n    _run_code(code, mod_globals, init_globals,\n  
File \"/usr/lib64/python3.9/runpy.py\", line 87, in _run_code\n    exec(code, run_globals)\n  
File \"/tmp/ansible_purestorage.flasharray.purefa_info_payload_wn6_ee1u/ansible_purestorage.flasharray.purefa_info_payload.zip/ansible_collections/purestorage/flasharray/plugins/modules/purefa_info.py\", line 2363, in <module>\n  
File \"/tmp/ansible_purestorage.flasharray.purefa_info_payload_wn6_ee1u/ansible_purestorage.flasharray.purefa_info_payload.zip/ansible_collections/purestorage/flasharray/plugins/modules/purefa_info.py\", line 2290, in main\n  
File \"/tmp/ansible_purestorage.flasharray.purefa_info_payload_wn6_ee1u/ansible_purestorage.flasharray.purefa_info_payload.zip/ansible_collections/purestorage/flasharray/plugins/modules/purefa_info.py\", line 1158, in generate_vol_dict\n 
File \"/usr/local/lib/python3.9/site-packages/pypureclient/flasharray/FA_2_6/models/space.py\", line 140, in __getattribute__\n    value = object.__getattribute__(self, item)\nAttributeError: 'Space' object has no attribute 'snapshots_effective'\n", "module_stdout": "", "msg": "MODULE FAILURE\nSee stdout/stderr for the exact error", "rc": 1}

To Reproduce Run above playbook.

Expected behavior List of volumes of the Pure array

Desktop (please complete the following information): Ansible version: 

[xxxxxxxx@xxxxxxxxxxx~]$  ansible --version
ansible [core 2.15.0]
  config file = /etc/ansible/ansible.cfg
  configured module search path = ['/home/xxxxxxx/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/local/lib/python3.9/site-packages/ansible
  ansible collection location = /home/xxxxxxxxx/.ansible/collections:/usr/share/ansible/collections
  executable location = /usr/local/bin/ansible
  python version = 3.9.16 (main, May 29 2023, 00:00:00) [GCC 11.3.1 20221121 (Red Hat 11.3.1-4)] (/usr/bin/python)
  jinja version = 3.1.2
  libyaml = True
[xxxxxxxx@xxxxxxxxxx~]$

Ansible Pure collection:

[xxxxxx@xxxxxxxxxxx~]$  ansible-galaxy collection list | grep -i pure
purestorage.flasharray        1.21.0 
purestorage.flasharray        1.18.0 
purestorage.flashblade        1.11.0 
purestorage.fusion            1.4.2  
[xxxxxxx@xxxxxxxxxx~]$
sdodsley commented 10 months ago

What version of Purity are you running?

colinet commented 10 months ago

I did a test with a Pure array with Purity: 6.4.8

sdodsley commented 10 months ago

OK - let me look at this. I think this might be part of an API change they recently brought in for EUC

sdodsley commented 10 months ago

@colinet please try the associated patch to see if this resolves your bug

colinet commented 10 months ago

I applied: 

--- purefa_info.py.orig 2023-09-07 17:17:48.891797545 +0200
+++ .ansible/collections/ansible_collections/purestorage/flasharray/plugins/modules/purefa_info.py      2023-09-07 17:20:31.789030666 +0200
@@ -893,9 +893,9 @@
             ].space.total_provisioned
             snap_info[snapshot]["unique_space"] = snapsv6[snap].space.unique
             if SHARED_CAP_API_VERSION in api_version:
-                snap_info[snapshot]["snapshots_effective"] = snapsv6[
-                    snap
-                ].space.snapshots_effective
+                snap_info[snapshot]["snapshots_effective"] = getattr(
+                    snapsv6[snap].space, "snapshots_effective", None
+                )
         offloads = list(arrayv6.get_offloads().items)
         for offload in range(0, len(offloads)):
             offload_name = offloads[offload].name
@@ -1076,12 +1076,12 @@
                 vol
             ].space.thin_provisioning
             if SHARED_CAP_API_VERSION in api_version:
-                volume_info[name]["snapshots_effective"] = vols_space[
-                    vol
-                ].space.snapshots_effective
-                volume_info[name]["unique_effective"] = vols_space[
-                    vol
-                ].space.unique_effective
+                volume_info[name]["snapshots_effective"] = getattr(
+                    vols_space[vol].space, "snapshots_effective", None
+                )
+                volume_info[name]["unique_effective"] = getattr(
+                    vols_space[vol].space, "unique_effective", None
+                )
                 volume_info[name]["used_provisioned"] = (
                     getattr(vols_space[vol].space, "used_provisioned", None),
                 )
@@ -1155,15 +1155,15 @@
                 vol
             ].space.total_physical
             if SHARED_CAP_API_VERSION in api_version:
-                volume_info[name]["snapshots_effective"] = vols_space[
-                    vol
-                ].space.snapshots_effective
-                volume_info[name]["unique_effective"] = vols_space[
-                    vol
-                ].space.unique_effective
-                volume_info[name]["total_effective"] = vols_space[
-                    vol
-                ].space.total_effective
+                volume_info[name]["snapshots_effective"] = getattr(
+                    vols_space[vol].space, "snapshots_effective", None
+                )
+                volume_info[name]["unique_effective"] = getattr(
+                    vols_space[vol].space, "unique_effective", None
+                )
+                volume_info[name]["total_effective"] = getattr(
+                    vols_space[vol].space, "total_effective", None
+                )
                 volume_info[name]["used_provisioned"] = (
                     getattr(vols_space[vol].space, "used_provisioned", None),
                 )
@@ -2361,3 +2361,4 @@

 if __name__ == "__main__":
     main()
+

But I still get: 

File \"/home/xxxxxxxxx/.ansible/tmp/ansible-tmp-1694100070.859686-665212-193894656258454/AnsiballZ_purefa_info.py\", line 107, in <module>\n    _ansiballz_main()\n  
File \"/home/xxxxxxxxx/.ansible/tmp/ansible-tmp-1694100070.859686-665212-193894656258454/AnsiballZ_purefa_info.py\", line 99, in _ansiballz_main\n    invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)\n  
File \"/home/xxxxxxxxx/.ansible/tmp/ansible-tmp-1694100070.859686-665212-193894656258454/AnsiballZ_purefa_info.py\", line 47, in invoke_module\n    runpy.run_module(mod_name='ansible_collections.purestorage.flasharray.plugins.modules.purefa_info', init_globals=dict(_module_fqn='ansible_collections.purestorage.flasharray.plugins.modules.purefa_info', _modlib_path=modlib_path),\n  
File \"/usr/lib64/python3.9/runpy.py\", line 225, in run_module\n    return _run_module_code(code, init_globals, run_name, mod_spec)\n  
File \"/usr/lib64/python3.9/runpy.py\", line 97, in _run_module_code\n    _run_code(code, mod_globals, init_globals,\n 
File \"/usr/lib64/python3.9/runpy.py\", line 87, in _run_code\n    exec(code, run_globals)\n  
File \"/tmp/ansible_purestorage.flasharray.purefa_info_payload_jm7t3_fu/ansible_purestorage.flasharray.purefa_info_payload.zip/ansible_collections/purestorage/flasharray/plugins/modules/purefa_info.py\", line 2363, in <module>\n 
File \"/tmp/ansible_purestorage.flasharray.purefa_info_payload_jm7t3_fu/ansible_purestorage.flasharray.purefa_info_payload.zip/ansible_collections/purestorage/flasharray/plugins/modules/purefa_info.py\", line 2290, in main\n  
File \"/tmp/ansible_purestorage.flasharray.purefa_info_payload_jm7t3_fu/ansible_purestorage.flasharray.purefa_info_payload.zip/ansible_collections/purestorage/flasharray/plugins/modules/purefa_info.py\", line 1212, in generate_vol_dict\n  
File \"/usr/local/lib/python3.9/site-packages/pypureclient/flasharray/FA_2_6/models/volume.py\", line 159, in __getattribute__\n    value = object.__getattribute__(self, item)\nAttributeError: 'Volume' object has no attribute 'priority'\n", "module_stdout": "", "msg": "MODULE FAILURE\nSee stdout/stderr for the exact error", "rc": 1}

The error points to an other attribute now: 'priority'

sdodsley commented 10 months ago

OK - it appears you also have 2 versions of the purestorage.flasharray collection. Please uninstall all versions and then reinstall only the latest version. Also, can you confirm the version of pypureclient you have installed

sdodsley commented 10 months ago

I think you are running an old version of py-pure-client. You should always upgrade to the latest version when ever a new Pure Ansible Collection is released. It only tests against the latest versions.

colinet commented 10 months ago

Hello,

I upgraded py-pure-client:

[xxxxxxx@xxxxxxxxxxxx~]$  pip list | grep pure
purestorage                        1.19.0
py-pure-client                     1.39.0

But I'am getting the following error (different context as previously because I had to switch to a higher prioriry work):

File \"/home/xxxxxx/.ansible/tmp/ansible-tmp-1694178621.9399414-728762-192146842038973/AnsiballZ_purefa_info.py\", line 107, in <module>\n    _ansiballz_main()\n  
File \"/home/xxxxxxx/.ansible/tmp/ansible-tmp-1694178621.9399414-728762-192146842038973/AnsiballZ_purefa_info.py\", line 99, in _ansiballz_main\n    invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)\n  
File \"/home/xxxxxx/.ansible/tmp/ansible-tmp-1694178621.9399414-728762-192146842038973/AnsiballZ_purefa_info.py\", line 47, in invoke_module\n    runpy.run_module(mod_name='ansible_collections.purestorage.flasharray.plugins.modules.purefa_info', init_globals=dict(_module_fqn='ansible_collections.purestorage.flasharray.plugins.modules.purefa_info', _modlib_path=modlib_path),\n  
File \"/usr/lib64/python3.9/runpy.py\", line 225, in run_module\n    return _run_module_code(code, init_globals, run_name, mod_spec)\n  
File \"/usr/lib64/python3.9/runpy.py\", line 97, in _run_module_code\n    _run_code(code, mod_globals, init_globals,\n 
File \"/usr/lib64/python3.9/runpy.py\", line 87, in _run_code\n    exec(code, run_globals)\n  
File \"/tmp/ansible_purestorage.flasharray.purefa_info_payload_91m_s165/ansible_purestorage.flasharray.purefa_info_payload.zip/ansible_collections/purestorage/flasharray/plugins/modules/purefa_info.py\", line 2363, in <module>\n  
File \"/tmp/ansible_purestorage.flasharray.purefa_info_payload_91m_s165/ansible_purestorage.flasharray.purefa_info_payload.zip/ansible_collections/purestorage/flasharray/plugins/modules/purefa_info.py\", line 2296, in main\n  
File \"/tmp/ansible_purestorage.flasharray.purefa_info_payload_91m_s165/ansible_purestorage.flasharray.purefa_info_payload.zip/ansible_collections/purestorage/flasharray/plugins/modules/purefa_info.py\", line 2080, in generate_hgroups_dict\nKeyError: 'ABVPURMGT005SAS:IT1-ABVPUR005-HG'\n", "module_stdout": "", "msg": "MODULE FAILURE\nSee stdout/stderr for the exact error", "rc": 1}
colinet commented 10 months ago

Oops, just saw your note about multiple flasharray module...

colinet commented 10 months ago

I removed my /home/xxxxxxx/.ansible folder. Then, there was then only one version of purestorage.flasharray: 1.18.0 Then, I upgraded the Anisble Pure collection with : ansible-galaxy collection install -U purestorage.flasharray 

[xxxxxxxxx@xxxxxxxxx~]$  ansible-galaxy collection list | grep -i pure
purestorage.flasharray        1.21.0 
purestorage.flasharray        1.18.0 
purestorage.flashblade        1.11.0 
purestorage.fusion            1.4.2  
[rcolinet@abvacmrct201sas ~]$

But I still have the error.

The FW of the array is : 6.4.8 (Fails as show above) On some arrays, it works with : 6.3.12 (Works as expected)

Shall I apply a patch ?

colinet commented 10 months ago

@sdodsley , Any update/advice on how to fix or work around this issue?

sdodsley commented 10 months ago

@colinet sorry for the delay - please try the listed patch and let me know if that resolves the issue

colinet commented 10 months ago

@sdodsley Should I apply the 2 patches below:

colinet commented 10 months ago

I've reinstalled Ansible Pure collection for flash array: # ansible-galaxy collection install -U purestorage.flasharray

I applied both 444 and 446 patches.

My setting is : 

[xxxxxxx@xxxxxxx~]$   ansible --version
ansible [core 2.15.0]
  config file = /etc/ansible/ansible.cfg
  configured module search path = ['/home/xxxxxxx/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/local/lib/python3.9/site-packages/ansible
  ansible collection location = /home/xxxxxxx/.ansible/collections:/usr/share/ansible/collections
  executable location = /usr/local/bin/ansible
  python version = 3.9.16 (main, May 29 2023, 00:00:00) [GCC 11.3.1 20221121 (Red Hat 11.3.1-4)] (/usr/bin/python)
  jinja version = 3.1.2
  libyaml = True
[xxxxxxx@xxxxxxx~]$

When I run below play:

- name: pure | helpers check | collect data
  purestorage.flasharray.purefa_info:
    fa_url: "{{ san_CRUD_purearray_name[_pure_check_array_name|lower].url }}"
    api_token: "{{ san_CRUD_purearray_name[_pure_check_array_name|lower].token }}"
    gather_subset:
      #- hgroups
      - hosts
      # - volumes
      # - vgroups
      # - pgroups
  register: _pure_info_content

It fails as show below:

fatal: [localhost]: FAILED! => {"changed": false, "module_stderr": 
"/home/rcolinet/.local/lib/python3.9/site-packages/urllib3/connectionpool.py:1045: InsecureRequestWarning: Unverified HTTPS request is being made to host '10.81.0.134'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings\n
 warnings.warn(\n/home/rcolinet/.local/lib/python3.9/site-packages/urllib3/connectionpool.py:1045: InsecureRequestWarning: Unverified HTTPS request is being made to host '10.81.0.134'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings\n  
warnings.warn(\n/home/rcolinet/.local/lib/python3.9/site-packages/urllib3/connectionpool.py:1045: InsecureRequestWarning: Unverified HTTPS request is being made to host '10.81.0.134'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings\n  
warnings.warn(\n/home/rcolinet/.local/lib/python3.9/site-packages/urllib3/connectionpool.py:1045: InsecureRequestWarning: Unverified HTTPS request is being made to host '10.81.0.134'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings\n  
warnings.warn(\n/home/rcolinet/.local/lib/python3.9/site-packages/urllib3/connectionpool.py:1045: InsecureRequestWarning: Unverified HTTPS request is being made to host '10.81.0.134'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings\n  
warnings.warn(\n/home/rcolinet/.local/lib/python3.9/site-packages/urllib3/connectionpool.py:1045: InsecureRequestWarning: Unverified HTTPS request is being made to host '10.81.0.134'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings\n  warnings.warn(\nTraceback (most recent call last):\n  
File \"/home/rcolinet/.ansible/tmp/ansible-tmp-1695114302.6446044-1127162-27861570072320/AnsiballZ_purefa_info.py\", line 107, in <module>\n    _ansiballz_main()\n  
File \"/home/xxxxxxxxx/.ansible/tmp/ansible-tmp-1695114302.6446044-1127162-27861570072320/AnsiballZ_purefa_info.py\", line 99, in _ansiballz_main\n    invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)\n  
File \"/home/rcolinet/.ansible/tmp/ansible-tmp-1695114302.6446044-1127162-27861570072320/AnsiballZ_purefa_info.py\", line 47, in invoke_module\n    runpy.run_module(mod_name='ansible_collections.purestorage.flasharray.plugins.modules.purefa_info', init_globals=dict(_module_fqn='ansible_collections.purestorage.flasharray.plugins.modules.purefa_info', _modlib_path=modlib_path),\n  
File \"/usr/lib64/python3.9/runpy.py\", line 225, in run_module\n    return _run_module_code(code, init_globals, run_name, mod_spec)\n  
File \"/usr/lib64/python3.9/runpy.py\", line 97, in _run_module_code\n    _run_code(code, mod_globals, init_globals,\n  
File \"/usr/lib64/python3.9/runpy.py\", line 87, in _run_code\n    exec(code, run_globals)\n  
File\"/tmp/ansible_purestorage.flasharray.purefa_info_payload_sl1wbfe4/ansible_purestorage.flasharray.purefa_info_payload.zip/ansible_collections/purestorage/flasharray/plugins/modules/purefa_info.py\", line 2364, in <module>\n  
File\"/tmp/ansible_purestorage.flasharray.purefa_info_payload_sl1wbfe4/ansible_purestorage.flasharray.purefa_info_payload.zip/ansible_collections/purestorage/flasharray/plugins/modules/purefa_info.py\", line 2289, in main\n  
File\"/tmp/ansible_purestorage.flasharray.purefa_info_payload_sl1wbfe4/ansible_purestorage.flasharray.purefa_info_payload.zip/ansible_collections/purestorage/flasharray/plugins/modules/purefa_info.py\", line 1289, in generate_host_dict\nKeyError: 'ABVPURMGT005SAS:abvisql06sas'\n", "module_stdout": "", "msg": "MODULE FAILURE\nSee stdout/stderr for the exact error", "rc": 1}

There is a key error in generate_host_dict.

For other subsets (hgroups, volumes, vgroups, pgroups) this works.

sdodsley commented 10 months ago

Got it. Will merge #446 and create a new one for the host_dict issue

sdodsley commented 10 months ago

@colinet please try applying #447

colinet commented 10 months ago

With patch 447, it works!

When will it be officially available ?

sdodsley commented 10 months ago

Not sure when the next release is scheduled. I need to check the Purity//FA release schedule and sync with that

colinet commented 10 months ago

Thx