vmware_cluster_vsan: vsan_auto_claim_storage does not work on 7.0U2

lvlie commented 3 years ago

SUMMARY

This issue is a continuation of https://github.com/ansible-collections/community.vmware/issues/728, which was closed while the automatic claiming of disks for vsan is not working.

Claiming disks using the "Claim unused disk" option in the GUI works like a charm (with no changes to the defaults)

ISSUE TYPE

Bug Report

COMPONENT NAME

community.vmware.vmware_cluster_vsan plugins/modules/vmware_cluster_vsan.py

ANSIBLE VERSION

# ansible --version
ansible 2.10.5
# ansible-galaxy collection list                    

Collection            Version
--------------------- -------
community.general     2.4.0  
community.vmware      1.9.0

CONFIGURATION

DEFAULT_HOST_LIST(**snap**/ansible.cfg) = ['**snap**/ansible/inventory.yaml']
DEFAULT_MODULE_PATH(**snap**/ansible.cfg) = ['**snap**/ansible/library']
DEFAULT_MODULE_UTILS_PATH(**snap**/ansible.cfg) = ['**snap**/ansible/module_utils']
INTERPRETER_PYTHON(**snap**/ansible.cfg) = /usr/bin/python3
LOCALHOST_WARNING(**snap**/ansible.cfg) = False

OS / ENVIRONMENT

vCenter 7.0.2 with nested esxi nodes

STEPS TO REPRODUCE

I'm using nested esxi nodes, with 2 extra disks; one small (10Gb) and one slightly bigger (40Gb). On the ESX shell I've marked one disk as flash via: storage nmp satp rule add -s VMW_SATP_LOCAL -d mpx.vmhba0:C0:T1:L0 -o enable_ssd

I then run the following task:

- name: Configure cluster vSAN
  community.vmware.vmware_cluster_vsan:
    hostname: "{{ host }}"
    username: "{{ user }}"
    password: "{{ pass }}"
    validate_certs: "{{ vcenter.validate_certs }}"
    datacenter_name: "{{ vcenter.datacenter }}"
    cluster_name: "{{ cluster.name }}"
    enable_vsan: True
    vsan_auto_claim_storage: True
    advanced_options:
      automatic_rebalance: True

EXPECTED RESULTS

vSAN disk is created with capacity (disks auto claimed)

ACTUAL RESULTS

vSAN disk is created with no capacity.

ansibullbot commented 3 years ago

Files identified in the description: None

If these files are inaccurate, please update the component name section of the description or use the !component bot command.

click here for bot help

ansibullbot commented 3 years ago

Files identified in the description:

[plugins/modules/vmware_cluster_vsan.py](https://github.com/['ansible-collections/amazon.aws', 'ansible-collections/community.aws', 'ansible-collections/community.vmware']/blob/main/plugins/modules/vmware_cluster_vsan.py)

If these files are inaccurate, please update the component name section of the description or use the !component bot command.

click here for bot help

ansibullbot commented 3 years ago

cc @Akasurde @Tomorrow9 @goneri @lparkes @mariolenz @pgbidkar @warthog9 click here for bot help

Akasurde commented 3 years ago

@lvlie Thanks for reporting this issue. Issue #728 was closed since the PR https://github.com/ansible-collections/community.vmware/pull/734 merged. Could you please check the existing version of the VMware collection and update it to the latest i.e. 1.9.0 and let me know?

Thanks,

needs_info

Akasurde commented 3 years ago

@mariolenz @sky-joker Could you please check if the issue still persists? Thanks in advance.

lvlie commented 3 years ago

@Akasurde yes it was closed but 2 of the findings reported there were not actually fixed yet. I am using v1.9.0 of the collection indeed.

Akasurde commented 3 years ago

@lvlie Thanks for the information. I pinged concerned persons.

mzlumin commented 3 years ago

I have had the same issue here lately, vsan get enabled, as well as an empty disk group created. However no disks are claimed.

Running collection 1.9.0

Additionally here are some more Infos:

Playbook

    - name: Enable vSAN and claim storage automatically
      community.vmware.vmware_cluster_vsan:
        hostname: '{{ vcenter_hostname }}.{{ domain }}'
        username: "{{ vcenter_username }}"
        password: "{{ vcenter_password }}"
        validate_certs: no
        datacenter_name: "{{ datacenter_name }}"
        cluster_name: "{{ cluster_name_compute }}"
        enable_vsan: True
        #ignore_errors: true
        vsan_auto_claim_storage: True
        advanced_options:
          automatic_rebalance: True
      delegate_to: localhost

Output:

TASK [Enable vSAN and claim storage automatically] **********************************************************************************************************************************************************
task path: /home/mzulmin/lab-images/ANSIBLE-vSphere-VCSA-OVA-Deploy/playbooks/04-01-vsan.yml:22
<localhost> ESTABLISH LOCAL CONNECTION FOR USER: mzulmin
<localhost> EXEC /bin/sh -c 'echo ~mzulmin && sleep 0'
<localhost> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo /home/mzulmin/.ansible/tmp `"&& mkdir "` echo /home/mzulmin/.ansible/tmp/ansible-tmp-1621946207.4411027-25685-102386453699711 `" && echo ansible-tmp-1621946207.4411027-25685-102386453699711="` echo /home/mzulmin/.ansible/tmp/ansible-tmp-1621946207.4411027-25685-102386453699711 `" ) && sleep 0'
Including module_utils file ansible/__init__.py
Including module_utils file ansible/module_utils/__init__.py
Including module_utils file ansible/module_utils/_text.py
Including module_utils file ansible/module_utils/basic.py
Including module_utils file ansible/module_utils/common/_collections_compat.py
Including module_utils file ansible/module_utils/common/__init__.py
Including module_utils file ansible/module_utils/common/_json_compat.py
Including module_utils file ansible/module_utils/common/_utils.py
Including module_utils file ansible/module_utils/common/arg_spec.py
Including module_utils file ansible/module_utils/common/file.py
Including module_utils file ansible/module_utils/common/parameters.py
Including module_utils file ansible/module_utils/common/collections.py
Including module_utils file ansible/module_utils/common/process.py
Including module_utils file ansible/module_utils/common/sys_info.py
Including module_utils file ansible/module_utils/common/text/converters.py
Including module_utils file ansible/module_utils/common/text/__init__.py
Including module_utils file ansible/module_utils/common/text/formatters.py
Including module_utils file ansible/module_utils/common/validation.py
Including module_utils file ansible/module_utils/common/warnings.py
Including module_utils file ansible/module_utils/compat/selectors.py
Including module_utils file ansible/module_utils/compat/__init__.py
Including module_utils file ansible/module_utils/compat/_selectors2.py
Including module_utils file ansible/module_utils/compat/selinux.py
Including module_utils file ansible/module_utils/distro/__init__.py
Including module_utils file ansible/module_utils/distro/_distro.py
Including module_utils file ansible/module_utils/errors.py
Including module_utils file ansible/module_utils/parsing/convert_bool.py
Including module_utils file ansible/module_utils/parsing/__init__.py
Including module_utils file ansible/module_utils/pycompat24.py
Including module_utils file ansible/module_utils/six/__init__.py
Including module_utils file ansible_collections/community/vmware/plugins/module_utils/vmware.py
Including module_utils file ansible_collections/__init__.py
Including module_utils file ansible_collections/community/__init__.py
Including module_utils file ansible_collections/community/vmware/__init__.py
Including module_utils file ansible_collections/community/vmware/plugins/__init__.py
Including module_utils file ansible_collections/community/vmware/plugins/module_utils/__init__.py
Using module file /home/mzulmin/.ansible/collections/ansible_collections/community/vmware/plugins/modules/vmware_cluster_vsan.py
<localhost> PUT /home/mzulmin/.ansible/tmp/ansible-local-24348vgeb3_nz/tmpt77ovyfb TO /home/mzulmin/.ansible/tmp/ansible-tmp-1621946207.4411027-25685-102386453699711/AnsiballZ_vmware_cluster_vsan.py
<localhost> EXEC /bin/sh -c 'chmod u+x /home/mzulmin/.ansible/tmp/ansible-tmp-1621946207.4411027-25685-102386453699711/ /home/mzulmin/.ansible/tmp/ansible-tmp-1621946207.4411027-25685-102386453699711/AnsiballZ_vmware_cluster_vsan.py && sleep 0'
<localhost> EXEC /bin/sh -c '/usr/bin/python3 /home/mzulmin/.ansible/tmp/ansible-tmp-1621946207.4411027-25685-102386453699711/AnsiballZ_vmware_cluster_vsan.py && sleep 0'
<localhost> EXEC /bin/sh -c 'rm -f -r /home/mzulmin/.ansible/tmp/ansible-tmp-1621946207.4411027-25685-102386453699711/ > /dev/null 2>&1 && sleep 0'
changed: [localhost] => {
    "changed": true,
    "invocation": {
        "module_args": {
            "advanced_options": {
                "automatic_rebalance": true,
                "disable_site_read_locality": null,
                "large_cluster_support": null,
                "object_repair_timer": null,
                "thin_swap": null
            },
            "cluster_name": "Compute",
            "datacenter": "ATX-01",
            "datacenter_name": "ATX-01",
            "enable": true,
            "enable_vsan": true,
            "hostname": "srv-vcenter-01.megasp.net",
            "password": "VALUE_SPECIFIED_IN_NO_LOG_PARAMETER",
            "port": 443,
            "proxy_host": null,
            "proxy_port": null,
            "username": "administrator@megasp.net",
            "validate_certs": false,
            "vsan_auto_claim_storage": true
        }
    },
    "result": null
}

❯ ansible --version
ansible [core 2.11.0] 
  config file = /etc/ansible/ansible.cfg
  configured module search path = ['/home/mzulmin/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/local/lib/python3.8/dist-packages/ansible
  ansible collection location = /home/mzulmin/.ansible/collections:/usr/share/ansible/collections
  executable location = /usr/local/bin/ansible
  python version = 3.8.5 (default, Jan 27 2021, 15:41:15) [GCC 9.3.0]
  jinja version = 2.11.3
  libyaml = True

I am running a 3 Node Cluster with 1xSSD and 1xHDD no other Disks are used. Nested Esxi running on 7.0.2 Build: 17630552

vSAN Management SDK for Python | 7.0 U2 directly from VMware.

snowsnoot commented 2 years ago

Still broken on the below versions:

ESXi 7.0U3-20036589 VCSA 7.0U3-19480866 community.vmware ansible collection v2.8.0 vsan-sdk-python 7.0U3-18502530

luischanu commented 2 years ago

@lvlie and @snowsnoot, this may be of help to you. I am a co-developer of SDDC.Lab, and we ran into issues with vSAN when v7.0.2 was released. After a bit of digging, the problem appeared to be (for us anyway) an issue around the setting of the drive type. We could find no solution to this at the time, so I ended up writing an Ansible module to do that, which has been working flawlessly for us in our project. So, I thought I'd just mention it to both of you, as it may help you in your situation.

The file I wrote is called "set_drive_type.py". Within the module, you'll find numerous examples of how the module can be used. The file can be found HERE. We will soon be renaming that branch to "v5" once the development branch is released, of it the URL fails, just go to the default branch, and look in the "library" folder.

We have two other vSAN related modules as well, which are in the same directory, which might be of assistance to you as well. So, check those out, too.

I hope this helps you.

WARNING: This is my first (and only) Ansible module that I have written, so I don't claim it to be the most efficient method, but as mentioned above, it works. :-)

snowsnoot commented 2 years ago

We have two other vSAN related modules as well, which are in the same directory, which might be of assistance to you as well. So, check those out, too.

Hey Luis, thanks for the reply! Actually, the workaround we found was to use your modules from SDDC.Lab and it works great, we're using enable_vsan and claim_vsan_disks modules instead of the community versions. Could I suggest you generate a PR for the community modules and fix them? I'm haven't yet looked myself to see what changes might be required, but I think it would be a great way to contribute back to the Ansible community.

ansible-collections / community.vmware