Trying to break the snapmirror throws error if primary cluster is not available

[christian-naenny]

Summary

I'm trying to break a snapmirror relationship where the primary cluster is not available. The situation is a disaster recovery exercise where we completely isolate the second datacenter (SAN and IP networks). Our DR cluster contains the DR SVMs. From the perspective of the DR cluster, the primary cluster is now completely unavailable. Upon breaking the snapmirror relationship the error Error retrieving SVM peer: calling: svm/peers: got HTTPSConnectionPool(host='clustername', port=443): Read timed out. (read timeout=60). is thrown.

Component Name

na_ontap_snapmirror

Ansible Version

$ ansible --version
ansible [core 2.15.9]
  config file = /etc/ansible/ansible.cfg
  configured module search path = ['/home/chn/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /opt/takeover/venvs/to_exec_py39_ans215/lib64/python3.9/site-packages/ansible
  ansible collection location = /opt/takeover/collections:/opt/takeover/venvs
  executable location = /opt/takeover/venvs/to_exec_py39_ans215/bin/ansible
  python version = 3.9.18 (main, Sep 22 2023, 17:58:34) [GCC 8.5.0 20210514 (Red Hat 8.5.0-20)] (/opt/takeover/venvs/to_exec_py39_ans215/bin/python3.9)
  jinja version = 3.1.3
  libyaml = True

ONTAP Collection Version

$ ansible-galaxy collection list
netapp.aws                    21.7.1
netapp.azure                  21.10.1
netapp.cloudmanager           21.22.1
netapp.elementsw              21.7.0
netapp.ontap                  22.8.3
netapp.storagegrid            21.11.1
netapp.um_info                21.8.1
netapp_eseries.santricity     1.4.0

ONTAP Version

NetApp Release 9.12.1P10: Wed Dec 20 00:24:59 UTC 2023

Playbook

- name: "snapmirror | Change the snapmirror state to {{ state }}"
  netapp.ontap.na_ontap_snapmirror:
    state: "present"
    relationship_state: "{{ state }}"
    source_endpoint:
      path: "svm:"
    destination_endpoint:
      path: "dr-svm:"
    hostname: "clustername"
    username: "hidden"
    password: "hidden"
    validate_certs: false
    use_rest: always
    validate_source_path: false
  when: not ansible_check_mode

Steps to Reproduce

ansible-playbook snapmirror.yml --extra-vars '{ "state": "broken" }' --extra-vars '{ "svm": "svm" }'

Expected Results

I simply expected the snapmirror state to end up like this:

Source            Destination Mirror  Relationship   Total             Last
Path        Type  Path        State   Status         Progress  Healthy Updated
----------- ---- ------------ ------- -------------- --------- ------- --------
svm:       XDP  dr-svm:
                              Broken-off
                                      Idle           -         true    -

Actual Results

TASK [snapmirror | Change the snapmirror state to broken] **********************
task path: /opt/takeover/execution/playbooks/svm/snapmirror.yml:46
<127.0.0.1> ESTABLISH LOCAL CONNECTION FOR USER: chn
<127.0.0.1> EXEC /bin/sh -c 'echo ~chn && sleep 0'
<127.0.0.1> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo /home/chn/.ansible/tmp `"&& mkdir "` echo /home/chn/.ansible/tmp/ansible-tmp-1711181819.8846548-929328-151196137967511 `" && echo ansible-tmp-          1711181819.8846548-929328-151196137967511="` echo /home/chn/.ansible/tmp/ansible-tmp-1711181819.8846548-929328-151196137967511 `" ) && sleep 0'
Using module file /opt/takeover/venvs/to_exec_py39_ans215/lib64/python3.9/site-packages/ansible_collections/netapp/ontap/plugins/modules/na_ontap_snapmirror.py
<127.0.0.1> PUT /home/chn/.ansible/tmp/ansible-local-929303ywb8lc8r/tmpr97ivl2_ TO /home/chn/.ansible/tmp/ansible-tmp-1711181819.8846548-929328-151196137967511/AnsiballZ_na_ontap_snapmirror.py
<127.0.0.1> EXEC /bin/sh -c 'chmod u+x /home/chn/.ansible/tmp/ansible-tmp-1711181819.8846548-929328-151196137967511/ /home/chn/.ansible/tmp/ansible-tmp-1711181819.8846548-929328-151196137967511/                  AnsiballZ_na_ontap_snapmirror.py && sleep 0'
<127.0.0.1> EXEC /bin/sh -c '/opt/takeover/venvs/to_exec_py39_ans215/bin/python3.9 /home/chn/.ansible/tmp/ansible-tmp-1711181819.8846548-929328-151196137967511/AnsiballZ_na_ontap_snapmirror.py && sleep 0'
<127.0.0.1> EXEC /bin/sh -c 'rm -f -r /home/chn/.ansible/tmp/ansible-tmp-1711181819.8846548-929328-151196
2024-03-23 09:18:00,468 INFO  takeover_menu.py:657   update_text - 137967511/ > /dev/null 2>&1 && sleep 0'
fatal: [localhost]: FAILED! => {
    "changed": false,
    "invocation": {
        "module_args": {
            "cert_filepath": null,
            "clean_up_failure": false,
            "connection_type": "ontap_ontap",
            "create_destination": null,
            "destination_cluster": null,
            "destination_endpoint": {
                "cluster": null,
                "consistency_group_volumes": null,
                "ipspace": null,
                "path": "dr-svm:",
                "svm": null
            },
            "destination_path": null,
            "destination_volume": null,
            "destination_vserver": null,
            "feature_flags": null,
            "force_ontap_version": null,
            "hostname": "clustername",
            "http_port": null,
            "https": false,
            "identity_preservation": null,
            "identity_preserve": null,
            "initialize": true,
            "key_filepath": null,
            "max_transfer_rate": null,
            "ontapi": null,
            "password": "VALUE_SPECIFIED_IN_NO_LOG_PARAMETER",
            "peer_options": null,
            "policy": null,
            "relationship_info_only": false,
            "relationship_state": "broken",
            "relationship_type": null,
            "schedule": null,
            "source_cluster": null,
            "source_endpoint": {
                "cluster": null,
                "consistency_group_volumes": null,
                "ipspace": null,
                "path": "svm:",
                "svm": null
            },
            "source_hostname": null,
            "source_password": null,
            "source_path": null,
            "source_snapshot": null,
            "source_username": null,
            "source_volume": null,
            "source_vserver": null,
            "state": "present",
            "transferring_time_out": 300,
            "update": true,
            "use_rest": "always",
            "username": "ansitako",
            "validate_certs": false,
            "validate_source_path": true
        }
    },
    "msg": "Error retrieving SVM peer: calling: svm/peers: got HTTPSConnectionPool(host='clustername', port=443): Read timed out. (read timeout=60)."
}

[suhasbshekar]

Hi,

In your playbook, since you have added "validate_source_path": true, which will check if another relationship exists as svm name for peer can be aliased in the source. In snapmirror resource, it is trying find out the svm peer relationship if there is an alias of the destination svm.

Hence, you are getting the error Error retrieving SVM peer: calling: svm/peers: got HTTPSConnectionPool(host='clustername', port=443): Read timed out. (read timeout=60).

We are still debugging to see if error is appropriate here, meanwhile can you run the same with "validate_source_path": false, so that the source path will not be validated here.

[christian-naenny]

As you can see in my snippet of the playbook, validate_source_path is set to false! That was the first thing I tried but without any luck. The first version of the playbook did not contain the parameter validate_source_path at all, but I got the same error message!

[suhasbshekar]

DEVOPS-6839