splunk / ansible-role-for-splunk

Splunk@Splunk's Ansible role for installing Splunk, upgrading Splunk, and installing apps/addons on Splunk deployments (VM/bare metal)
Apache License 2.0
147 stars 103 forks source link

Cluster deployment fails in configure_dmc.yml #214

Open ChrisHills463 opened 11 months ago

ChrisHills463 commented 11 months ago

The task Configure systems as search peers to be monitored except indexers is defined thus:-

# Add all hosts to the DMC except indexer cluster hosts as per the docs
- name: Configure systems as search peers to be monitored except indexers
  ansible.builtin.shell: |
    {{ splunk_home }}/bin/splunk add search-server https://{{ item }}:{{ splunkd_port }} -auth "{{ splunk_auth }}" -remoteUsername "{{ splunk_admin_username }}" -remotePassword "{{ splunk_admin_password }}"
  loop: "{{ query('inventory_hostnames', 'all:!indexer') }}"
  become: true
  become_user: "{{ splunk_nix_user }}"
  no_log: true

When I call the deployment task configure_dmc.yml it fails:-

TASK [splunk : Configure systems as search peers to be monitored except indexers] ********************************************************************************************
changed: [splunk-mgr-1-staging.example] => (item=splunk-sh-1-staging.example)
changed: [splunk-mgr-1-staging.example] => (item=splunk-sh-2-staging.example)
changed: [splunk-mgr-1-staging.example] => (item=splunk-sh-3-staging.example)
failed: [splunk-mgr-1-staging.example] (item=splunk-mgr-1-staging.example) => {"ansible_loop_var": "item", "changed": true, "cmd": "/opt/splunk/bin/splunk add search-server https://splunk-mgr-1-staging.example:8089 -auth \"user:password\" -remoteUsername \"user\" -remotePassword \"password\"\n", "delta": "0:00:01.217357", "end": "2023-11-29 14:41:43.956036", "item": "splunk-mgr-1-staging.example", "msg": "non-zero return code", "rc": 24, "start": "2023-11-29 14:41:42.738679", "stderr": "WARNING: Server Certificate Hostname Validation is disabled. Please see server.conf/[sslConfig]/cliVerifyServerName for details.\n\nAn error occurred:\nPeer with server name splunk-mgr-1-staging.example conflicts with this server's name.", "stderr_lines": ["WARNING: Server Certificate Hostname Validation is disabled. Please see server.conf/[sslConfig]/cliVerifyServerName for details.", "", "An error occurred:", "Peer with server name splunk-mgr-1-staging.example conflicts with this server's name."], "stdout": "", "stdout_lines": []}

For convenience, this is the pertinent part of the log:-

Peer with server name splunk-mgr-1-staging.example conflicts with this server's name.

It appears that the dmc host is trying to add itself, causing the play to fail, but I am not sure what the resolution should be.

In the ansible hosts inventory, I have the following:-

[clustermanager]
splunk-mgr-1-staging.example

[deploymentserver]
splunk-mgr-1-staging.example

[dmc]
splunk-mgr-1-staging.example

[indexer]
splunk-idx-[1:3]-staging.example

[licensemaster]
splunk-mgr-1-staging.example

[search]
splunk-sh-[1:3]-staging.example

[shc]
splunk-sh-[1:3]-staging.example

[shcaptain]
splunk-sh-1-staging.example

[shdeployer]
splunk-mgr-1-staging

In main.yml I have:-

- hosts: dmc
  become: true
  vars:
    deployment_task: "configure_dmc.yml"
  roles:
    - role: splunk