search

splunk / splunk-platform-automator

Ansible framework providing a fast and simple way to spin up complex Splunk environments.
Apache License 2.0
117 stars 46 forks source link

Splunk Hosts index.html creation broken #18

Closed tonyprinzler closed 2 years ago

tonyprinzler commented 3 years ago

Hi Marco,

by executing the ansible-playbook ansible/deploy_site.yml i got the following error by the task:

TASK [splunk_control : Create Splunk Hosts index.html] *************************************************************************************************************************************************************************************************************************************** fatal: [localhost]: FAILED! => {"changed": false, "msg": "AnsibleError: template error while templating string: no test named 'true'. String: <!doctype html>\n<html lang=\"en\">\n <head>\n <!-- Required meta tags -->\n <meta charset=\"utf-8\">\n <meta name=\"viewport\" content=\"width=device-width, initial-scale=1, shrink-to-fit=no\">\n\n <!-- Bootstrap CSS -->\n <link rel=\"stylesheet\" href=\"https://cdn.jsdelivr.net/npm/bootstrap@4.6.0/dist/css/bootstrap.min.css\" integrity=\"sha384-B0vP5xmATw1+K9KRQjQERJvTumQW0nPEzvF6L/Z6nronJ3oUOFUFpCjEUQouq2+l\" crossorigin=\"anonymous\">\n\n <title>Splunkenizer Host List</title>\n </head>\n <body>\n\n <div class=\"container-fluid\" class=\"mx-auto\" class=\"border border-primary\">\n <h1><p class=\"text-center\">Splunkenizer Host List</p></h1>\n </div>\n\n <div class=\"container-fluid\">\n <div class=\"row\">\n {%- for splunk_env in groups|map('regex_search','splunk_env_.*')|select('string')|list -%}\n {%- for hostname in groups[splunk_env] -%}\n <div class=\"col-sm-3\">\n <div class=\"card text-white bg-secondary mb-3\">\n <div class=\"card-header\">{{ hostname }}</div>\n <div class=\"card-body\">\n <p class=\"card-text\">Env: {{ hostvars[hostname]['splunk_env_name'] }}</p>\n <p class=\"card-text\">IP Addr: {{ hostvars[hostname]['ip_addr']|default(\"unknown\") }}</p>\n </div>\n <div class=\"card-footer\">\n {%- set web_port = '8000' -%}\n {%- if hostvars[hostname]['splunk_ssl']['web']['enable'] is true -%}\n {%- set url = 'https://' -%}\n {%- else -%}\n {%- set url = 'http://' -%}\n {%- endif -%}\n {%- for role_name in groups|map('regex_search','role_.*')|select('string')|list -%}\n {%- if hostname in groups[role_name] -%}\n {%- if role_name == \"role_search_head\" -%}\n <a class=\"btn btn-success\" target=\"_blank\" href=\"{{ url }}{{ hostvars[hostname]['public_dns_name']|default(hostvars[hostname]['ip_addr']|default(\"unknown\")) }}:{{web_port}}/{{ url_locale|default(\"en-GB\") }}/app/launcher/home\">Search Head</a>\n {%- elif role_name == \"role_indexer\" -%}\n <span class=\"d-inline-block\" tabindex=\"0\" data-toggle=\"tooltip\" title=\"No web access on indexer\">\n <button class=\"btn btn-primary\" style=\"pointer-events: none;\" type=\"button\" disabled=\"\">Indexer</button>\n </span>\n {%- elif role_name == \"role_license_master\" -%}\n <a class=\"btn btn-success\" target=\"_blank\" href=\"{{ url }}{{ hostvars[hostname]['public_dns_name']|default(hostvars[hostname]['ip_addr']|default(\"unknown\")) }}:{{web_port}}/{{ url_locale|default(\"en-GB\") }}/manager/system/licensing\">License Master</a>\n {%- elif role_name == \"role_cluster_master\" -%}\n <a class=\"btn btn-success\" target=\"_blank\" href=\"{{ url }}{{ hostvars[hostname]['public_dns_name']|default(hostvars[hostname]['ip_addr']|default(\"unknown\")) }}:{{web_port}}/{{ url_locale|default(\"en-GB\") }}/manager/system/clustering\">Cluster Master</a>\n {%- elif role_name == \"role_deployment_server\" -%}\n <a class=\"btn btn-success\" target=\"_blank\" href=\"{{ url }}{{ hostvars[hostname]['public_dns_name']|default(hostvars[hostname]['ip_addr']|default(\"unknown\")) }}:{{web_port}}/{{ url_locale|default(\"en-GB\") }}/manager/system/deploymentserver\">Deployment Server</a>\n {%- elif role_name == \"role_deployer\" -%}\n <a class=\"btn btn-success\" target=\"_blank\" href=\"{{ url }}{{ hostvars[hostname]['public_dns_name']|default(hostvars[hostname]['ip_addr']|default(\"unknown\")) }}:{{web_port}}/{{ url_locale|default(\"en-GB\") }}/app/launcher/home\">Deployer</a>\n {%- elif role_name == \"role_monitoring_console\" -%}\n <a class=\"btn btn-success\" target=\"_blank\" href=\"{{ url }}{{ hostvars[hostname]['public_dns_name']|default(hostvars[hostname]['ip_addr']|default(\"unknown\")) }}:{{web_port}}/{{ url_locale|default(\"en-GB\") }}/app/splunk_monitoring_console\">Monitoring Console</a>\n {%- elif role_name == \"role_heavy_forwarder\" -%}\n <a class=\"btn btn-success\" target=\"_blank\" href=\"{{ url }}{{ hostvars[hostname]['public_dns_name']|default(hostvars[hostname]['ip_addr']|default(\"unknown\")) }}:{{web_port}}/{{ url_locale|default(\"en-GB\") }}/app/launcher/home\">Heavy Forwarder</a>\n {%- elif role_name == \"role_universal_forwarder\" -%}\n <span class=\"d-inline-block\" tabindex=\"0\" data-toggle=\"tooltip\" title=\"No web access on universal forwarder\">\n <button class=\"btn btn-primary\" style=\"pointer-events: none;\" type=\"button\" disabled=\"\">Universal Forwarder</button>\n </span>\n {%- elif role_name == \"role_universal_forwarder_windows\" -%}\n <span class=\"d-inline-block\" tabindex=\"0\" data-toggle=\"tooltip\" title=\"No web access on universal forwarder\">\n <button class=\"btn btn-primary\" style=\"pointer-events: none;\" type=\"button\" disabled=\"\">Universal Forwarder (Windows)</button>\n </span>\n {%- endif -%}\n {%- endif -%}\n {%- endfor -%}\n </div>\n </div>\n </div>\n {%- endfor -%}\n {%- endfor -%}\n </div>\n </div>\n </body>\n</html>\n"}

Tested this with single node and cluster. There are no custom modifications done. Only copied files from examples to config.

OS: CentOS 8 Ansible: 2.9.25 Python: 3.6.8

splunkenizer commented 3 years ago

Hi Tony

Can you please explain a bit more about your setup? Are you using Vagrant? I assume you are using virtualbox, since the examples are all for virtualbox.

You can try with this to skip the index.html creation:

ansible-playbook ansible/deploy_site.yml --skip-tags splunk_control

If that works, the something with the index.html task is wrong.

Please let me know how it goes.

Thanks

Marco

On Wed, Oct 13, 2021 at 10:45 AM tonyprinzler @.***> wrote:

[ External sender. Exercise caution. ]

Hi Marco,

by executing the ansible-playbook ansible/deploy_site.yml i got the following error by the task:

TASK [splunk_control : Create Splunk Hosts index.html]

fatal: [localhost]: FAILED! => {"changed": false, "msg": "AnsibleError: template error while templating string: no test named 'true'. String: <!doctype html>\n<html lang=\"en\">\n \n \n <meta charset=\"utf-8\">\n <meta name=\"viewport\" content=\"width=device-width, initial-scale=1, shrink-to-fit=no\">\n\n \n <link rel=\"stylesheet\" href=\" @.**/dist/css/bootstrap.min.css\" integrity=\"sha384-B0vP5xmATw1+K9KRQjQERJvTumQW0nPEzvF6L/Z6nronJ3oUOFUFpCjEUQouq2+l\" crossorigin=\"anonymous\">\n\n Splunkenizer Host List\n \n \n\n <div class=\"container-fluid\" class=\"mx-auto\" class=\"border border-primary\">\n

<p class=\"text-center\">Splunkenizer Host List

\n
\n\n <div class=\"container-fluid\">\n <div class=\"row\">\n {%- for splunk_env in groups|map('regex_search','splunkenv.')|select('string')|list -%}\n {%- for hostname in groups[splunk_env] -%}\n <div class=\"col-sm-3\">\n <div class=\"card text-white bg-secondary mb-3\">\n <div class=\"card-header\">{{ hostname }}
\n <div class=\"card-body\">\n <p class=\"card-text\">Env: {{ hostvars[hostname]['splunk_env_name'] }}

\n <p class=\"card-text\">IP Addr: {{ hostvars[hostname]['ip_addr']|default(\"unknown\") }}

\n \n <div class=\"card-footer\">\n {%- set web_port = '8000' -%}\n {%- if hostvars[hostname]['splunk_ssl']['web']['enable'] is true -%}\n {%- set url = 'https://' -%}\n {%- else -%}\n {%- set url = 'http://' -%}\n {%- endif -%}\n {%- for role_name in groups|map('regexsearch','role.*')|select('string')|list -%}\n {%- if hostname in groups[role_name] -%}\n {%- if role_name == \"role_search_head\" -%}\n <a class=\"btn btn-success\" target=\"_blank\" href=\"{{ url }}{{ hostvars[hostname]['public_dns_name']|default(hostvars[hostname]['ip_addr']|default(\"unknown\")) }}:{{web_port}}/{{ url_locale|default(\"en-GB\") }}/app/launcher/home\">Search Head\n {%- elif role_name == \"role_indexer\" -%}\n <span class=\"d-inline-block\" tabindex=\"0\" data-toggle=\"tooltip\" title=\"No web access on indexer\">\n <button class=\"btn btn-primary\" style=\"pointer-events: none;\" type=\"button\" disabled=\"\">Indexer\n \n {%- elif role_name == \"role_license_master\" -%}\n <a class=\"btn btn-success\" target=\"_blank\" href=\"{{ url }}{{ hostvars[hostname]['public_dns_name']|default(hostvars[hostname]['ip_addr']|default(\"unknown\")) }}:{{web_port}}/{{ url_locale|default(\"en-GB\") }}/manager/system/licensing\">License Master\n {%- elif role_name == \"role_cluster_master\" -%}\n <a class=\"btn btn-success\" target=\"_blank\" href=\"{{ url }}{{ hostvars[hostname]['public_dns_name']|default(hostvars[hostname]['ip_addr']|default(\"unknown\")) }}:{{web_port}}/{{ url_locale|default(\"en-GB\") }}/manager/system/clustering\">Cluster Master\n {%- elif role_name == \"role_deployment_server\" -%}\n <a class=\"btn btn-success\" target=\"_blank\" href=\"{{ url }}{{ hostvars[hostname]['public_dns_name']|default(hostvars[hostname]['ip_addr']|default(\"unknown\")) }}:{{web_port}}/{{ url_locale|default(\"en-GB\") }}/manager/system/deploymentserver\">Deployment Server\n {%- elif role_name == \"role_deployer\" -%}\n <a class=\"btn btn-success\" target=\"_blank\" href=\"{{ url }}{{ hostvars[hostname]['public_dns_name']|default(hostvars[hostname]['ip_addr']|default(\"unknown\")) }}:{{web_port}}/{{ url_locale|default(\"en-GB\") }}/app/launcher/home\">Deployer\n {%- elif role_name == \"role_monitoring_console\" -%}\n <a class=\"btn btn-success\" target=\"_blank\" href=\"{{ url }}{{ hostvars[hostname]['public_dns_name']|default(hostvars[hostname]['ip_addr']|default(\"unknown\")) }}:{{web_port}}/{{ url_locale|default(\"en-GB\") }}/app/splunk_monitoring_console\">Monitoring Console\n {%- elif role_name == \"role_heavy_forwarder\" -%}\n <a class=\"btn btn-success\" target=\"_blank\" href=\"{{ url }}{{ hostvars[hostname]['public_dns_name']|default(hostvars[hostname]['ip_addr']|default(\"unknown\")) }}:{{web_port}}/{{ url_locale|default(\"en-GB\") }}/app/launcher/home\">Heavy Forwarder\n {%- elif role_name == \"role_universal_forwarder\" -%}\n <span class=\"d-inline-block\" tabindex=\"0\" data-toggle=\"tooltip\" title=\"No web access on universal forwarder\">\n <button class=\"btn btn-primary\" style=\"pointer-events: none;\" type=\"button\" disabled=\"\">Universal Forwarder\n \n {%- elif role_name == \"role_universal_forwarder_windows\" -%}\n <span class=\"d-inline-block\" tabindex=\"0\" data-toggle=\"tooltip\" title=\"No web access on universal forwarder\">\n <button class=\"btn btn-primary\" style=\"pointer-events: none;\" type=\"button\" disabled=\"\">Universal Forwarder (Windows)\n \n {%- endif -%}\n {%- endif -%}\n {%- endfor -%}\n \n \n \n {%- endfor -%}\n {%- endfor -%}\n \n \n \n\n"}

Tested this with single node and cluster. There are no custom modifications done. Only copied files from examples to config.

OS: CentOS 8 Ansible: 2.9.25 Python: 3.6.8

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/splunkenizer/Splunkenizer/issues/18, or unsubscribe https://github.com/notifications/unsubscribe-auth/AHWAYLTZSM4A6JERSL2GDFTUGVBJXANCNFSM5F4TXGSQ . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

JuGuSm commented 2 years ago

Hi,

Same issue and when I try to skip the index.html creation, no hosts matched.

$ansible -m ping all
[WARNING]: Ansible is being run in a world writable directory (/mnt/c/Splunkenizer/Vagrant/Splunkenizer), ignoring it as an ansible.cfg source. For more information see
https://docs.ansible.com/ansible/devel/reference_appendices/config.html#cfg-in-world-writable-dir
[WARNING]: No inventory was parsed, only implicit localhost is available
[WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all'

How should I specify the host created with vagrant?

I'm using Vagrant and VirtualBox.

splunkenizer commented 2 years ago

Hi

I can see you are using Splunkenizer on Windows with WSL2. Have you set everything according to the Setup steps here? https://github.com/splunkenizer/Splunkenizer#install-and-configure-aws-support-optional

Could you please send me the output of your vagrant up command? It seems that the ansible inventory has not been created.

What is in the file Splunkenizer/inventory/hosts file after you created the vagrant hosts?

JuGuSm commented 2 years ago

Hi, here is the output of vagrant up:

Bringing machine 'shidx' up with 'virtualbox' provider...
==> shidx: Checking if box 'centos/7' version '2004.01' is up to date...
==> shidx: Clearing any previously set network interfaces...
==> shidx: Preparing network interfaces based on configuration...
    shidx: Adapter 1: nat
    shidx: Adapter 2: hostonly
==> shidx: Forwarding ports...
    shidx: 22 (guest) => 2222 (host) (adapter 1)
==> shidx: Running 'pre-boot' VM customizations...
==> shidx: Booting VM...
==> shidx: Waiting for machine to boot. This may take a few minutes...
    shidx: SSH address: 127.0.0.1:2222
    shidx: SSH username: vagrant
    shidx: SSH auth method: private key
    shidx: 
    shidx: Vagrant insecure key detected. Vagrant will automatically replace
    shidx: this with a newly generated keypair for better security.
    shidx: 
    shidx: Inserting generated public key within guest...
    shidx: Removing insecure key from the guest if it's present...
    shidx: Key inserted! Disconnecting and reconnecting using new SSH key...
==> shidx: Machine booted and ready!
[shidx] No Virtualbox Guest Additions installation found.
Updating GuestAdditions skipped.
==> shidx: Checking for guest additions in VM...
    shidx: No guest additions were detected on the base box for this VM! Guest
    shidx: additions are required for forwarded ports, shared folders, host only
    shidx: networking, and more. If SSH fails on this machine, please install
    shidx: the guest additions and repackage the box to continue.
    shidx:
    shidx: This is not an error message; everything may continue to work properly,
    shidx: in which case you may ignore this message.
==> shidx: Setting hostname...
==> shidx: Configuring and enabling network interfaces...
==> shidx: Running action triggers after up ...
==> shidx: Running trigger...
==> shidx: Update local Ansible inventory

And here is the file Splunkenizer/inventory/hosts:

shidx
splunkenizer commented 2 years ago

The inventory created by vagrant looks good so far. But from the warning in your first output you can see, that ansible is ignoring the ansible.cfg file if the directory is world writable. This means it does not find the inventory which is explaining the issue. You should change the mount options like mentioned here: https://docs.ansible.com/ansible/devel/reference_appendices/config.html#cfg-in-world-writable-dir

tonyprinzler commented 2 years ago

Hi Marco,

i used Vagrant and VirtualBox. With skipping the deployment is working fine.

splunkenizer commented 2 years ago

Hi Marco,

i used Vagrant and VirtualBox. With skipping the deployment is working fine.

Could you please share your splunk_config.yml file, so I can reproduce the issue. It should work without skipping things.

tonyprinzler commented 2 years ago

Hi Marco, i used Vagrant and VirtualBox. With skipping the deployment is working fine.

Could you please share your splunk_config.yml file, so I can reproduce the issue. It should work without skipping things.

I tried the configurations from the examples:

splunkenizer commented 2 years ago

Hi Marco, i used Vagrant and VirtualBox. With skipping the deployment is working fine.

Could you please share your splunk_config.yml file, so I can reproduce the issue. It should work without skipping things.

I tried the configurations from the examples:

  • cm_2idxc_sh_uf.yml
  • single_node.yml

I did try with the single instance like this. Did you change something in the files? Have you updated to the latest git master branch (git pull)?

`➜ Vagrant git clone https://github.com/splunkenizer/Splunkenizer.git Splunkenizer_Test Klone nach 'Splunkenizer_Test' ... remote: Enumerating objects: 4044, done. remote: Counting objects: 100% (1122/1122), done. remote: Compressing objects: 100% (607/607), done. remote: Total 4044 (delta 695), reused 829 (delta 434), pack-reused 2922 Empfange Objekte: 100% (4044/4044), 2.19 MiB | 80.00 KiB/s, fertig. Löse Unterschiede auf: 100% (2319/2319), fertig. ➜ Vagrant cd Splunkenizer_Test ➜ Splunkenizer_Test git:(master) ls CHANGELOG.md README.md VERSION ansible defaults examples template LICENSE ROADMAP.md Vagrantfile ansible.cfg docs pic ➜ Splunkenizer_Test git:(master) mkdir config ➜ Splunkenizer_Test git:(master) cd config ➜ config git:(master) cp ../examples/single_node.yml splunk_config.yml ➜ config git:(master) cd .. ➜ Splunkenizer_Test git:(master) vagrant up; ap ansible/deploy_site.yml Bringing machine 'shidx' up with 'virtualbox' provider... ==> shidx: Importing base box 'centos/7'... ==> shidx: Matching MAC address for NAT networking... ==> shidx: Checking if box 'centos/7' version '2004.01' is up to date... ==> shidx: Setting the name of the VM: Splunkenizer_Test_shidx_1637070011089_420 ==> shidx: Clearing any previously set network interfaces... ==> shidx: Preparing network interfaces based on configuration... shidx: Adapter 1: nat shidx: Adapter 2: hostonly ==> shidx: Forwarding ports... shidx: 22 (guest) => 2222 (host) (adapter 1) ==> shidx: Running 'pre-boot' VM customizations... ==> shidx: Booting VM... ==> shidx: Waiting for machine to boot. This may take a few minutes... shidx: SSH address: 127.0.0.1:2222 shidx: SSH username: vagrant shidx: SSH auth method: private key shidx: shidx: Vagrant insecure key detected. Vagrant will automatically replace shidx: this with a newly generated keypair for better security. shidx: shidx: Inserting generated public key within guest... shidx: Removing insecure key from the guest if it's present... shidx: Key inserted! Disconnecting and reconnecting using new SSH key... ==> shidx: Machine booted and ready! [shidx] No Virtualbox Guest Additions installation found. Updating GuestAdditions skipped. ==> shidx: Checking for guest additions in VM... shidx: No guest additions were detected on the base box for this VM! Guest shidx: additions are required for forwarded ports, shared folders, host only shidx: networking, and more. If SSH fails on this machine, please install shidx: the guest additions and repackage the box to continue. shidx: shidx: This is not an error message; everything may continue to work properly, shidx: in which case you may ignore this message. ==> shidx: Setting hostname... ==> shidx: Configuring and enabling network interfaces... ==> shidx: Running action triggers after up ... ==> shidx: Running trigger... ==> shidx: Update local Ansible inventory [WARNING]: Could not match supplied host pattern, ignoring: role_universal_forwarder_windows

PLAY [apply common configurations] **

TASK [Gathering Facts] ** ok: [shidx]

TASK [common : set os hostname to shidx] **** skipping: [shidx]

TASK [common : install additional os packages] ** skipping: [shidx]

TASK [common : Adding splunk aliases] *** changed: [shidx]

TASK [common : insert/update bashrc for splunk stuff] *** changed: [shidx]

TASK [common : set timezone] **** changed: [shidx]

TASK [common : check SELinux state] ***** ok: [shidx]

TASK [common : disable SELinux] ***** [WARNING]: SELinux state temporarily changed from 'enforcing' to 'permissive'. State change will take effect next reboot. changed: [shidx]

TASK [common : reboot the server and wait for it to come back up.] ** changed: [shidx]

TASK [common : collect facts about system services] ***** ok: [shidx]

TASK [common : stop and disable AppArmor] *** skipping: [shidx]

TASK [common : remove vagrant-hostmanager entries] ** ok: [shidx]

TASK [common : add splunk hosts to /etc/hosts] ** changed: [shidx]

TASK [common : setup time sync cronjob] ***** changed: [shidx]

PLAY [create splunk hosts link page] ****

TASK [splunk_control : start role tasks] **** ok: [localhost] => { "msg": "role 'splunk_control' start" }

TASK [splunk_control : Create Splunk Hosts index.html] ** changed: [localhost] [WARNING]: Could not match supplied host pattern, ignoring: role_cluster_master [WARNING]: Could not match supplied host pattern, ignoring: role_deployer [WARNING]: Could not match supplied host pattern, ignoring: role_deployment_server [WARNING]: Could not match supplied host pattern, ignoring: role_heavy_forwarder [WARNING]: Could not match supplied host pattern, ignoring: role_license_master [WARNING]: Could not match supplied host pattern, ignoring: role_monitoring_console

PLAY [install the splunk enterprise software] ***

TASK [Gathering Facts] ** ok: [shidx]

...

TASK [splunk_common : set use_splunk_systemd] *** ok: [shidx]

TASK [splunk_conf : loop through the config files] ** skipping: [shidx] [WARNING]: Could not match supplied host pattern, ignoring: role_ldap_server

PLAY [setup ldap server role] *** skipping: no hosts matched

PLAY RECAP ** localhost : ok=2 changed=1 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0 shidx : ok=170 changed=50 unreachable=0 failed=0 skipped=83 rescued=0 ignored=0 `

tonyprinzler commented 2 years ago

I used the latest version. Downloaded today and changed nothing inside of the configuration files.

splunkenizer commented 2 years ago

I used the latest version. Downloaded today and changed nothing inside of the configuration files.

How did you download/install? Can you try with the git clone command?

tonyprinzler commented 2 years ago

I created the folder Vagrant and Vagrant/Software. After this i moved to Vagrant and used the git clone command, showed in the documentation.

tonyprinzler commented 2 years ago

I tried this also without vagrant and virtualbox. Added the needed entrys into the splunk_config.yml and got the same result.

For this test i used a new installed CentOS8 with the latest ansible. Additional you need on CentOS lxml and PyYAML. Without lxml its not possible to use a license in your deployment and without PyYAML ansible is not able to parse the splunk_config.yml

splunkenizer commented 2 years ago

Yes, there are some dependencies to be installed for it to work.

Could you please run this command and paste the output here?

ansible-inventory --list --export

tonyprinzler commented 2 years ago

ansible-inventory --list --export { "_meta": { "hostvars": { "host-8945": { "site": "site1" }, "host-8947": { "site": "site0" }, "host-8948": { "site": "site1" } } }, "all": { "children": [ "idxcluster_idxc1", "idxcluster_idxc1_site1", "output_splk_all", "role_cluster_master", "role_deployer", "role_deployment_server", "role_indexer", "role_license_master", "role_monitoring_console", "role_search_head", "role_universal_forwarder", "search_peer_splk_all", "shcluster_shc1", "splunk_env_splk", "ungrouped" ], "vars": { "ansible_port": 22, "ansible_user": "user1", "become_user": "root", "disable_apparmor": true, "disable_selinux": true, "packages": [ "unzip", "epel-release", "sysstat", "lsof", "net-tools" ], "splunk_admin_password": "splunklab", "splunk_apps_dir": "../app_repo", "splunk_auth_dir": "../auth", "splunk_baseconfig_dir": "../Software", "splunk_env_name": "splk", "splunk_indexes": [ "test1", "test2" ], "splunk_license_file": "Splunk_Enterprise.lic", "splunk_outputs": "all", "splunk_save_baseconfig_apps": false, "splunk_save_baseconfig_apps_dir": "apps", "splunk_save_serverclass": false, "splunk_search_peers": "all", "splunk_secret_share": { "equal": false, "splunk": false, "splunkforwarder": false }, "splunk_software_dir": "../Software", "splunk_ssl": { "inputs": { "config": { "rootCA": "$SPLUNK_HOME/etc/auth/cacert.pem", "serverCert": "$SPLUNK_HOME/etc/auth/server.pem", "sslPassword": "password" }, "enable": false, "own_certs": false }, "outputs": { "config": { "sslCertPath": "$SPLUNK_HOME/etc/auth/server.pem", "sslPassword": "password", "sslRootCAPath": "$SPLUNK_HOME/etc/auth/cacert.pem" }, "enable": false, "own_certs": false }, "web": { "config": { "enableSplunkWebSSL": true }, "enable": false, "own_certs": false } }, "splunk_systemd_services": { "splunk": { "Service": { "LimitDATA": "infinity", "LimitFSIZE": "infinity", "LimitNOFILE": 65536, "LimitNPROC": 20480, "TasksMax": 20480 } } }, "splunk_version": "8.2.2.1", "time_zone": "Europe/Berlin", "update_hosts_file": false, "url_locale": "en-GB" } }, "idxcluster_idxc1": { "hosts": [ "host-8945", "host-8947", "host-8948" ], "vars": { "idxc_available_sites": [ "site1" ], "idxc_name": "idxc1", "idxc_password": "splunkidxc", "idxc_replication_port": 9887, "idxc_site_rf": "origin:2, total:2", "idxc_site_sf": "origin:2, total:2" } }, "idxcluster_idxc1_site1": { "hosts": [ "host-8945", "host-8948" ] }, "output_splk_all": { "hosts": [ "host-8946", "host-8947", "host-8958", "host-8959", "host-8960", "host-8961" ], "vars": { "splunk_output_list": { "idxcluster": { "idxc1": [ "host-8948", "host-8945" ] } } } }, "role_cluster_master": { "hosts": [ "host-8947" ] }, "role_deployer": { "hosts": [ "host-8947" ] }, "role_deployment_server": { "hosts": [ "host-8947" ] }, "role_indexer": { "hosts": [ "host-8945", "host-8948" ] }, "role_license_master": { "hosts": [ "host-8947" ] }, "role_monitoring_console": { "hosts": [ "host-8947" ] }, "role_search_head": { "hosts": [ "host-8946", "host-8958", "host-8961" ] }, "role_universal_forwarder": { "hosts": [ "host-8959", "host-8960" ] }, "search_peer_splk_all": { "hosts": [ "host-8946", "host-8947", "host-8958", "host-8961" ], "vars": { "splunk_search_peer_list": { "idxcluster": { "idxc1": "host-8947" } } } }, "shcluster_shc1": { "hosts": [ "host-8946", "host-8947", "host-8958", "host-8961" ], "vars": { "shc_name": "shc1", "shc_password": "splunkshc", "shc_replication_port": 9887, "shc_site": "site0" } }, "splunk_env_splk": { "hosts": [ "host-8945", "host-8946", "host-8947", "host-8948", "host-8958", "host-8959", "host-8960", "host-8961" ] }

tonyprinzler commented 2 years ago

This was the last deployment. Runs without any problems with skipping splunk_control

splunkenizer commented 2 years ago

Do you have an entry for localhost in your /etc/hosts file?

tonyprinzler commented 2 years ago

Yes. IP, hostname and FQDN

splunkenizer commented 2 years ago

Could you try to add an entry with 'localhost' to the file 'inventory/hosts' and the run the playbook for the index page manually like this:

ansible-playbook ansible/create_linkpage.yml

tonyprinzler commented 2 years ago

`TASK [splunk_control : Create Splunk Hosts index.html] *** fatal: [localhost]: FAILED! => {"changed": false, "msg": "AnsibleError: template error while templating string: no test named 'true'. String: <!doctype html>\n<html lang=\"en\">\n \n \n <meta charset=\"utf-8\">\n <meta name=\"viewport\" content=\"width=device-width, initial-scale=1, shrink-to-fit=no\">\n\n \n <link rel=\"stylesheet\" href=\"https://cdn.jsdelivr.net/npm/bootstrap@4.6.0/dist/css/bootstrap.min.css\" integrity=\"sha384-B0vP5xmATw1+K9KRQjQERJvTumQW0nPEzvF6L/Z6nronJ3oUOFUFpCjEUQouq2+l\" crossorigin=\"anonymous\">\n\n Splunkenizer Host List\n \n \n\n <div class=\"container-fluid\" class=\"mx-auto\" class=\"border border-primary\">\n

<p class=\"text-center\">Splunkenizer Host List

\n
\n\n <div class=\"container-fluid\">\n <div class=\"row\">\n {%- for splunk_env in groups|map('regex_search','splunkenv.')|select('string')|list -%}\n {%- for hostname in groups[splunk_env] -%}\n <div class=\"col-sm-3\">\n <div class=\"card text-white bg-secondary mb-3\">\n <div class=\"card-header\">{{ hostname }}
\n <div class=\"card-body\">\n <p class=\"card-text\">Env: {{ hostvars[hostname]['splunk_env_name'] }}

\n <p class=\"card-text\">IP Addr: {{ hostvars[hostname]['ip_addr']|default(\"unknown\") }}

\n \n <div class=\"card-footer\">\n {%- set web_port = '8000' -%}\n {%- if hostvars[hostname]['splunk_ssl']['web']['enable'] is true -%}\n {%- set url = 'https://' -%}\n {%- else -%}\n {%- set url = 'http://' -%}\n {%- endif -%}\n {%- for role_name in groups|map('regexsearch','role.')|select('string')|list -%}\n {%- if hostname in groups[role_name] -%}\n {%- if role_name == \"role_search_head\" -%}\n <a class=\"btn btn-success\" target=\"_blank\" href=\"{{ url }}{{ hostvars[hostname]['public_dns_name']|default(hostvars[hostname]['ip_addr']|default(\"unknown\")) }}:{{web_port}}/{{ url_locale|default(\"en-GB\") }}/app/launcher/home\">Search Head\n {%- elif role_name == \"role_indexer\" -%}\n <span class=\"d-inline-block\" tabindex=\"0\" data-toggle=\"tooltip\" title=\"No web access on indexer\">\n <button class=\"btn btn-primary\" style=\"pointer-events: none;\" type=\"button\" disabled=\"\">Indexer\n \n {%- elif role_name == \"role_license_master\" -%}\n <a class=\"btn btn-success\" target=\"_blank\" href=\"{{ url }}{{ hostvars[hostname]['public_dns_name']|default(hostvars[hostname]['ip_addr']|default(\"unknown\")) }}:{{web_port}}/{{ url_locale|default(\"en-GB\") }}/manager/system/licensing\">License Master\n {%- elif role_name == \"role_cluster_master\" -%}\n <a class=\"btn btn-success\" target=\"_blank\" href=\"{{ url }}{{ hostvars[hostname]['public_dns_name']|default(hostvars[hostname]['ip_addr']|default(\"unknown\")) }}:{{web_port}}/{{ url_locale|default(\"en-GB\") }}/manager/system/clustering\">Cluster Master\n {%- elif role_name == \"role_deployment_server\" -%}\n <a class=\"btn btn-success\" target=\"_blank\" href=\"{{ url }}{{ hostvars[hostname]['public_dns_name']|default(hostvars[hostname]['ip_addr']|default(\"unknown\")) }}:{{web_port}}/{{ url_locale|default(\"en-GB\") }}/manager/system/deploymentserver\">Deployment Server\n {%- elif role_name == \"role_deployer\" -%}\n <a class=\"btn btn-success\" target=\"_blank\" href=\"{{ url }}{{ hostvars[hostname]['public_dns_name']|default(hostvars[hostname]['ip_addr']|default(\"unknown\")) }}:{{web_port}}/{{ url_locale|default(\"en-GB\") }}/app/launcher/home\">Deployer\n {%- elif role_name == \"role_monitoring_console\" -%}\n <a class=\"btn btn-success\" target=\"_blank\" href=\"{{ url }}{{ hostvars[hostname]['public_dns_name']|default(hostvars[hostname]['ip_addr']|default(\"unknown\")) }}:{{web_port}}/{{ url_locale|default(\"en-GB\") }}/app/splunk_monitoring_console\">Monitoring Console\n {%- elif role_name == \"role_heavy_forwarder\" -%}\n <a class=\"btn btn-success\" target=\"_blank\" href=\"{{ url }}{{ hostvars[hostname]['public_dns_name']|default(hostvars[hostname]['ip_addr']|default(\"unknown\")) }}:{{web_port}}/{{ url_locale|default(\"en-GB\") }}/app/launcher/home\">Heavy Forwarder\n {%- elif role_name == \"role_universal_forwarder\" -%}\n <span class=\"d-inline-block\" tabindex=\"0\" data-toggle=\"tooltip\" title=\"No web access on universal forwarder\">\n <button class=\"btn btn-primary\" style=\"pointer-events: none;\" type=\"button\" disabled=\"\">Universal Forwarder\n \n {%- elif role_name == \"role_universal_forwarder_windows\" -%}\n <span class=\"d-inline-block\" tabindex=\"0\" data-toggle=\"tooltip\" title=\"No web access on universal forwarder\">\n <button class=\"btn btn-primary\" style=\"pointer-events: none;\" type=\"button\" disabled=\"\">Universal Forwarder (Windows)\n \n {%- endif -%}\n {%- endif -%}\n {%- endfor -%}\n \n \n \n {%- endfor -%}\n {%- endfor -%}\n \n \n \n\n"}

PLAY RECAP *** localhost : ok=1 changed=0 unreachable=0 failed=1 skipped=0 rescued=0 ignored=0 `

splunkenizer commented 2 years ago

Could you please change the line in this file: ansible/roles/splunk_control/templates/config/index.html.j2

from {%- if hostvars[hostname]['splunk_ssl']['web']['enable'] is true -%}

to {%- if hostvars[hostname]['splunk_ssl']['web']['enable'] is sameas true -%}

and try the ansible playbook again.

tonyprinzler commented 2 years ago

Error is there: TASK [splunk_control : Create Splunk Hosts index.html] *************************************************************************************************************************************************************************************************************************************** fatal: [localhost]: FAILED! => {"changed": false, "msg": "AnsibleError: template error while templating string: Encountered unknown tag 'endfor'. You probably made a nesting mistake. Jinja is expecting this tag, but currently looking for 'elif' or 'else' or 'endif'. The innermost block that needs to be closed is 'if'.. String: <!doctype html>\n<html lang=\"en\">\n <head>\n <!-- Required meta tags -->\n <meta charset=\"utf-8\">\n <meta name=\"viewport\" content=\"width=device-width, initial-scale=1, shrink-to-fit=no\">\n\n <!-- Bootstrap CSS -->\n <link rel=\"stylesheet\" href=\"https://cdn.jsdelivr.net/npm/bootstrap@4.6.0/dist/css/bootstrap.min.css\" integrity=\"sha384-B0vP5xmATw1+K9KRQjQERJvTumQW0nPEzvF6L/Z6nronJ3oUOFUFpCjEUQouq2+l\" crossorigin=\"anonymous\">\n\n <title>Splunkenizer Host List</title>\n </head>\n <body>\n\n <div class=\"container-fluid\" class=\"mx-auto\" class=\"border border-primary\">\n <h1><p class=\"text-center\">Splunkenizer Host List</p></h1>\n </div>\n\n <div class=\"container-fluid\">\n <div class=\"row\">\n {%- for splunk_env in groups|map('regex_search','splunk_env_.*')|select('string')|list -%}\n {%- for hostname in groups[splunk_env] -%}\n <div class=\"col-sm-3\">\n <div class=\"card text-white bg-secondary mb-3\">\n <div class=\"card-header\">{{ hostname }}</div>\n <div class=\"card-body\">\n <p class=\"card-text\">Env: {{ hostvars[hostname]['splunk_env_name'] }}</p>\n <p class=\"card-text\">IP Addr: {{ hostvars[hostname]['ip_addr']|default(\"unknown\") }}</p>\n </div>\n <div class=\"card-footer\">\n {%- set web_port = '8000' -%}\n\t{%- if hostvars[hostname]['splunk_ssl']['web']['enable'] is sameas true -%}\n# {%- if hostvars[hostname]['splunk_ssl']['web']['enable'] is true -%}\n {%- set url = 'https://' -%}\n {%- else -%}\n {%- set url = 'http://' -%}\n {%- endif -%}\n {%- for role_name in groups|map('regex_search','role_.*')|select('string')|list -%}\n {%- if hostname in groups[role_name] -%}\n {%- if role_name == \"role_search_head\" -%}\n <a class=\"btn btn-success\" target=\"_blank\" href=\"{{ url }}{{ hostvars[hostname]['public_dns_name']|default(hostvars[hostname]['ip_addr']|default(\"unknown\")) }}:{{web_port}}/{{ url_locale|default(\"en-GB\") }}/app/launcher/home\">Search Head</a>\n {%- elif role_name == \"role_indexer\" -%}\n <span class=\"d-inline-block\" tabindex=\"0\" data-toggle=\"tooltip\" title=\"No web access on indexer\">\n <button class=\"btn btn-primary\" style=\"pointer-events: none;\" type=\"button\" disabled=\"\">Indexer</button>\n </span>\n {%- elif role_name == \"role_license_master\" -%}\n <a class=\"btn btn-success\" target=\"_blank\" href=\"{{ url }}{{ hostvars[hostname]['public_dns_name']|default(hostvars[hostname]['ip_addr']|default(\"unknown\")) }}:{{web_port}}/{{ url_locale|default(\"en-GB\") }}/manager/system/licensing\">License Master</a>\n {%- elif role_name == \"role_cluster_master\" -%}\n <a class=\"btn btn-success\" target=\"_blank\" href=\"{{ url }}{{ hostvars[hostname]['public_dns_name']|default(hostvars[hostname]['ip_addr']|default(\"unknown\")) }}:{{web_port}}/{{ url_locale|default(\"en-GB\") }}/manager/system/clustering\">Cluster Master</a>\n {%- elif role_name == \"role_deployment_server\" -%}\n <a class=\"btn btn-success\" target=\"_blank\" href=\"{{ url }}{{ hostvars[hostname]['public_dns_name']|default(hostvars[hostname]['ip_addr']|default(\"unknown\")) }}:{{web_port}}/{{ url_locale|default(\"en-GB\") }}/manager/system/deploymentserver\">Deployment Server</a>\n {%- elif role_name == \"role_deployer\" -%}\n <a class=\"btn btn-success\" target=\"_blank\" href=\"{{ url }}{{ hostvars[hostname]['public_dns_name']|default(hostvars[hostname]['ip_addr']|default(\"unknown\")) }}:{{web_port}}/{{ url_locale|default(\"en-GB\") }}/app/launcher/home\">Deployer</a>\n {%- elif role_name == \"role_monitoring_console\" -%}\n <a class=\"btn btn-success\" target=\"_blank\" href=\"{{ url }}{{ hostvars[hostname]['public_dns_name']|default(hostvars[hostname]['ip_addr']|default(\"unknown\")) }}:{{web_port}}/{{ url_locale|default(\"en-GB\") }}/app/splunk_monitoring_console\">Monitoring Console</a>\n {%- elif role_name == \"role_heavy_forwarder\" -%}\n <a class=\"btn btn-success\" target=\"_blank\" href=\"{{ url }}{{ hostvars[hostname]['public_dns_name']|default(hostvars[hostname]['ip_addr']|default(\"unknown\")) }}:{{web_port}}/{{ url_locale|default(\"en-GB\") }}/app/launcher/home\">Heavy Forwarder</a>\n {%- elif role_name == \"role_universal_forwarder\" -%}\n <span class=\"d-inline-block\" tabindex=\"0\" data-toggle=\"tooltip\" title=\"No web access on universal forwarder\">\n <button class=\"btn btn-primary\" style=\"pointer-events: none;\" type=\"button\" disabled=\"\">Universal Forwarder</button>\n </span>\n {%- elif role_name == \"role_universal_forwarder_windows\" -%}\n <span class=\"d-inline-block\" tabindex=\"0\" data-toggle=\"tooltip\" title=\"No web access on universal forwarder\">\n <button class=\"btn btn-primary\" style=\"pointer-events: none;\" type=\"button\" disabled=\"\">Universal Forwarder (Windows)</button>\n </span>\n {%- endif -%}\n {%- endif -%}\n {%- endfor -%}\n </div>\n </div>\n </div>\n {%- endfor -%}\n {%- endfor -%}\n </div>\n </div>\n </body>\n</html>\n"}

splunkenizer commented 2 years ago

What is this saying?

ansible --version

tonyprinzler commented 2 years ago

ansible 2.9.27 config file = /home/tony/Splunkenizer/ansible.cfg configured module search path = ['/home/tony/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules'] ansible python module location = /usr/lib/python3.6/site-packages/ansible executable location = /usr/bin/ansible python version = 3.6.8 (default, Sep 10 2021, 09:13:53) [GCC 8.5.0 20210514 (Red Hat 8.5.0-3)]

splunkenizer commented 2 years ago

I will try to reproduce this issue with the same version from above. What was the version of CentOS you are using?

tonyprinzler commented 2 years ago

Im using CentOS Linux release 8.5.2111

splunkenizer commented 2 years ago

I could reproduce the issue. Looks like older python versions are more strict to boolean comparison. I will be able to fix that.