Ever wanted to build a complex Splunk environment for testing, which looks as close as possible to a production deployment? Need to test a Splunk upgrade? See how Splunk indexer- or search head clustering works? Or just need to verify some configuration changes? This is the right place for you! The aim of this framework is to produce a Splunk environment in a fast and convenient way for testing purposes or maybe also for production use. The created Splunk installation and setup follows best practices. There are many ways to configure a Splunk environment in terms of configuration file locations, so this is just another example how to do it.
Note: This framework is not officially supported by Splunk. I am developing this on best effort in my spare time.
See the upcoming features in the Roadmap
Implemented changes are to be found in the Changelog
The Framework is currently tested on Mac OSX and Linux, but any other Unix, which is supported by Virtualbox, should work too.
brew install ansible
. For supported Ansible versions check hereVagrant
and change into it.git clone https://github.com/splunk/splunk-platform-automator.git
Software
.Software
directory
Software
directory
Splunk_Enterprise.lic
inside the Software
directory.Your directory structure should now look like this:
./Vagrant/splunk-platform-automator/...
./Vagrant/Software/Configurations - Base/...
./Vagrant/Software/Configurations - Index Replication/...
./Vagrant/Software/splunk-8.1.2-545206cc9f70-Linux-x86_64.tgz
./Vagrant/Software/splunkforwarder-8.1.2-545206cc9f70-Linux-x86_64.tgz
./Vagrant/Software/Splunk_Enterprise.lic
vagrant plugin install vagrant-vbguest
The Windows Subsystem for Linux does allow to run Splunk Platform Automator on Windows. It even allows to create virtualbox hosts from it on the windows host directly.
Execute the steps above. To allow vagrant to talk to virtualbox follow the steps below.
wsl --shutdown
)[automount]
options = "metadata"
vagrant plugin install virtualbox_WSL2
~/.bashrc
)export VAGRANT_WSL_ENABLE_WINDOWS_ACCESS="1"
export PATH="$PATH:/mnt/c/Program Files/Oracle/VirtualBox"
vagrant plugin install vagrant-aws
. The plugin is not maintained anymore and has issues with newer vagrant versions on OSX. The last working version of vagrant is 2.3.4vagrant box add aws-dummy https://github.com/mitchellh/vagrant-aws/raw/master/dummy.box
Type | Protocol | Port Range | Source | Description |
---|---|---|---|---|
All TCP | TCP | 0 - 65535 | 172.31.0.0/16 | Allow all internal traffic |
Custom TCP | TCP | 8000 | 0.0.0.0/0 | Splunk Web Interface |
SSH | TCP | 22 | 0.0.0.0/0 | SSH to all hosts |
Type | Protocol | Port Range | Destination | Description |
---|---|---|---|---|
All Traffic | All | All | 0.0.0.0/0 | Allow all traffic |
To upgrade your Splunk Platform Automator, just update your local code from the repo
git pull
From Splunk Platform Automator 2.0 the Framework does use an Ansible Inventory Plugin to build the inventory on the fly during execution. The local inventory
directory does only hold minimum settings based on the virtualization you choose. The rest calculated in flight and not stored somewhere.
You can verify your inventory with
ansible-inventory --list --export
The steps here apply to all environments.
plugin
setting to the top of your config file
# splunk_config.yml
plugin: splunk-platform-automator
The steps here only apply if your current environment is built on virtualbox.
rm -rf inventory/group*
start_ip
in the general
section to the virtualbox
section.The steps here only apply if your current environment is built on AWS.
rm -rf inventory/*
vagrant status
tag:SplunkEnvID:
and add a tag SplunkEnvID
to every host in your AWS environment with that GUIDSplunkHostname
for every AWS host with the name of your hosts from the splunk_hosts sectionIf you have the aws cli available, this can be done with the following one liner
for machine in $(ls -1d .vagrant/machines/*); do aws ec2 create-tags --resources $(cat $machine/aws/id) --tags Key=SplunkHostname,Value=$(basename $machine) Key=SplunkEnvID,Value=$(grep "tag:SplunkEnvID:" config/aws_ec2.yml | cut -d: -f3 | tr -d " ") Key=Name,Value=$(basename $machine) --no-cli-pager; done
The steps here only apply if your current environment is not built with vagrant.
The process is not so traight forward, since I do not know how you built your ansible inventory. Basically, you have to make sure everything you defined in your inventory files is reflected in the splunk_config.yml file.
ansible-inventory --list --export > inventory_1.txt
rm -rf inventory/*
ansible-inventory
command and compare it with your dump from version 1.xWith Splunk 9.x some configuration terms have been renamed and are considered deprecated. The Base Config Apps habe been changed and need to be updated and some apps have been renamed. Please download the latest version. The Splunk Platform Automator Framework does taking care of this automatically when Splunk version 9.x or higher is installed. Existing installation are working as is but the settings should be changed manually in the apps.
Those settings have been changed:
manager
or peer
Due to this some roles have been renamed in the config file. Please update your exsiting configs:
To build your own windows vagrant image follow Setup Windows Vagrant image
Run vagrant the first time to initialize itself and create needed directories. You must execute vagrant always in side the Splunk Platform Automator directory where the Vagrantfile
sits, otherwise it will not work correctly. You will see the usage page, when executing vagrant without options.
cd splunk-platform-automator
vagrant
There is one single configuration file, where all settings for your deployment are defined. Copy one configuration file from the examples to config/splunk_config.yml
and adjust the setting to your needs. For a standard setup you should be fine with most of the default settings, but there are a lot of things you can adjust for special cases. See the configuration description file, where all existing values are described.
AWS: See instruction here when deploying into Amazon Cloud. You can start with splunk_config_aws.yml for a simple environment. Copy splunk_idxclusters
, splunk_shclusters
and splunk_hosts
sections from other examples for more complex deployments.
When building virtual machines (for virtualbox) the first time it will pull an os image from the internet. The box images are cached here: ~/.vagrant.d/boxes
.
vagrant up
The vagrant up
command only creates the virtual machines. To deploy Splunk afterwards, run this command:
ansible-playbook ansible/deploy_site.yml
To run both steps with one command use:
vagrant up; ansible-playbook ansible/deploy_site.yml
This will gracefully shutdown all the virtual machines.
vagrant halt
You can destroy all the virtual machines with one command.
vagrant destroy [-f] [<hostname>]
Ansible playbooks can be run over and over again. If the virtual machine is already built, you can rerun the playbooks on a certain host again. This can be needed if something fails and you fixed the error.
ansible-playbook ansible/deploy_site.yml [--limit <hostname>]
To login to one of the hosts just open the index.html
file created in the splunk-platform-automator/config directory. You will find links to every role of your deployment.
If something changes along the way and you need to update the linkpage just call this playbook:
ansible-playbook ansible/create_linkpage.yml
Vagrant deployes an ssh key for the vagrant user to login without a password.
vagrant ssh <hostname>
Vagrant uses a dedicated user to work inside the virtual machines. The user name is vagrant
and has sudo rights to switch to root or other users.
Splunk Enterprise is installed and run as user splunk
. You can switch to this user by sudo su - splunk
. For convenience, I have added some command aliases to the user vagrant
and user splunk
.
alias
You can copy files from your host system to the virtual nodes with the vagrant command. You need to install the vagrant plugin vagrant-scp
to have this feature available. Check Vagrant Docs on how to do this.
vagrant scp <files> <target_on_dest> [vm_name]
vagrant scp ../app_dir/splunk-add-on-for-unix-and-linux_831.tgz /var/tmp uf
Splunk Platform Automator can talk to the AWS cloud and create virtual machines with Splunk in the cloud. Vagrant is using the plugin vagrant-aws for that. Follow these steps to setup Splunk Platform Automator for AWS. In the example there is a simple network setup, with only one Security group, covering all ports. More complex network setups should be possible, but make sure the host, where Splunk Platform Automator is running does have ssh access to all instances.
To prepare the configuration file for Amazon deployments
You can copy splunk_hosts and cluster configs from other example files to the AWS template to create more complex environments. There can be all configuration option used, which are described in the vargant-aws plugin. They can also set individually on the splunk hosts, if needed. Just add a aws: section to the host.
You can also use the ansible playbooks without vagrant. For that you have to create your virtual or physical machines by other means. You can use the ansible playbooks to
deploy the Splunk roles onto the existing servers. Specify the hostnames in the splunk_config.yml
file in the splunk_hosts
section.
Ansible needs to know where to connect to via ssh to run the playbooks. For this you need to create some custom variables in the
splunk_config.yml
file.
As a minimum specify the ssh user for ansible and the ssh private key which has been deployed on the systems. This user must be able to elevate to the root
user with sudo.
custom:
ansible_user: ansible
ansible_ssh_private_key_file: '~/.ssh/id_rsa'
If you have host specific variables the custom section can also be added on host level. This could be for example ansible_host
if different from the hostname. Also check configuration description
You can verify things like this first with an ansible ping:
ansible -m ping all
And then some more ansible prerequisites with this playbook
ansible-playbook ansible/test_ansible_prereqs.yml
You can build your own python version, if you need a specific python version or your local one is outdated. Please install the needed development tools in order to be able to compile stuff.
This examples installs Python 3.9.9 into your home directory.
cd
mkdir tmp
mkdir -p ~/Python/3.9.9
cd tmp
wget https://www.python.org/ftp/python/3.9.9/Python-3.9.9.tgz
tar -xzf Python-3.9.9.tgz
cd Python-3.9.9/
./configure --prefix=~/Python/3.9.9 --with-ensurepip=install
make
make install
If you need a specific Ansible version you can create it inside a virtualenv environment. This can be useful when deploying older linux images, which too old python versions.
python3 -m venv ansible_414
source ansible_414/bin/activate
python -m pip install ansible==7.7.0 # (which is ansible 2.14.10)
You must install some additional modules for Splunk Platform Automator to work
python -m pip install jmespath # required for json_query calls
python -m pip install lxml # required for license file checks
python -m pip install boto3 # required for ec2 (aws) plugin
Check the ansible version.
ansible --version
If the version is not correct, open a new terminal and activate the virtual environment again with the command from above.
source ansible_414/bin/activate
The following Ansible versions are tested and working with Splunk Platform Automator, but any newer version should work as well.
Check the Ansible Support Matrix for the most current information.
Copyright 2022 Splunk Inc.
Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.