The _splunkmonitor custom resource creates monitor:// stanzas in the inputs.conf file, with various optional properties. One of the properties, index, is validated against a fixed list of seemingly site-specific index names defined in the resource itself, _builtinindexes.
💔 Problem Statement
The _splunkmonitor resource fails when given a valid (for Splunk) index name that is not in the _builtinindexes list.
The code does seem to work as intended, it just unnecessarily limits the usefulness of the resource in deployments with custom indexes.
:grey_question: Possible Solution
Drop use of the _builtinindexes list to validate the index property of the _splunkmonitor resource, and adopt the regex validation method used by the _splunkindex resource instead (but accept a leading underscore).
And maybe separately, update the regex used to validate the index property for both the _splunkindex and _splunkmonitor resources, to align better with the Splunk (custom) index name requirements:
no uppercase characters (maybe valid pre-v6 ?)
no equals sign (typo ?)
may only begin with a number, lowercase character, or underscore--not just anything other than a hyphen.
may have hyphens (but not begin with one)
anchor the regex at the end (instead of accepting anything after the match)
:arrow_heading_up: Describe alternatives you've considered
Use the chef-splunk::client recipe to render inputs.conf instead of generating it with the _splunkmonitor resource.
:heavy_plus_sign: Additional context
Custom index naming requirements are the same at the latest iterations of v6, v7, v8 Splunk Enterprise.
User-defined index names must consist of only numbers, lowercase letters, underscores, and hyphens. They cannot begin with an underscore or hyphen, or contain the word "kvstore".
Edit test/fixtures/cookbooks/test/recipes/splunk_monitor.rb: change the index value (to some value not in the _builtinindexes list from resources/splunk_monitor.rb).
diff --git a/test/fixtures/cookbooks/test/recipes/splunk_monitor.rb b/test/fixtures/cookbooks/test/recipes/splunk_monitor.rb
index a4fb593..c025781 100644
--- a/test/fixtures/cookbooks/test/recipes/splunk_monitor.rb
+++ b/test/fixtures/cookbooks/test/recipes/splunk_monitor.rb
@@ -8,6 +8,6 @@ end
splunk_monitor '/var/log/httpd/access.log' do
inputs_conf_path "#{splunk_dir}/etc/apps/SplunkUniversalForwarder/default/inputs.conf"
sourcetype 'access_combined'
- index 'access_combined'
+ index 'my_custom_index'
only_if { ::File.exist?('/var/log/httpd/access.log') }
end
Change the index value in test/integration/client-resources/inspec/client_resources_spec.rb to match that used in [1].
Run the client-resources Test Kitchen suite.
kitchen verify client-resources-centos-7
:x: Actual Result
================================================================================
Recipe Compile Error in /opt/kitchen/cache/cookbooks/test/recipes/splunk_monitor.rb
================================================================================
Chef::Exceptions::ValidationFailed
----------------------------------
Option index must be equal to one of: "_internal", "access_combined", "access_combined_wcookie", "apache_error", "catalina", "cisco_syslog", "history", "linux_messages_syslog", "linux_secure", "log4j", "main", "os", "postfix_syslog", "rabbitmq", "sample", "shared_json", "splunklogger"! You passed "my_custom_index".
Cookbook Trace: (most recent call first)
----------------------------------------
/opt/kitchen/cache/cookbooks/test/recipes/splunk_monitor.rb:11:in `block in from_file'
/opt/kitchen/cache/cookbooks/test/recipes/splunk_monitor.rb:8:in `from_file'
Relevant File Content:
----------------------
/opt/kitchen/cache/cookbooks/test/recipes/splunk_monitor.rb:
4: file '/var/log/httpd/access.log' do
5: action :create_if_missing
6: end
7:
splunk_monitor '/var/log/httpd/access.log' do
9: inputs_conf_path "#{splunk_dir}/etc/apps/SplunkUniversalForwarder/default/inputs.conf"
10: sourcetype 'access_combined'
11>> index 'my_custom_index'
12: only_if { ::File.exist?('/var/log/httpd/access.log') }
13: end
14:
System Info:
------------
chef_version=16.9.32
platform=centos
platform_version=7.9.2009
ruby=ruby 2.7.2p137 (2020-10-01 revision 5445e04352) [x86_64-linux]
program_name=/opt/chef/bin/chef-client
executable=/opt/chef/bin/chef-client
Running handlers:
[2021-09-22T18:49:28+00:00] ERROR: Running exception handlers
Running handlers complete
[2021-09-22T18:49:28+00:00] ERROR: Exception handlers complete
Chef Infra Client failed. 4 resources updated in 03 seconds
[2021-09-22T18:49:28+00:00] FATAL: Stacktrace dumped to /opt/kitchen/cache/chef-stacktrace.out
[2021-09-22T18:49:28+00:00] FATAL: Please provide the contents of the stacktrace.out file if you file a bug report
[2021-09-22T18:49:28+00:00] FATAL: Chef::Exceptions::ValidationFailed: Option index must be equal to one of: "_internal", "access_combined", "access_combined_wcookie", "apache_error", "catalina", "cisco_syslog", "history", "linux_messages_syslog", "linux_secure", "log4j", "main", "os", "postfix_syslog", "rabbitmq", "sample", "shared_json", "splunklogger"! You passed "my_custom_index".
✔️ : Desired Result
-----> Starting Test Kitchen (v3.0.0)
-----> Verifying <client-resources-centos-7>...
Detected alternative framework tests for `inspec`
Loaded tests from {:path=>".home.vnaipaul.git.chef.chef-splunk.test.integration.client-resources.inspec"}
Profile: tests from {:path=>"/home/vnaipaul/git/chef/chef-splunk/test/integration/client-resources/inspec"} (tests from {:path=>".home.vnaipaul.git.chef.chef-splunk.test.integration.client-resources.inspec"})
Version: (not specified)
Target: docker://133780a2087eb47e0ede8ced85b0ba46f648e7ae5b536c70190987c5b3ae126d
✔ Custom Resources: Verify custom resources provided by this cookbook
✔ File /opt/splunkforwarder/etc/apps/bistro-1.0.2 is expected to exist
✔ File /opt/splunkforwarder/etc/apps/bistro-1.0.2 is expected to be directory
✔ Command: `/opt/splunkforwarder/bin/splunk btool --app=bistro-1.0.2 app list` exit_status is expected to eq 0
✔ Command: `/opt/splunkforwarder/bin/splunk btool --app=bistro-1.0.2 app list` stdout is expected not to match /disabled\s*=\s*(0|false)/
INI /opt/splunkforwarder/etc/apps/SplunkUniversalForwarder/default/inputs.conf
✔ ["monitor:///var/log/httpd/access.log", "index"] is expected to cmp == /my_custom_index/
✔ ["monitor:///var/log/httpd/access.log", "sourcetype"] is expected to match /access_combined/
Profile Summary: 1 successful control, 0 control failures, 0 controls skipped
Test Summary: 6 successful, 0 failures, 0 skipped
Finished verifying <client-resources-centos-7> (0m6.63s).
-----> Test Kitchen is finished. (0m7.80s)
:speaking_head: Foreword
The _splunkmonitor custom resource creates
monitor://stanzas in theinputs.conffile, with various optional properties. One of the properties, index, is validated against a fixed list of seemingly site-specific index names defined in the resource itself, _builtinindexes.💔 Problem Statement
The _splunkmonitor resource fails when given a valid (for Splunk) index name that is not in the _builtinindexes list.
The code does seem to work as intended, it just unnecessarily limits the usefulness of the resource in deployments with custom indexes.
:grey_question: Possible Solution
Drop use of the _builtinindexes list to validate the index property of the _splunkmonitor resource, and adopt the regex validation method used by the _splunkindex resource instead (but accept a leading underscore).
And maybe separately, update the regex used to validate the index property for both the _splunkindex and _splunkmonitor resources, to align better with the Splunk (custom) index name requirements:
:arrow_heading_up: Describe alternatives you've considered
Use the
chef-splunk::clientrecipe to renderinputs.confinstead of generating it with the _splunkmonitor resource.:heavy_plus_sign: Additional context
Custom index naming requirements are the same at the latest iterations of v6, v7, v8 Splunk Enterprise.
https://docs.splunk.com/Documentation/Splunk/6.6.12/Indexer/Setupmultipleindexes https://docs.splunk.com/Documentation/Splunk/7.3.9/Indexer/Setupmultipleindexes https://docs.splunk.com/Documentation/Splunk/8.2.2/Indexer/Setupmultipleindexes
The _splunkmonitor resource defaults the index property to
_internal.mainmight be a better default, per https://docs.splunk.com/Documentation/Splunk/8.2.2/Admin/Inputsconf, but that could be considered a separate issue.:pancakes: Cookbook Version
Version: 7.2.2
:woman_cook: Chef-Infra Version
Version: 16.9.32
🎩 Platform Details
CentOS 7.9
Steps to Reproduce
Edit
test/fixtures/cookbooks/test/recipes/splunk_monitor.rb: change the index value (to some value not in the _builtinindexes list fromresources/splunk_monitor.rb).Change the index value in
test/integration/client-resources/inspec/client_resources_spec.rbto match that used in [1].Run the client-resources Test Kitchen suite.
:x: Actual Result
✔️ : Desired Result