Open yaroslav-nakonechnikov opened 6 months ago
@yaroslav-nakonechnikov I think it's an expected behavior for this particular library. I believe it does not check for incorrect SPL syntax in values.
You might try using btool
to check for syntax errors in .conf
files.
btool is too heavy to run. and it also skips a lot of syntax issues.
also noticed: when it parses, it also trims lines. Which is fine, if there will be need to store file.
btool is too heavy to run. and it also skips a lot of syntax issues.
do you mind sharing an example of such behavior?
I can ask around but I never saw a library/tool to parse and validate .conf files.
sorry for late response.
example is quite easy: conf file is being created locally, without splunk installed. and installing btool - means to install splunk.
as well for ci pipelines, using splunk is heavy and consumes resources (time, cpu, storage, network)
and moreover - splunk containers are not consistent. we were trying to use as recent versions as possible, but from version to version - it breaks. Never was related to btool tool, but still not comfortable to see how new version breaks working setup.
so, i managed to run btool without splunk itself... binary looks working, but still - it has too many dependencies like SPLUNK_HOME just to start. then i had to copy libraries from $SPLUNK_HOME/lib, or do symlinks... but something, which took additional effort. I believe in some OSes it would be possible to install needed ones from official repos. In ubuntu 24.04 it wasn't possible for some.
and then i put file in $SPLUNK_HOME/etc/system/local/savedsearches.conf where i have issues with syntax - and it failes to detect:
~/splunkforwarder/bin$ ./btool --debug check
No spec file for: /home/user/splunkforwarder/etc/apps/SplunkUniversalForwarder/default/app.conf
Checking: /home/user/splunkforwarder/etc/apps/SplunkUniversalForwarder/default/default-mode.conf
Checking: /home/user/splunkforwarder/etc/apps/SplunkUniversalForwarder/default/health.conf
Checking: /home/user/splunkforwarder/etc/apps/SplunkUniversalForwarder/default/inputs.conf
Checking: /home/user/splunkforwarder/etc/apps/SplunkUniversalForwarder/default/limits.conf
Checking: /home/user/splunkforwarder/etc/apps/SplunkUniversalForwarder/default/outputs.conf
Checking: /home/user/splunkforwarder/etc/apps/SplunkUniversalForwarder/default/props.conf
Checking: /home/user/splunkforwarder/etc/apps/SplunkUniversalForwarder/default/server.conf
Checking: /home/user/splunkforwarder/etc/apps/SplunkUniversalForwarder/default/web.conf
No spec file for: /home/user/splunkforwarder/etc/apps/introspection_generator_addon/default/app.conf
Checking: /home/user/splunkforwarder/etc/apps/introspection_generator_addon/default/inputs.conf
Checking: /home/user/splunkforwarder/etc/apps/introspection_generator_addon/default/server.conf
Checking: /home/user/splunkforwarder/etc/apps/journald_input/default/authorize.conf
Checking: /home/user/splunkforwarder/etc/apps/journald_input/default/inputs.conf
No spec file for: /home/user/splunkforwarder/etc/apps/search/default/app.conf
Checking: /home/user/splunkforwarder/etc/apps/search/default/props.conf
Checking: /home/user/splunkforwarder/etc/apps/search/default/restmap.conf
Checking: /home/user/splunkforwarder/etc/apps/search/default/transforms.conf
Checking: /home/user/splunkforwarder/etc/apps/splunk_httpinput/default/inputs.conf
No spec file for: /home/user/splunkforwarder/etc/apps/splunk_internal_metrics/default/app.conf
Checking: /home/user/splunkforwarder/etc/apps/splunk_internal_metrics/default/props.conf
Checking: /home/user/splunkforwarder/etc/apps/splunk_internal_metrics/default/transforms.conf
No spec file for: /home/user/splunkforwarder/etc/manager-apps/_cluster/default/indexes.conf
Checking: /home/user/splunkforwarder/etc/system/default/alert_actions.conf
No spec file for: /home/user/splunkforwarder/etc/system/default/app.conf
Checking: /home/user/splunkforwarder/etc/system/default/audit.conf
Checking: /home/user/splunkforwarder/etc/system/default/authentication.conf
Checking: /home/user/splunkforwarder/etc/system/default/authorize.conf
No spec file for: /home/user/splunkforwarder/etc/system/default/conf.conf
Checking: /home/user/splunkforwarder/etc/system/default/default-mode.conf
Checking: /home/user/splunkforwarder/etc/system/default/federated.conf
Checking: /home/user/splunkforwarder/etc/system/default/field_filters.conf
Checking: /home/user/splunkforwarder/etc/system/default/global-banner.conf
Checking: /home/user/splunkforwarder/etc/system/default/health.conf
Checking: /home/user/splunkforwarder/etc/system/default/inputs.conf
Checking: /home/user/splunkforwarder/etc/system/default/limits.conf
Checking: /home/user/splunkforwarder/etc/system/default/livetail.conf
Checking: /home/user/splunkforwarder/etc/system/default/messages.conf
Checking: /home/user/splunkforwarder/etc/system/default/metric_alerts.conf
Checking: /home/user/splunkforwarder/etc/system/default/metric_rollups.conf
Checking: /home/user/splunkforwarder/etc/system/default/outputs.conf
Checking: /home/user/splunkforwarder/etc/system/default/procmon-filters.conf
Checking: /home/user/splunkforwarder/etc/system/default/props.conf
Checking: /home/user/splunkforwarder/etc/system/default/restmap.conf
Checking: /home/user/splunkforwarder/etc/system/default/server.conf
Checking: /home/user/splunkforwarder/etc/system/default/source-classifier.conf
No spec file for: /home/user/splunkforwarder/etc/system/default/telemetry.conf
Checking: /home/user/splunkforwarder/etc/system/default/transforms.conf
Checking: /home/user/splunkforwarder/etc/system/default/visualizations.conf
Checking: /home/user/splunkforwarder/etc/system/default/web-features.conf
Checking: /home/user/splunkforwarder/etc/system/default/web.conf
No spec file for: /home/user/splunkforwarder/etc/system/local/savedsearches.conf
so, still, library does it better even in current version.
thanks @yaroslav-nakonechnikov, your points are valid.
I did some homework and I see that VSCode extension for Splunk (https://github.com/splunk/vscode-extension-splunk) offers some syntax highlighting and linting capabilities, let us explore it next week. There are also some other extensions that offer similar functionality.
If that would be easy to incorporate into this library (or maybe another one) - I think we can go with it.
Hello,
with help of this library i've created simple script to validate conf files:
and in that script when file has syntax error - it thinks that all good.
example: as you see, empty space is there, but it should be wiped.
and it passes wrong multine search: