Question - workflows with Splunk appinspect & slim?

[SID800]

Question on whats the best way of incorporating ksconf into a workflow for packing apps for Splunk Base/Cloud with AppInspect and SLIM?

Background we're been using on-prem Splunk for years with obviously tons of /local customization to addons/apps. /local is not allowed in Cloud and AppInspect will also fail. ksconf combine appears to be a great solution for the problem of keeping our code customization separate in Git, while also maintaining the vendor's original code base. Attempting few different implementations of using the tool, in conjunction with appinsect and slim have encountered blocking issues. Hopefully I've overlooked or missed something while learning these new processes.

1) First attempt was combine the entire addon into a release folder. SLIM would then fail due to the mismatched folder names. Not seeing anything in the SLIM manual for changing this behavior.

ksconf combine * --target=release

C:\Temp\TA_windows>slim package release
slim package: Packaging app at "release"
slim package: [NOTE] Generating app manifest for release...
  C:\Temp\TA_windows\release\README
slim package: [ERROR] App folder name "release" does not match App ID "Splunk_TA_windows"

folder layout

Splunk_TA_windows
-- _src
---- appserver
---- bin
---- default
---- lookups
-- my-org
---- default
-- release
---- appserver
---- bin
---- default
---- lookups

2) Next attempt was to follow the example folder structure outlined for combine (i.e. only the default folder mods are merged). Different problems. a) SLIM also ingested the default.d folder, so created a custom .slimignore with default.d/* Folder is still present in the package, but at least its empty and appinspect doesnt throw an error. b) AppInspect throws error for the presence of .ksconf_controlled

Splunk app packaging standards These checks validate that a Splunk app has been
correctly packaged, and can be provided safely for package validation.
    Check that the extracted Splunk App does not contain any directories or
    files that start with a ., or directories that start with __MACOSX.
        FAILURE: A prohibited file or directory was found in the
            extracted Splunk App: default\.ksconf_controlled

ksconf combine default.d/* --target=default slim package Splunk_TA_windows

Splunk_TA_windows
-- appserver
-- bin
-- default
-- default.d
---- _src
------ eventtypes.conf
------ props.conf
---- my-org
------ eventtypes.conf
------ props.conf
-- lookups

[lowell80]

I need a little bit of time to process this in more detail. Overall this sounds like exactly the kind of use case that ksconf was built for. (The idea is to do simple things well, be modular, and fit into many different kinds of workflows, such a build processes.)

One initial question that comes to mind is: Have you used slim much before for packaging or other management activities? Has it worked well, or is this the first use case you're attempting to use it for?

BTW, if the .ksconf_controlled thing is getting in the way, that can certainly be addressed. The stupid-fast workaround is just to delete the marker file after ksconf creates it; but the long-term solution is that I can easily add a command line option to prevent it from being created in the first place. The file is intended to be used for repeated merge scenarios, but in the case where ksconf is being entirely executed as part of a build process, there's no need for it.

I'm also looking at adding a packaging command to ksconf directly. It just hasn't happened yet.

[SID800]

I've been restraining myself to just make a build script (powershell / batch) to handle these various issues, along with keeping it flexible enough for others to use within the company and git friendly.

First time using SLIM. I've basically been doing this manually onprem for years (customize the addons for the CM/IDX, SH, and FWD to remove components not needed). Using it solely to be more standard/compatible with Cloud needs.

Yes, it would be helpful if there was an option to omit .ksconf_controlled :)

[lowell80]

My long term plans include #47. But in the meantime, I think v0.7.7rc3 will fit your immediate needs.

I've attempted to use slim on several occasions, but never got it doing the things I wanted, so I've kida given up on it. The future ksconf packaging command will be able to run the combine virtually (internally) and produce a tarball (.tgz/.spl) file on demand. That should make integration with AppInspect much easier.

In the meantime, an external packing creation process will be required. Which could be tar, slim or something similar.

[lowell80]

I think that the new ksconf package command in v0.8.x addresses all the issues discussed here. And if you need more, then please check out the build framework. Unfortunately the docs are still lagging on the build side of things, but hopefully some of the examples will be enough to give you some ideas. Glad for any additional feedback!