Modular Input Decorators

[micahkemp-splunk]

In an attempt to have similar design patterns to how custom search commands are created, and also to make the definition of the scheme, member access to configuration values, automatic casting of configuration types, and convenient .spec file generation, I implemented a proof of concept for decorators for the Script class. It mimics much of how the search command decorators are implemented, with changes where necessary. The functionality isn't fully complete, and is lacking validation of decorator parameters, but before spending a lot more time polishing and making it complete I wanted to get feedback regarding the willingness of the SDK team to merge such a PR, and also to get early feedback regarding which pieces the team would want implemented differently.

The summary of what this PR accomplishes is shown in this code snippet:

from splunklib.modularinput import Configuration, Script, Argument
import sys

@Configuration(use_single_instance=True)
class DevSDKScript(Script):
    a_string_argument = Argument(data_type=Argument.data_type_string, description="A String Argument")
    a_boolean_argument = Argument(data_type=Argument.data_type_boolean, description="A Boolean Argument")
    a_number_argument = Argument(data_type=Argument.data_type_number, description="A Number Argument")

    # for inputs that need to perform preliminary actions (preparation) with knowledge of the entire set of inputs
    def preflight(self, input_items, ew):
        for input_name, input_item in input_items:
            ew.log("INFO", "{0} preflight".format(input_name))

    # process_input is called once per configured input to be more convenient, not requiring the developer
    #  to implement the interation themselves
    # stream_events is still overrideable if the developer prefers the existing method
    def process_input(self, input_name, input_item, ew):
        ew.log("INFO", "{0} stream_event".format(input_name))
        ew.log("INFO", " a_string_argument: {0}".format(input_item.a_string_argument))
        ew.log("INFO", " a_boolean_argument: {0}".format(input_item.a_boolean_argument))
        ew.log("INFO", " a_number_argument: {0}".format(input_item.a_number_argument))

    # for inputs that need to perform post actions (cleanup) with knowledge of the entire set of inputs
    def postflight(self, input_items, ew):
        for input_name, input_item in input_items:
            ew.log("INFO", "{0} postflight".format(input_name))

if __name__ == "__main__":
    sys.exit(DevSDKScript().run(sys.argv))

[fantavlik]

Hi @micahkemp-splunk, right now the team is focused on addressing bug fixes and security updates so I'm not sure when an enhancement like this would be possible but could you break down what you are trying to address/make easier a bit more? This will help us determine the right priority. Also let us know if this is still something you are interested in pursuing, I know it's been nearly two years since this was put up.

For example, I think you're addressing at least four different usability problems:

• Decorators for easier specification of configuration
• Allowing users to define modular input arguments and validators similar to search commands such that argument parsing and validation is enforced by default by the Script class rather than requiring users to implement parsing and validation themselves
• Providing separate methods for each stage of input gathering - preflight, process_input, and postflight
• Enforcing that process_input is called only once per configured input: it would be great to understand how you intend to accomplish this or the issue you have with the current stream_events implementation