Feature Request: support multiple, independently configured, sensor instances.

[cognifloyd]

Please add the ability to have one sensor running more than once with separate configuration. Others have already worked on this feature as (See PR #1739), but the PR was closed due to (I think) inactivity of @manasdk the initial author.

I am writing a PollingSensor where it would be very nice to have separate configuration for multiple instances of the same sensor. For instance, I have some number of servers with a service that need to be polled for changes. However some servers are with one hosting provider (one set of configuration), and some in a different hosting provider (a separate set of configuration). I would like one instance of my sensor for each hosting provider, and possibly one instance per server.

Though there are only a few of these servers today, that number could become elastic in the future. Also, it would be nice to have separate config for different environments (production vs testing vs dev) without maintaining separate instances of stackstorm. Plus that would allow me to have one environment (say dev) using a higher version of the service than another (say production) with different sensor instances to make sure integrations work as expected.

Also, I think it would make sense for the sensors in git, github, and other packs (aside: I wish bitbucket had a sensor) to allow watching multiple things with perhaps different credentials. Then if there are any issues with one sensor (say misconfiguration, credentials stop working, api version changes for one instance but not another, or something) it won't affect the other sensors.

[koenbud]

This feature is exactly what I am looking for too. I have sensors that do statistical analysis and I need to run multiple similar sensors, but with a different set of parameters. I want to avoid repeating code in multiple sensors.

I am new to StackStorm, so I am not fully familiar with the design principles, but one of the solutions that came to mind is to allow configuration parameters in the sensor metadata yaml file. In the sensor python script the parameter self._config would then contain a merge the pack and the sensor configuration, where the sensor configuration would take precedence. You can then create multiple sensors that point to the same python sensor script.

Example where I have added a configuration node:

---
  class_name: "SampleSensor"
  entry_point: "sample_sensor.py"
  description: "Sample sensor that emits triggers."
  configuration:
    foo: "bar"
    timeout: "60"
  trigger_types:
    -
      name: "event"
      description: "An example trigger."
      payload_schema:
        type: "object"
        properties:
          executed_at:
            type: "string"
            format: "date-time"
            default: "2014-07-30 05:04:24.578325"

[nmaludy]

+1

Semi-related is the "multi-tenant" / "multi-credential" issue in packs. Many integrations need to talk to multiple endpoints. This has been accomplished in the following packs (that i know of):

• vsphere supports multiple credentials in the configuration
• activedirectory supports multiple credentials in the configuration AND the option to override those credentials and pass them in as parameters to an action (so it can be used via the CLI, or potentially dynamically from the k/v store)

And i've raised an issue on the Jenkins pack to support a similar scenario based on a request form @romeoblade in Slack.

This seems like a common design pattern and might be interesting to see if we can reduce some of the code complexity in these packs by providing a centralized feature that accomplishes the same task?

If this is not related i can extract it out into another issue.

[AlexRezid]

+1 on this. I would require this on almost all packs. Windows, Linux, Datadog, PagerDuty...

I'll need to change this by myself if not globally implemented...

[daBONDi]

+1 Anything new on this field?

[ankittyagi20]

IMO, this is going to be a must have feature moving forwards, liked the idea @nmaludy of having a centralized feature for to deal with multiple configs with a single pack.

[LindsayHill]

this is going to be a must have feature

@ankittyagi20 you're welcome to contribute a PR, or at least a detailed design for how it could work. It needs some thought into how those configurations would be managed, where they would live (packs dir? config dir? which file?), how different sensors would know which configuration they should use, etc. Or should this be some sort of broader credential configuration management service, connected with some way of mapping packs/sensors/actions to that credential service. Or is it better to tie this with Vault, or something similar?

[djha-skin]

Possible solution -- If st2reactor were published onto pypi, I could simply create my own python module that inherited from the PollingSensor class. It would then be trivial to include that python module and use it in my sensor. So my sensor in my pack would inherit from UniversalSensor which would inherit from PollingSensor:

class MySensor(UniversalSensor):
    pass

That would work for me.

[punkrokk]

I would like to resurface this. Anyone thought about a design? @cognifloyd ?