Feature Request: Standard mechanism to share python code within a pack

[tim-ireland]

Often times actions and sensors might need to share logic that would ideally not be duplicated within a pack, i.e., support DRY principles. Today, the action folder has the concept of a lib directory that is accessible to all actions to import from, however the sensors folder does not have access to this import location (in a standardized way).

First Example: When using 3rd party python bindings to establish a session to a remote api, the code for connection logic needs to be implemented in both the sensor which receives events and the actions which makes changes in the api. Ideally they would both import from the same shared module.

Second Example: Common functions and type definitions etc should be able to be shared between the different modules within a pack.

Suggestion: Move the lib folder location from within actions to the top level of the pack and have that be the standard way to share code within a pack.

[Kami]

Thanks for the feature request / proposal.

I do agree that something like that would indeed be useful (repeating and duplication is bad!). It's in fact something we have already talked about in the past, but we didn't have the chance to do it yet.

I personally like the "pack level" lib/ directory approach. We don't have an ETA for this feature yet, but if others agree that something like this would be useful, we should be able to do it relatively fast since it shouldn't require too many changes. There are some caveats (we also support copying lib/ directory over for remote runner actions and there are is some backward compatibility stuff we need to keep in mind), but it shouldn't be too bad.

[bri365]

@Kami any idea if this proposal is moving forward? Its beginning to account for a vast number of TODOs in our code :-)

[Kami]

@bri365 Hey, sorry, we still didn't get a change to look into this yet.

I will try to look into it and come up with some easy and simple 80% solution (e.g. maybe ignore remote runner to begin with and only solve it for python runner actions) once I'm done with improved pack configuration (probably some time next week).

In the mean time - another potential solution would be to distribute common code as a Python package which gets installed into your pack virtual environment (I do know that's not always the best approach though).

[bri365]

Thanks, @Kami, much appreciated! We do package up some of our shared code (mainly code shared between packs at this point).

[nmaludy]

+1

[ytjohn]

For me, I do create "common" code for my stackstorm actions and sensors. This code is often highly integrated with the stackstorm environment.

For example, building an integration style pack with rackhd. We already have a python client library that we package up as a python module. But our common stackstorm code is a wrapper around that - self.config to find the endpoints, implements caching in the stackstorm datastore, and formats some of the results to better suit the corresponding actions and sensors. It would make less sense for us to separate this common code into its own python module.

What would make the most sense for us would be to ship a pack level "lib" or "code" or "resources" directory, or alternatively, be able to modify the PYTHONPATH to include actions/lib and/or sensors/lib.

[timsk-ru]

+1

[arm4b]

Just a heads up, @lakshmi-kannan is implementing this incredibly useful feature. If you want to take a look, review or share more opinions, - please do in #3658.