ChatOps Roadmap Meeting (17 Nov 2020)

[blag]

One-off @StackStorm/tsc and @StackStorm/contributors 1 hour meeting will take place on Tuesday, 17th Nov 2020, 09:30 AM US Pacific.

JOIN ON ZOOM. Meeting guidelines described in #33.

Host: @blag

Meeting Agenda

We have gathered some community feedback on our ChatOps roadmap for how to rewrite ChatOps in Python for better integration into the rest of StackStorm. This meeting will be focused on deciding which Python-based chat framework we want to use (Errbot, OpsDroid, or something else). In the unlikely even that we have extra time we will start charting the roadmap for the ChatOps switchover.

• Is there any guidance from the ChatOps User Survey results?
• What are the licensing restrictions for each option?
  • Compatible with Apache 2?
  • License exceptions for plugins?
• What chat providers does each framework offer?
  • Official chat provider support?
  • Unofficial chat provider support?
• What features does each framework offer?
• What does the Python version support look like?
  • Is anything already packaged for currently supported distributions (eg: Ubuntu 16 & 18, and Cent OS 7 & 8)?
• What does the switchover/replacement roadmap look like?

[m4dcoder]

@blag Do you currently have a proposal for some of these questions? Maybe document your thoughts here for the team to research & align with?

[blag]

Background

2020 ChatOps User Survey

Click to expand

ChatOps is a majorly used feature of StackStorm, according to the results of our 2020 ChatOps User Survey Results (question #5):  One of the major blockers is that ChatOps does not have RBAC (question #6):  Most users just use action aliases to define commands (question #11):  The vast majority of StackStorm users integrate ChatOps with Slack, although MS Teams is used more than we would like or realize (questions 14 and 15):   The top three highly requested features are: RBAC, more interactivity, and better integration (questions 16 and 17): 

Conclusions

Click to expand

Most users find that ChatOps are important, but they want more from it. st2chatops right now works just fine, but does not have the deep integration with the rest of StackStorm that it could have, nor does it support more interactive features of chat platforms. To provide the most value to users, we should focus on pushing integration with both StackStorm and chat providers to the fullest extent reasonably possible

Comparison of Alternatives

Alternatives:

• err-stackstorm
• OpsDroid

Chat Provider Support

Click to expand

| Chat Service | ST2chatops w/ Hubot | ErrBot with err-stackstorm | OpsDroid with custom plugin | |---------------------------|-----------------------------------------------------------|-------------------------------------------------------------------------------------------------------|------------------------------------------------------------------------| | Slack | :heavy_check_mark: Official | :heavy_check_mark: included in core | :heavy_check_mark: included in core | | Microsoft Teams | :white_check_mark: Official (awkward use of BotFramework) | :grey_question: https://github.com/errbotio/errbot/issues/1239
:white_check_mark: via BotFramework | :x: Needs a connector
BotFramework doesn't play well with OpsDroid | | Mattermost | :heavy_check_mark: Official (v5) | :white_check_mark: [external backend](https://github.com/Vaelor/errbot-mattermost-backend) | :heavy_check_mark: included in core | | Rocket.Chat | :heavy_check_mark: Official | :white_check_mark: [external backend](https://github.com/cardoso/errbot-rocketchat) | :heavy_check_mark: included in core | | Webex Teams (Cisco Spark) | :heavy_check_mark: Official for Cisco Spark | :white_check_mark: [external backend for Webex Teams](https://github.com/marksull/err-backend-cisco-webex-teams) | :heavy_check_mark: included in core | | | | | | | Flowdock | :white_check_mark: Provided but unsupported | :x: https://github.com/errbotio/errbot/issues/169 | :x: | | XMPP | :white_check_mark: Provided but unsupported | :heavy_check_mark: included in core | :x: | | IRC | :white_check_mark: Provided but unsupported | :heavy_check_mark: included in core | :x: |

Python Support

Click to expand

| Python Version | ErrBot with err-stackstorm | OpsDroid with custom plugin | |---------------------------|-------------------------------------------------------------------------------------------------------|------------------------------------------------------------------------| | 3.6 | ✅ | ✔️ [Versions >0.19.0 won't support Python 3.6 anymore](https://github.com/opsdroid/opsdroid/releases/tag/v0.19.0) | | 3.7 | ✅ | ✅ | | 3.8 | ✅ [Tested](https://github.com/errbotio/errbot/blob/89c2fb2a6399739f8f9c97fdfce6b0414fff2150/tox.ini#L2) but [not advertised yet](https://github.com/errbotio/errbot/blob/master/setup.py#L135) | ✅ |

Feature Support

Click to expand

| Chat Service | ST2chatops w/ Hubot | ErrBot with err-stackstorm | OpsDroid with custom plugin | |---------------------------|-----------------------------------------------------------|-------------------------------------------------------------------------------------------------------|------------------------------------------------------------------------| | ChatOps auth mechanism (required for RBAC) | :x: | ✅ | ??? | | Abstractions for advanced chat provider features | ✅ (VERY minimal) | ✅ | ??? | | High Availability | :x: | :x: [Upstream discussion but no implementation](https://github.com/StackStorm/discussions/issues/8#issuecomment-617870081) | :x: [No intention of upstream support yet](https://github.com/StackStorm/discussions/issues/8#issuecomment-617870081), [Upstream discussion](https://github.com/opsdroid/opsdroid/issues/299) |

Additional Considerations

• How complex do we want to make bots? Historically we have tried to keep them simple, but any discussion of HA will make them more complex. Additionally, the more lightweight we make the ChatOps integration, the more processing we pawn off onto the other StackStorm components.

[nzlosh]

errbot does support Python3.8, it looks like the setup.py needs updating. Errbot main branch is tested against Python3.6, 3.7 and 3.8 https://github.com/errbotio/errbot/blob/89c2fb2a6399739f8f9c97fdfce6b0414fff2150/tox.ini#L2

[arm4b]

Errbot GPL-3 license related:

• License: https://github.com/errbotio/errbot/blob/master/COPYING
• Exceptions file (do we need to be included in there?): https://github.com/errbotio/errbot/blob/master/gplv3-exceptions.txt
• Errbot 2016 License Discussion (insightful): https://github.com/errbotio/errbot/pull/762#issuecomment-223808352

[blag]

Follow Ups

• @armab
  • [ ] Talk to Linux Foundation lawyers about distributing Errbot (GPLv3) packages alongside st2chatops/err-stackstorm (Apache 2) packages
• @amanda11
  • [ ] Ask around about whether or not GPLv3 license of Errbot will hurt ChatOps adoption
• @blag
  • [x] Architecture diagram - see this blog post on Medium (slightly out of date since it includes PostgreSQL for Mistral, but the rest is accurate)
  • [ ] Ask Errbot developers about GPLv3 exception intent
  • [x] Talk to OpsDroid author to estimate difficulty with integrating with MS Teams - thank you @cognifloyd!

Discussion Details

• Errbot seems more mature than OpsDroid
  • Wider chat platform support, hopefully fewer bugs
  • Neither support Slack blocks yet
  • Errbot implements an abstraction mechanism to help users write a single format that then gets written out specifically for each chat platform
  • We will need a punchthrough to allow users to explicitly specify the formatting for different chat providers
  • err-stackstorm supports a punchthrough
  • BotFramework and OpsDroid event loops conflict, would need to rely on a non-BotFramework adapter for MS Teams
• Python 3 point version support is not a deciding factor
• Keeping compatibility with existing supported chat providers is important, including MS Teams
• Keeping compatibility with existing not-supporting-but-still-bundled chat providers (Flowdock, XMPP, IRC) is not a priority
• Tighter integrations with StackStorm would be:
  • ChatOps-specific RBAC controls
  • Builds on top of ChatOps authentication
  • Limiting permutations of users/channels/actions
  • Writing ChatOps state to StackStorm's existing MongoDB
  • Caching in StackStorm's Redis (optional, if StackStorm is configured with Redis)
• ChatOps auth can be implemented one of two ways
  • Current implementation in err-stackstorm
  • Similar to OAuth flow (ChatOps user requests a verification token, logs into StackStorm and provides verification token)
  • Requires user to be on their company VPN to authenticate their ChatOps user to StackStorm
  • Alternative implementation building on RBAC/LDAP mappings
  • StackStorm admins map StackStorm users to an immutable chat provider ID
  • ChatOps would then provide this to StackStorm when initiating executions
• ChatOps could be implemented as a very thin sensor, and aliases could be implemented as a special type of rule
  • Easier to code
  • Relies on pre-existing StackStorm infrastructure
  • Make it more difficult for users to develop/test/track aliases
  • Not a priority yet

Conclusions

• Action items need to be completed before we can continue discussion
• Will need to have another ChatOps meeting before we reach a final conclusion

[dzimine]

Folks, GPL is no-go:

While there are ways to make it work with StackStorm opensource, it makes StackStorm toxic for anyone who considers basing an installable product on top of it. Noone will be able to add StackStorm to their commercial offerings.

This - deferring companies who want to build a product powered by StackStorm - will be a terrible blow for us. It is vendors who embed stackstorm who are most likely to contribute to the core - what we miss the most. We already have some - HP (ex Plexxi), or a new company orchestral.io, we might turn them into contributors, and we need more because _this is the kind of companies who will most likely.

[blag]

@dzimine I think we have a chicken and egg problem with attracting outside contributors. We want ChatOps to be a well developed "killer app" for StackStorm to attract users, some of whom will turn into contributors. But we also need contributors to keep the StackStorm project healthy. So which one do we attract first?

Frankly, I don't see us attracting corporate contributors much more than we really already have any time soon. The StackStorm community has lost the corporate outreach and business connections from Extreme, and to my knowledge we don't really have any salespeople working on StackStorm. The results of the 2020 StackStorm ChatOps Users Survey seem to be evidence of this point - we really didn't see any resellers represented in those results. If we do have corporate resellers and they are simply being silent on the direction of ChatOps, I think that tells us that they aren't reselling ST2 ChatOps. We have also been pretty clear that we need donations from the community, and I would think any resellers would take that pretty seriously since they are basing their product on our code.

If we never attract corporate resellers then the licensing issue is entirely moot.

I think there's a real risk of st2chatops dying on the vine unless we invest significant resources into improving st2chatops integration with the rest of StackStorm (eg: RBAC for ChatOps). If corporate resellers aren't willing to get involved in the direction of the project, or devote resources towards our upstream project, then I really don't have a lot of empathy for them regarding GPL licensing. They need to get and remain involved in StackStorm, or they have to accept the direction we decide to take the project.

Finally, if corporate resellers do devote development resources to st2chatops directly, instead of trying to maintain their own fork, they will not be subject to the GPLv3 anyway. The GPL only applies if/when they distribute their own version of Errbot, or if they bundle Errbot itself into their package. Thanks to the exception clause in Errbot, merely writing an Errbot plugin does not mean that the plugin must be GPLed. The GPL would only apply when they go to distribute their version of st2chatops with Errbot already bundled in. Now, perhaps even this is too much of a risk for them and they wouldn't do that, but they could still resell and distribute their own fork of StackStorm without st2chatops and they would remain free of the GPL obligations.

To sum up, I don't see the point in already catering to potential future resellers, or to current resellers who aren't already involved in the development process. If they want to improve the current JavaScript based st2chatops, they're more than welcome to, but the TSC subcommittee working on ChatOps thinks that with our current development resources the best solution for the StackStorm project is to switch to using err-stackstorm. It is unacceptable to me for them to dictate how st2chatops is developed without being involved in the process so if those people already exist, let them speak now or forever hold their peace.

[amanda11]

As a contributor, I wanted to add that I had raised concerns about the GPL license at the ChatOps meeting and how it might be an issue for some companies. I did promise I'd explore further, and I'm still exploring the consequences. I had paused further investigation, as at the last TSC meeting we seemed to state that we were trying to explore non-GPL options, but I'll resume to see if I can add to discussion.

At the moment StackStorm bundles hubot. If StackStorm were to bundle errbot then would that not have license consequences on StackStorm? Or would we have to avoid it, and therefore loose our single installation. If st2chatops ends up being GPL, then it does affect StackStorm as a whole, and I think being able to only use StackStorm without chatops would lessen it's advantage over other offerings.

[nzlosh]

My understanding is the GPL3 licence can only be imposed on work derived from it. Since errbot would be conveyed in an unmodified form it would not require other software in the st2chatops package to use the GPL3 because they are independent works. https://www.gnu.org/licenses/gpl-faq.html#GPLCompatInstaller.

[amanda11]

This is where I am unclear, as that example was about an installer. So the installer doesn't have to be GPL but what about the software that relies on the GPL software.

There was arguments before on forums (and not necessarily resolved in agreement!) about a separate UI that was for a GPL CLI program. As the UI didn't stand up by itself as a product - therefore did it also have to be GPL, and you could see st2chatops being similar scenario, if it relies on errbot.

But then I am not sure if the exception on plugins means that isn't true. ie does the copy left stop before the plug-in. So st2chatops if left of the plug-in - so not included.

But I am not a lawyer. And it sounded like we didn't get a definite answer from Linux Foundation.

But I would agree that it would put people off, as if nothing else they might look elsewhere rather than researching with lawyers what exactly they can and can't do.

Being able to use stackstorm but not chatops I think would be a big limitation.

[nzlosh]

usual disclaimer: I'm not a lawyer, so this is just my understanding of the situation

Software that relies on GPL3 work could be tainted by the licence based on how that software integrates with it. For example, I've read in forums that if code calls out to a REST API written with a GPL3 licence, the GPL3 does not apply to the calling code. (couldn't find the source it was too long ago). If the code is a plugin of the GPL3 work and there's significant sharing of data structures, it can be considered a single combined program and would be subject to having the GPL3 applied to it. https://www.gnu.org/licenses/gpl-faq.html#GPLPlugins

Based on that information, one would argue that err-stackstorm is a single combined program under the definition of the GPL3, however the authors of errbot wrote an explicit exemption for plugins permitting them to be allowed to use a licence other than the GPL3. https://github.com/errbotio/errbot/blob/master/gplv3-exceptions.txt. err-stackstorm is licenced using Apache-2.0

I hear what you and Dmitry are saying that the GPL3 is unacceptable for some use cases.

[amanda11]

I think this is the main problem with GPL, it gets unclear what does and does not cover derivative work, and that seems up to interpretation. Most think statically linking with a library is covered by GPL, differing opinions on dynamically linked, and then wider interpretation on other cases. Couldn't find definitive answer on calling GPL program, as you mentioned data structure sharing etc gets considered. The UI case would have been a distinct interface, but there wasn't agreement as to whether it was derivative or not.

I don't know if the errbot plug-in caveat is enough,it might be.

It all means some will just flat refuse GPL, as it's the easy option, rather than get legal departments to work out if their use case is ok or not.

[cognifloyd]

Following up Teams support in OpsDroid is WIP: https://github.com/opsdroid/opsdroid/pull/1679

[esarakaitis]

any teams users want to share their integration how-to?

[punkrokk]

@blag nice work