2020 ChatOps User Survey Results

[blag]

Question 2

Question 3

Question 4

Question 5

Question 6

Other responses:

• Busy with other priorities

• We use a Matrix based chat platform which I don't think StackStorm supports. We also have limited resources to devote just to implement chatops

• Not convinced ChatOps in general is a good operational model

• We are using Microsoft Teams. And we have a barrier of house rule about security. When we use chatbot, we have to break the barrier.

• Terrible chat tools.

Question 7

What chat application/s do you use that are not supported?

Microsoft Teams

Question 8

Question 9

Question 10

Other responses:

• Operation

• Change control ; Information Access for field crews

• Helpdesk

• Trigger custom application workflows on demand that can reduce toil

• Company-wide incident management, and intra-team assistance.

Question 11

Other responses:

• Comunication by StackStorm

• Disabled pending an access control solution.

• Mainly reporting triggered actions into Slack, quasi-logging style.

• Notifications

• Inquiries and responding via slack

• Working on integrating our own BotKit framework so we can utilize Slack forms with action-aliases.

Question 12

Other responses:

• Provide simplistic access to items in 3rd party APIs - eg. who is oncall in PD

• OpsGenie notifications / stage database refresh and app restart

• Start video meetings, create documents from a template, page teams by name, manage alias notifications (on/off)

• Allow tier 1 support to run an Ops process (an ansible playbook) that updates config at a customer location.

Question 13

Question 14

Question 15

Summary: Slack and Microsoft Teams

Question 16

Other responses:

• More straightforward/easier MSTeams

• If not RBAC, at least some kind of manual user whitelist

• Interactive aliases

• Utilize / (slash) commands in slack rather than though hubot

• Better chatops support for Webex Teams, EWC released to the wild and a key to support the installation

• Better control over output (paging, maximum message size, text formatting respected for paged output.)

• MS Teams

• Permitting and restricting different aliases per Slack channel (may get this with better RBAC integration)

Question 17

Question 18

What does StackStorm get right with its existing ChatOps implementation?

1. Slack integration and configuration
2. User-defined ChatOps is useful
3. It's easy to write and run action aliases
4. ChatOps and workflow notifications are nice
5. ChatOps just works
6. Modularity
7. Notifications and customization of action aliases
8. Chat provider agnostic
9. The ability to interact with ChatOps in the middle of a workflow and respond via ChatOps
10. Action aliases make it easy to be DRY

Question 19

What does StackStorm get wrong with its ChatOps implementation?

1. No RBAC for ChatOps
2. Inquiries are difficult, and interactivity is limited
3. Microsoft Teams configuration is difficult and integration is not good
4. Third party plugins are sometimes unmaintained or unresponsive, which makes it difficult to get pull requests merged to fix bugs in hubot-stackstorm.
5. ChatOps should be written in Python
6. Action alias formatting isn't simple and robust
7. Lack of two-way communication with the user (trying to remember formatting is difficult for some users)

Question 20

Is there anything we should do to make it easier to use ChatOps with StackStorm?

1. Support RBAC in ChatOps, or a simple "user whitelist" access control at a bare minimum
2. Improve quality and quantity of examples and documentation; possibly a beginner's guide
3. Rewrite ChatOps in Python
4. Better Microsoft Teams integration
5. Better support for interactivity, including inquiries
6. Better Webex Teams support
7. Auto-parse actions so any action can be used without special config
8. Allow for inquiries to be sent to users to gather the information [note: I believe this refers to when ChatOps users don't know a full command, so ChatOps should query for the missing commands]

Question 21

Is there anything we absolutely should NOT do regarding ChatOps in StackStorm?

1. Inquiries are complex

2. Don't drop support for ChatOps! This is a main driver of ST2 within an organization

3. Do not use BotFramework for anything else. Microsoft Teams is bad enough.

Question 22

Would you be willing to help us refactor, redesign, develop, or test new ChatOps features?

Summary: Mostly yes.

Question 23

Would you be willing to have a deeper discussion with us about your ChatOps use case/s?

Summary: Half yes, half no.

[arm4b]

Based on users feedback we know that people really want the StackStorm ChatOps RBAC.

There is more new data that was interesting to find:

• ChatOps is known to be used in relatively small groups. However data reported that ChatOps is starting to be adopted in 1K or bigger orgs. I mean this is happening. Previously almost everyone from the bigger corps I've talked with told "It's a toy, it doesn't work" about it. With improvements like better Security, RBAC, channel restrictions and more we could make the adoption easier for the bigger players to use StackStorm ChatOps. This could be a new driving factor getting the corp support.
• There is still a huge number of people/orgs "Who never tried ChatOps". That's promising as new potential group to explore.
• NLP idea didn't find good support. I guess It's hard to reliase NLP chatops benefits in infra/operations context where clear command instructions are essential.
• Microsoft Teams is more important than we think. There were a lot of pain points on this side with the st2chatops due to technical reasons. Wondering how that could be improved.
• Surprisingly, ChatOps sometimes could be the main driver for using ST2 as a platform. Digging deeper into Analytics, - chatops was the major traffic source for docs.stackstorm.com for many years. It's a shame we couldn't use that properly during some past years.
• I also noticed that really good number of people are happy to help us moving the ChatOps story forward: refactor, redesign, develop, test. This is very inspiring and I hope we could engage chatops advocates from StackStorm Community more.

@StackStorm/maintainers @StackStorm/contributors What are other's observations based on this this Survey? Please share.