Passing user context around to rules, sensors and actions

[lakshmi-kannan]

Context

User scoped variables are a new feature in st2. Currently, users can refer to an item in key value store using {{system.foo}} syntax. We want to support {{user.foo}}. The context of who the user is straight forward when actions are run manually (via CLI or UI) but not so obvious when automations are kicked off via rules. Read further.

Related: https://github.com/StackStorm/st2/pull/2669

What?

The problem is pretty simple - User scoped variables cannot be used anywhere in content if the event entering the StackStorm system is not kicked off manually by the user (CLI or UI command to run an action/workflow).

Rules is a good example. There is no notion of "user" when a rule is enforced. The rule is enforced by the "system". This means rules cannot use user scoped variables. As an extension, actions/workflows on the south side of rules can't access them either. We want to address that.

Why?

If we allowed user scoped variables only as values to action parameters when run manually, we will have a serious UX problem. Users will be confused where they can use user scoped variables and where they cannot.

How?

@manasdk, @dzimine and I sat together and discussed some options. To solve the problem, we need to create a concept of "user context".

@manasdk had a suggestion that we can create a "system_user" concept (usually stanley). stanley will need to be valid entry with valid credentials PAM/LDAP/Random_auth_provider. So when people use {{user.foo}} in rule, we'll resolve that to scope=user name=stanley.foo and use the corresponding value from key value store.

I wanted to solve it a similar way but I also want an ability to specify explicitly the user context as a field in rules. So in my model, the rule will look like

---
name: "notify"
pack: "chatops"
enabled: true
description: "Notification rule to send results of action executions to stream for chatops"
trigger:
  type: "core.st2.generic.notifytrigger"
criteria:
  trigger.route:
    pattern: "hubot"
    type: "equals"
action:
  ref: chatops.post_result
  parameters:
    channel: "{{trigger.data.source_channel}}"
    user: "{{trigger.data.user}}"
    execution_id: "{{trigger.execution_id}}"
context:
    run_as: lakshmi  # Should be an authenticatable user in st2 land

The concept of run_as opens up an interesting problem while providing more flexibility. RBAC is a problem that we need to address if we introduced run_as. A rule owner cannot specify an arbitrary user as run_as (unless they are admin). So a rule author can only name themselves as run_as (provided they already have permissions to create the rule.). So in theory this should be possible to implement, st2ctl poses another problem. Today we let anyone register content. This means we lose the idea of who owns the rule. We don't have this problem when authoring rules via API. One potential option is to disable st2ctl from being accessed by non-admins via file permissions so regular users would always use the API thereby RBAC can be enforced.

@dzimine wants more (like usual!). My idea was that rules being the connecting link between triggers and actions, it was a good place to throw the user context. One other reason is that the incoming events and outgoing actions can work with entirely different auth systems which don't have a corresponding mapping in st2 auth land. For example, AWS users need not mirror LDAP users one-to-one. You can say the same thing about incoming triggers (new relic or nagios or ceilometer). So dzimine wants to define the user context based on incoming triggers as well. In his mental model, an incoming trigger would call out the user (optional). Rule can now specify run_as user as a dynamic user. An example rule:

---
name: "notify"
pack: "chatops"
enabled: true
description: "Notification rule to send results of action executions to stream for chatops"
trigger:
  type: "core.st2.generic.notifytrigger"
criteria:
  trigger.route:
    pattern: "hubot"
    type: "equals"
action:
  ref: chatops.post_result
  parameters:
    channel: "{{trigger.data.source_channel}}"
    user: "{{trigger.data.user}}"
    execution_id: "{{trigger.execution_id}}"
context:
    run_as: {{trigger.user}}  # XXX: ``user`` should have a valid mapping in st2 auth land

The problem with doing this is that we need to maintain a map of this external system user to an authenticatable user in st2 land. The mapping will be provided by admin via a YAML. Note that this just means that we can use the variables of the st2 user. It DOES NOT mean you can run the action or workflow as that user. So I don't think this solution is complete. I see the need for two users 1. Incoming triggers with some user 2. Outgoing actions with some user. Also note that there needs to be mapping of those different users to an actual authenticatable st2 user. To give an example, see the rule below:

---
name: "notify"
pack: "chatops"
enabled: true
description: "Notification rule to send results of action executions to stream for chatops"
trigger:
  type: "core.st2.generic.notifytrigger"
criteria:
  trigger.route:
    pattern: "hubot"
    type: "equals"
action:
  ref: chatops.post_result
  parameters:
    channel: "{{trigger.data.source_channel}}"
    user: "{{trigger.data.user}}"
    execution_id: "{{trigger.execution_id}}"
context:
    trigger_user: "{{input_user_map.get(trigger.user)}}"  # Resultant should have a valid mapping in st2 auth land
    action_user: "{{input_output_user_maps.get(trigger.user)}}" # Resultant user should be a valid mapping in st2 auth land. 

Please don't focus on the jinja syntax but rather on the idea.

[manasdk]

an incoming trigger would call out the user (optional).

I do not understand how a TriggerInstance can have a user context. If the TriggerInstance has a user property then that is part of the opaque schema of the TriggerInstance however StackStorm itself does not need to provide any semantic meaning to this concept. Point being any property coming from the payload of a TriggerInstance could be designated by the Rule author as a user making StackStorm unaware of this concept.

[Kami]

The concept of run_as opens up an interesting problem while providing more flexibility. RBAC is a problem that we need to address if we introduced run_as. A rule owner cannot specify an arbitrary user as run_as (unless they are admin). So a rule author can only name themselves as run_as (provided they already have permissions to create the rule.). So in theory this should be possible to implement, st2ctl poses another problem. Today we let anyone register content. This means we lose the idea of who owns the rule. We don't have this problem when authoring rules via API. One potential option is to disable st2ctl from being accessed by non-admins via file permissions so regular users would always use the API thereby RBAC can be enforced.

We have been going back on forth on this many, many times. I thought we all agreed on this in the past - that's a feature (of the infrastructure as code approach) and not a bug.

We have two ways to manipulate the system - via on disk content (infrastructure as code approach) and via API (breaks infrastructure as code approach in some situations when using operations which mutate state, but some users are fine with that).

As far as the API goes, if we need more flexibility and also want to allow non-admin users to specify different run_as user, we could also introduce another PermissionType (e.g. rule_action_run_delegate or similar).

The problem with doing this is that we need to maintain a map of this external system user to an authenticatable user in st2 land. The mapping will be provided by admin via a YAML. Note that this just means that we can use the variables of the st2 user. It DOES NOT mean you can run the action or workflow as that user. So I don't think this solution is complete. I see the need for two users 1. Incoming triggers with some user 2. Outgoing actions with some user. Also note that there needs to be mapping of those different users to an actual authenticatable st2 user. To give an example, see the rule below:

We discussed this in the past and I do think that having a user context in TriggerInstance would be valuable (ff nothing else, for the audit purposes).

As far as the mapping goes - ideally we could outsource this and this would be users problem. I do know that this won't always work (e.g. would work fine for custom webhooks, less so for sensors) so maybe we don't have an option.

In any case, no matter which approach we chose (we allow user to provide a file with mappings) or the user itself needs to provide a valid st2 user in the TriggerInstance, this won't be straight-forward and cheap for the end user, but I can't think of a better approach right now (user will probably need some custom code which synchronizes and maintains the mapping).

As far as the bigger picture goes - I think we will need to go with something similar to what @dzimine has suggested (and we might need rule run_as as an addition / supplemental thing to a different, more dynamic approach). Rule approach is indeed simpler (and simpler is always better), but value in the rule is static so I don't think this adds much value and it doesn't solve the original problem.

In most cases I can think of, you really want this value to be dynamic and infer it from an incoming trigger / event (e.g. with GithubPushEvent, you probably want to use the user who pushed a change, etc.).