search

StackStorm / stackstorm-k8s

K8s Helm Chart that codifies StackStorm (aka "IFTTT for Ops" https://stackstorm.com/) Highly Availability fleet as a simple to use reproducible infrastructure-as-code app
https://helm.stackstorm.com/
Apache License 2.0
105 stars 107 forks source link

RBAC while Sensor accessing datastore #329

Closed anrajme closed 2 years ago

anrajme commented 2 years ago

Hi,

I have a fully working JIRA sensor that started failing after enabling RBAC on k8s-based HA implementation. 

2022-09-07 03:17:16,811 ERROR [-]   File "/opt/stackstorm/packs/jira/common/jira_init.py", line 80, in _get_ims_token
2022-09-07 03:17:16,811 ERROR [-]
2022-09-07 03:17:16,811 ERROR [-] auth_token_kvp = client.keys.get_by_name(name='app_auth_token', decrypt=True)
2022-09-07 03:17:16,811 ERROR [-]
2022-09-07 03:17:16,811 ERROR [-]   File "/opt/stackstorm/st2/lib/python3.6/site-packages/st2client/models/core.py", line 42, in decorate
2022-09-07 03:17:16,811 ERROR [-]
2022-09-07 03:17:16,811 ERROR [-] return func(*args, **kwargs)
2022-09-07 03:17:16,812 ERROR [-]
2022-09-07 03:17:16,812 ERROR [-]   File "/opt/stackstorm/st2/lib/python3.6/site-packages/st2client/models/core.py", line 335, in get_by_name
2022-09-07 03:17:16,812 ERROR [-]
2022-09-07 03:17:16,812 ERROR [-] instances = self.query(name=name, **kwargs)
2022-09-07 03:17:16,812 ERROR [-]
2022-09-07 03:17:16,812 ERROR [-]   File "/opt/stackstorm/st2/lib/python3.6/site-packages/st2client/models/core.py", line 42, in decorate
2022-09-07 03:17:16,812 ERROR [-]
2022-09-07 03:17:16,812 ERROR [-] return func(*args, **kwargs)
2022-09-07 03:17:16,812 ERROR [-]
2022-09-07 03:17:16,812 ERROR [-]   File "/opt/stackstorm/st2/lib/python3.6/site-packages/st2client/models/core.py", line 322, in query
2022-09-07 03:17:16,812 ERROR [-]
2022-09-07 03:17:16,812 ERROR [-] instances, _ = self._query_details(**kwargs)
2022-09-07 03:17:16,812 ERROR [-]
2022-09-07 03:17:16,812 ERROR [-]   File "/opt/stackstorm/st2/lib/python3.6/site-packages/st2client/models/core.py", line 315, in _query_details
2022-09-07 03:17:16,813 ERROR [-]
2022-09-07 03:17:16,813 ERROR [-] self.handle_error(response)
2022-09-07 03:17:16,813 ERROR [-]
2022-09-07 03:17:16,813 ERROR [-]   File "/opt/stackstorm/st2/lib/python3.6/site-packages/st2client/models/core.py", line 195, in handle_error
2022-09-07 03:17:16,813 ERROR [-]
2022-09-07 03:17:16,813 ERROR [-] response.raise_for_status()
2022-09-07 03:17:16,813 ERROR [-]
2022-09-07 03:17:16,813 ERROR [-]   File "/opt/stackstorm/st2/lib/python3.6/site-packages/requests/models.py", line 943, in raise_for_status
2022-09-07 03:17:16,813 ERROR [-]
2022-09-07 03:17:16,813 ERROR [-] raise HTTPError(http_error_msg, response=self)
2022-09-07 03:17:16,813 ERROR [-]
2022-09-07 03:17:16,813 ERROR [-] requests.exceptions
2022-09-07 03:17:16,813 ERROR [-]
2022-09-07 03:17:16,813 ERROR [-] HTTPError
2022-09-07 03:17:16,814 ERROR [-] :
2022-09-07 03:17:16,814 ERROR [-] 403 Client Error: Forbidden
MESSAGE: Decrypt option requires administrator access for url: http://myapp-st2api:9101/v1/keys/?name=app_auth_token&decrypt=True
2022-09-07 03:17:16,814 ERROR [-]
2022-09-07 03:17:16,814 ERROR [-] During handling of the above exception, another exception occurred:
2022-09-07 03:17:16,814 ERROR [-] Traceback (most recent call last):
2022-09-07 03:17:16,814 ERROR [-]   File "/opt/stackstorm/st2/lib/python3.6/site-packages/st2reactor/container/sensor_wrapper.py", line 449, in <module>
2022-09-07 03:17:16,814 ERROR [-]
2022-09-07 03:17:16,814 ERROR [-] obj.run()
2022-09-07 03:17:16,814 ERROR [-]
2022-09-07 03:17:16,814 ERROR [-]   File "/opt/stackstorm/st2/lib/python3.6/site-packages/st2reactor/container/sensor_wrapper.py", line 273, in run
2022-09-07 03:17:16,814 ERROR [-]
2022-09-07 03:17:16,814 ERROR [-] self._sensor_instance.setup()
2022-09-07 03:17:16,814 ERROR [-]
2022-09-07 03:17:16,814 ERROR [-]   File "/opt/stackstorm/packs/jira/sensors/jira_sensor_gcc.py", line 53, in setup
2022-09-07 03:17:16,814 ERROR [-]
2022-09-07 03:17:16,815 ERROR [-] self._jira_client = self._jira_auth.get_client()
2022-09-07 03:17:16,815 ERROR [-]
2022-09-07 03:17:16,815 ERROR [-]   File "/opt/stackstorm/packs/jira/common/jira_init.py", line 51, in get_client
2022-09-07 03:17:16,815 ERROR [-]
2022-09-07 03:17:16,815 ERROR [-] token=self._get_ims_token(),
2022-09-07 03:17:16,815 ERROR [-]
2022-09-07 03:17:16,815 ERROR [-]   File "/opt/stackstorm/packs/jira/common/jira_init.py", line 85, in _get_ims_token
2022-09-07 03:17:16,815 ERROR [-]
2022-09-07 03:17:16,815 ERROR [-] "Exception in retrieving value from datastore for key ipass_auth_token %s", e
2022-09-07 03:17:16,815 ERROR [-]
2022-09-07 03:17:16,815 ERROR [-] Exception
2022-09-07 03:17:16,815 ERROR [-] :
2022-09-07 03:17:16,815 ERROR [-] ('Exception in retrieving value from datastore for key ipass_auth_token %s', HTTPError('403 Client Error: Forbidden\nMESSAGE: Decrypt option requires administrator access for url: http://myapp-st2api:9101/v1/keys/?name=app_auth_token&decrypt=True',))
2022-09-07 03:17:16,815 ERROR [-]
2022-09-07 03:17:16,815 INFO [-] Stopping trigger watcher
2022-09-07 03:17:16,816 WARNING [-] Received method (50, 21) during closing channel 1. This method will be ignored

It fails while reading the datastore key-value pair due to the above 403 exception.

The sensor pack code reads the key-value from the data store. The code snippet is below. 

     if not self.auth_token:
            # self.auth_token = self.action_service.get_value(name='app_auth_token', decrypt=True)
            try:
                auth_token_kvp = client.keys.get_by_name(name='app_auth_token', decrypt=True)
                if auth_token_kvp:
                    self.auth_token = auth_token_kvp.value
            except Exception as e:
                raise Exception(
                    "Exception in retrieving value from datastore for key ipass_auth_token %s", e
                )

The sensor works fine if we disable the RBAC.

RBAC definitions. 

 assignments:
      # TIP: set files to an empty string to remove them (st2admin.yaml: "")
      st2admin.yaml: |
        ---
        username: st2admin
        roles:
          - system_admin
      service.yaml: |
        ---
        username: app-generic-user
        roles:
          - admin

    mappings:
      appdev.yaml: |
        ---
        group: "cn=APP_LDAP_ADMIN,ou=Groups,o=org"
        description: "admin role to core dev"
        roles:
          - "admin"
      observer.yaml: |
        ---
        group: "cn=APP_LDAP_USER,ou=Groups,o=org"
        description: "Observer role to the team"
        roles:
          - "observer"

Scope of the token is system by default

root@app-st2client-xxxxx:/opt/stackstorm# st2 key list +-----------------------------+-----------------------------+--------+-----------+--------------+------+------------------+ | name | value | secret | encrypted | scope | user | expire_timestamp | +-----------------------------+-----------------------------+--------+-----------+--------------+------+------------------+ | app_auth_token | 303030303032** | True | True | st2kv.system | | |

As per the documentation here - https://docs.stackstorm.com/datastore.html#storing-and-retrieving-key-value-pairs-via-cli 

> # NOTE: When RBAC is enabled, only admins can list key-value pairs scoped to
> # a different user. Regular users can only list key-value pairs scoped to
> # themselves.

But, I'm unsure how to enable this admin privilege for this sensor execution runtime. Also, not sure how this privilege is assigned to the sensor container. At the process level, it's running as st2 user and it doesn't have access by default to the mentioned API with/without RBAC. 

st2@app-st2sensorcontainer-xxxxxx-wftvr:/opt/stackstorm$ st2 key list
ERROR: 401 Client Error: Unauthorized
MESSAGE: Unauthorized - One of Token or API key required. for url: http://app-st2api:9101/keys/?decrypt=false&scope=all&limit=50

Logs from the st2-api pod

2022-09-12 07:01:24,750 ERROR [-] Failed to call controller function "get_all" for operation "st2api.controllers.v1.keyvalue:key_value_pair_controller.get_all": Decrypt option requires administrator access
Traceback (most recent call last):
  File "/opt/stackstorm/st2/lib/python3.8/site-packages/st2common/router.py", line 632, in __call__
    resp = func(**kw)
  File "/opt/stackstorm/st2/lib/python3.8/site-packages/st2api/controllers/v1/keyvalue.py", line 173, in get_all
    self._validate_decrypt_query_parameter(
  File "/opt/stackstorm/st2/lib/python3.8/site-packages/st2api/controllers/v1/keyvalue.py", line 506, in _validate_decrypt_query_parameter
    raise AccessDeniedError(message=msg, user_db=requester_user)
st2common.exceptions.rbac.AccessDeniedError: Decrypt option requires administrator access
2022-09-12 07:01:24,752 INFO [-] 2ff73733-2986-46e9-ba71-16e6e930e84d - 403 62 14.421ms (method='GET',path='/v1/keys/',remote_addr='172.22.192.10',status=403,runtime=14.421,content_length=62,request_id='2ff73733-2986-46e9-ba71-16e6e930e84d')

Can you suggest how to get this working in the sensor?

cheers!

anrajme commented 2 years ago

Found the solution from the database.

image

thanks!

Aliskaa commented 5 months ago

Hello anrajme,

I have the same problem. Can you tell me how you solved your problem?

Thanks!