SSO/SAML support

[pimguilherme]

Hi dear stackstorm maintainers/community!

I've been working on a PR to implement SSO/SAML using whatever we currently have as a base (st2-auth-backend-sso-saml2) and some other bits and pieces of SSO through the main st2 repository.

I asked around in slack if this had any traction and apparently not, so I just went ahead and started implementing some of it because we need that for our strategy at the company I work for.

Are there any other guidelines regarding contribution other than these items? https://github.com/StackStorm/st2/blob/abb694b85f75dc543ecea7df6b87def4bce53309/CONTRIBUTING.rst https://docs.stackstorm.com/development/index.html

I am currently focusing on getting it working on manual tests and will write proper tests afterwards, but can I count on some assistance to further validate/refine this as it progresses?

here's a bit of how it looks so far.. https://user-images.githubusercontent.com/1406885/164324464-29150514-1b5c-415d-bb71-d1e5f2cf568a.mp4

if you have any SAML/SSO material, that would be lovely as well.. currently I've been trying to base the implementation off gravitational/teleport

thanks!!

[rush-skills]

@pimguilherme This demo part looks great. I wanted to test out SSO login, but couldn't find any proper implementation method, however this looks like a good start.

A few questions:

1. Do you handle any groups/roles level permissions while authenticating with SSO?
2. Does it work in parallel to ldap-auth or replaces it?
3. How does the CLI login handled in this case?

[pimguilherme]

thanks for the input @rush-skills !

1 - not yet... i have not looked into the RBAC handling on stackstorm, but if it is as simple as I think it might be, it would just be a matter of mapping groups to roles from the SSO backend (so just a config in the backend should do it)

2 - it should work in parallel to ldap but I am not sure.. the SSO backend as it's currently implemented is parallel to the standard auth mechanisms 3 - the CLI login is somewhat like this:

• user issues 'st2 login --sso'
• the CLI creates a local HTTP (unencrypted) web server on the user PC, at a random port, with urls:
  • /*** -> random string to initiate the login flow on the browser
  • /callback -> endpoint for the browser to finish the login flow with the SSO response
  • /success -> redirect endpoint to just show the login succesful details
• the CLI creates a random AES key
• the CLI issues a SSO request on the auth componnt at /auth/sso/request/cli, providing the random AES key and the callback URL. the key will be used to encrypt the response back to the CLI
  • the auth generates a random ID for this SSO request, and creates a SSO request object in the database (containing the ID, expiration time (eg 2min), the key, and the type (eg web/cli))
  • the auth component then sends the request to the SSO backend, providing the ID it should use for the request (the same as the SSO request above)
  • the SSO backend does its thing and provides back the SSO URL, that should be used for the user to login on the browser..
  • as part of the login flow, it's configured to send the SSO response back to ST2 at /auth/sso/callback
  • the SSO URL is propagated back to the CLI on the response to this SSO request
• the CLI then provides the user with the message "Please finish the SSO login in the URL:".. providing the local web server's URL
  • once this is done, the CLI sits idle waiting for the response at /callback, for 90 seconds, or it times out
• the user visits the local URL (e.g. http://localhost:42343/ee4b73ff-73df-4f3c-9830-026f913bc072), and then gets redirected to the proper IDP
  • the IDP does the authentication, and then redirects the user to /auth/sso/callback, with the SSO response
  • the auth component parses the incoming response, and asks the SSO backend for the ID associated with that response
  • the auth backend then validates if the ID is valid by checking the database, and whether it has expired.. upon validation, the request is deleted from the DB and may not be used again
  • once the SSO request itself is validated, the auth component goes back to the SSO backend and asks it to parse it and provide the associated details of the logged in user
  • here we would (i guess) just need to do a role/group mapping and provide back to the SSO backend for rbac to work
  • the auth component gets these details and then creates a token
  • the auth backend detects that this was a CLI request (which needs key encryption due to the HTTP webserver locally), and then encrypts the content with the key
  • once encrypted, the content is sent back to the CLI using the callback url (e.g. http://localhost:42343/callback), and the user is redirected to /success for the OK message on the browser
• the CLI decrypts the content with the key and stores the token just like it used to do with regular auth
• done :)

this is more or less what goes on.. in the CLI case there's this encryption key, but in the web case I think we just need to do some cross-site request checks, but that's still to be implemented and validated

[rush-skills]

@pimguilherme These steps seem good to me.

Can you open your PR(s) and I can review the code/test the feature and come back at you better

[pimguilherme]

Hi @rush-skills ! Thanks for getting back to me :)

I will create the PR in the code this week and let you know! Cheers

[rush-skills]

Hey @pimguilherme Were you able to make any progress on this? Something you need help with?

[pimguilherme]

Hey @rush-skills ! Thanks for reaching out.. unfortunately I did not make any progress as there were other internal priorities in the company.. many of them involved in internal stackstorm automations but this PR got a bit of a halt.. gladly we should be putting more effort into it starting this week.. hopefully to resolve and finish it :)

Here are our prs:

• st2web - https://github.com/StackStorm/st2web/pull/983
• saml_backend - https://github.com/StackStorm/st2-auth-backend-sso-saml2/pull/16
• st2 -> https://github.com/StackStorm/st2/pull/5664

Some next items to be followed up on:

• finish the SSO/SAML web authentication and security
• figure out which configuration is useful in the config file
• finish RBAC details
• finish the tests

Cheers!

[rush-skills]

Hey @pimguilherme

That looks great and a lot of effort. Thanks for the contribution!

If you can start adding more docs and descriptions to the PR, I can try setting them up next week and testing the changes.

[pimguilherme]

Thanks for the feedback @rush-skills ! It's still not fully done as automated tests and some tweaks here and there are still needed, but it can definitely be tested manually as as starting point.. I am finally going to work on in again this week, so I will provide some more updates and all

cheers :)

[pimguilherme]

hey @rush-skills , I am starting by trying to make tests here: https://github.com/StackStorm/st2-auth-backend-sso-saml2/pull/16

are you familiar with this code? it looks like the makefile is broken and it doesn't look like other st2-auth-backend modules use a Makefile in general.. I just wanted to check before doing any changes, otherwise I will just try to tinker around and make it work

cheers!

[rush-skills]

hey @rush-skills , I am starting by trying to make tests here: StackStorm/st2-auth-backend-sso-saml2#16

are you familiar with this code? it looks like the makefile is broken and it doesn't look like other st2-auth-backend modules use a Makefile in general.. I just wanted to check before doing any changes, otherwise I will just try to tinker around and make it work

cheers!

Haven't got much experience, but indeed the tests are broken for the repo. From what I see, it still uses CircleCI (we are migrating to GitHub Actions now) with Python 2 build (which again is deprecated). I can try creating an issue and picking it up later, but that won't be soon I guess.

Feel free to tinker around and change the tests however you want in the meanwhile.

[pimguilherme]

hi @rush-skills ! I have deleted some old comments and compiled the latest status:

To implement this change I had to update the following repositories:

• st2 (https://github.com/StackStorm/st2/pull/5664)

  • everything basically ready

  • code changes

  • st2auth + st2common

  • added saml-backend as a default dependency of st2auth (still need to point to st2's org, it's currently using my own repo)

  • added cli/web login endpoints on SSO request (GET /auth/sso/request/web or POST /cli)

  • added SSO request database model for storing pending/requested logins and validate callbacks and expiration

  • added a UUID request_id to SSO requests

  • added a BaseSingleSignOnBackendResponse class to better communicate response with sso backends

  • added remote group + role syncing to the ProxyAuthHandler, based on provided proxy groups on the request (reusing same code as the SStandaloneAuthHandler

  • added SSO backend required method get_request_id_from_response, to be able to check agaisnt pending SSO requests in the database

  • created a TokenResourceManager to implement some SSO logic in to token generation

  • updated openapi endpoints for sso

  • addede some helpers methods to st2common access services

  • added a crypto method to read keys from dict directly

  • st2client

  • added --sso option to CLI and integrated with the backend (still needs some tests/validation but should be working)

  • copied crypto.py code to the st2client package for key encryption (not sure how to deal with this best)

  • added a SSOInterceptorProxy class responsible for spinning up a local server to wait for incoming callback from the backend, with the auth details :)

  • misc details

  • prior to processing callbacks, the incoming SSO request id is validated and database model is deleted so it's not longer usable

  • sso uses ProxyAuthenticationHandler

  • callback response differs between CLI and WEB initial requests

  • cli-based login has encrypted response based on incoming key provided by caller (since the callback to the CLI webserver is usually HTTP)

  • web-based login callback populates the st2session cookie so the user is authenticated and redirects them to normal usage of the GUI

• st2web (https://github.com/StackStorm/st2web/pull/983/files)

  • should be 100% ready (hopefully haha)
  • added a login button that redirects to /auth/sso/request/web
  • configurable via config.json
  • added some tests

• st2-auth-backend-sso-saml2 (https://github.com/StackStorm/st2-auth-backend-sso-saml2/pull/16)

  • updated some pylint definitions from base repository
  • updating pysaml2 lib to latest
  • implemented and tested the saml2 backend and integration with the SSO handler
  • added some fixtures for testing with keycloak responses
  • added some logging :)

• st2-dockerfiles (https://github.com/StackStorm/st2-dockerfiles/pull/55)

  • adding xmlsec1 as a dependency to st2auth container

• st2-docker (https://github.com/StackStorm/st2-docker/pull/246/files)

  • added a local keycloak at 3011 with pre-populated stanley user and StackStorm realm
  • enabling RBAC by default
  • enabling SSO by default (in the backend and gui)
  • adding RBAC mappings to st2admin (via assignments) and stanley (via group mapping)

• st2docs (https://github.com/StackStorm/st2docs/pull/1146)

  • added SAML2 related documentation (both SSO config + RBAC config)

to test this, you should:

mkdir /tmp/st2-test
cd /tmp/st2-test
# clone test docker files
git clone git@github.com:pimguilherme/st2-dockerfiles.git
cd st2-dockerfiles

# checkout feat/saml-test
git checkout feat/saml-test

#build images base and then st2auth and st2web locally
docker build -t stackstorm/st2:3.8dev base --build-arg ST2_VERSION=3.8dev
docker build -t stackstorm/st2auth:3.8dev st2auth --build-arg ST2_VERSION=3.8dev
docker build -t stackstorm/st2web:3.8dev st2web --build-arg ST2_VERSION=3.8dev

cd /tmp/st2-test

# clone st2-docker modified with keycloak
git clone https://github.com/pimguilherme/st2-docker
cd st2-docker

git checkout feat/saml-backend

# cleanup before :)
docker-compose down

# cleanup old volumes (BEWARE IF YOU DONT WANT TO DELETE LOCAL TEST DATA)
docker volume rm $(docker volume ls | grep -Po '(docker_stackstorm|st2)-\S+')

# bring up the composer
ST2_VERSION=3.8dev docker-compose up -d

# wait for st2web to be healthy (you may need to restart it once)
# login to web ui at http://localhost/
# click the SSO button
# login user with `stanley/Ch@ngeMe`

to test the st2cli:

cd /tmp/st2-test
git clone https://github.com/pimguilherme/st2
cd st2
git checkout feat/saml
make .st2client-install-check
. ./virtualenv-st2client/bin/activate

# then you just run st2 normally :) (-s = --sso)
st2 login -s 

i think this is pretty much it!! please let me know

cheers!

[rush-skills]

@pimguilherme Wow, this is a lot of work. Thanks for the update!

I haven't got the chance to test these changes yet, will try to follow the instructions and test them next week.

Cheers, Ankur

[pimguilherme]

sounds good @rush-skills , thanks! :)

I was able to validate the steps worked, and I will keep on working further on this and should very soon be fully done.. in any case the testing steps should be working in the meantime

cheers!

[pimguilherme]

hi @rush-skills @cognifloyd @armab (tagging some people I see are quite active around here :)

I just wanted to follow up on this PR.. would you be able to help us merge it? I think it's basically ready, pending basically some docs, and some help on getting the CI to pass

Please let me know if there's anything I can do to help

thanks!

[rush-skills]

Hey @pimguilherme. Apologies, I had been on leave during the summer and didn't get a chance to test it out. Ideally, we will need the docs updated and CI passing to merge the PRs, while at least one person from @StackStorm/tsc doing hands-on testing with the same. I can try to dedicate some effort to it, but that involves building and including the packages from the PRs and using that in our test env, which is quite a pain currently with the rather complicated test/build/package pipeline (which is being fixed by @cognifloyd with the introduction of pants).

In the meanwhile, if you need any specific help with CI, feel free to point out the errors and one of us will have a look and try to get it sorted.

[pimguilherme]

Hey @rush-skills , thanks! I will work on updating the docs and the CI, and also make myself available to help on whatever you guys need to test it out (perhaps following the instructions on one of the last posts could be a starting point for some initial tests)

The CI was passing a few weeks ago, so it is probably related to some recent changes.. probably also not something very big.. I will try to work it out and reach out if needed

Cheers!

[pimguilherme]

Hi @rush-skills ! Further updates.. I think things are 100% ready for the merge :)

CIs are passing and DOC has been updated

I have updated the summary comment a few comments back with all the PRs

Please let me know if theres any further help I can provide

Cheers!

[rush-skills]

Hey @pimguilherme

So I am finally having a look at this and starting to test the PR(s). I will do the docker testing followed by testing the core packages later.

For now, the first issue I face is when I run

docker build -t stackstorm/st2:3.8dev base --build-arg ST2_VERSION=3.8dev

I get

Step 17/34 : RUN TEMP_DEB="/tmp/st2.deb" &&   curl -o "$TEMP_DEB" -L 'https://output.circle-artifacts.com/output/job/54542eb6-74d4-4211-a3a2-785d4f57ed71/artifacts/1/packages/focal/st2_3.8dev-1_amd64.deb' &&   apt update &&   apt install --fix-missing -y "$TEMP_DEB" &&   rm -f "$TEMP_DEB"
 ---> Running in a11658607680
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100    23  100    23    0     0     47      0 --:--:-- --:--:-- --:--:--    47

WARNING: apt does not have a stable CLI interface. Use with caution in scripts.

Get:1 http://security.ubuntu.com/ubuntu focal-security InRelease [114 kB]
Get:2 http://archive.ubuntu.com/ubuntu focal InRelease [265 kB]
Get:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease [114 kB]
Get:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease [108 kB]
Get:5 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 Packages [27.5 kB]
Get:6 http://security.ubuntu.com/ubuntu focal-security/universe amd64 Packages [915 kB]
Get:7 http://archive.ubuntu.com/ubuntu focal/restricted amd64 Packages [33.4 kB]
Get:8 http://archive.ubuntu.com/ubuntu focal/main amd64 Packages [1,275 kB]
Get:9 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 Packages [1,556 kB]
Get:10 http://archive.ubuntu.com/ubuntu focal/universe amd64 Packages [11.3 MB]
Get:11 http://security.ubuntu.com/ubuntu focal-security/main amd64 Packages [2,183 kB]
Get:12 http://archive.ubuntu.com/ubuntu focal/multiverse amd64 Packages [177 kB]
Get:13 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 Packages [1,212 kB]
Get:14 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 Packages [30.2 kB]
Get:15 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages [2,650 kB]
Get:16 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 Packages [1,671 kB]
Get:17 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 Packages [55.2 kB]
Get:18 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 Packages [27.4 kB]
Fetched 23.8 MB in 7s (3,405 kB/s)
Reading package lists...
Building dependency tree...
Reading state information...
4 packages can be upgraded. Run 'apt list --upgradable' to see them.

WARNING: apt does not have a stable CLI interface. Use with caution in scripts.

Reading package lists...
E: Invalid archive signature
E: Internal error, could not locate member control.tar.{zstlz4gzxzbz2lzma}
E: Could not read meta data from /tmp/st2.deb
E: The package lists or status file could not be parsed or opened.
The command '/bin/sh -c TEMP_DEB="/tmp/st2.deb" &&   curl -o "$TEMP_DEB" -L 'https://output.circle-artifacts.com/output/job/54542eb6-74d4-4211-a3a2-785d4f57ed71/artifacts/1/packages/focal/st2_3.8dev-1_amd64.deb' &&   apt update &&   apt install --fix-missing -y "$TEMP_DEB" &&   rm -f "$TEMP_DEB"' returned a non-zero code: 100

[pimguilherme]

oh thatś great to hear @rush-skills ! :D

i have merged the code with the master branch and released the circle CI packages again, and then updated them in the docker files (that's what was failing, because they were purged I think)

can you please try it from scratch?

I have also updated the notes with how to test the st2 client as well

Cheers!

[pimguilherme]

Hi guys! Checking to see if there was any luck here and anything I can do to help

Cheers!

[cydergoth]

Our org is migrating to Okta for SSO, and this is a critical feature for our continued use of Stackstorm. Is there an ETA for this feature becoming available?

[pimguilherme]

Our org is migrating to Okta for SSO, and this is a critical feature for our continued use of Stackstorm. Is there an ETA for this feature becoming available?

Hi @cydergoth ! happy to see interested parties here :)

we have been using this branched implementation of SSO in our company and it's been working well.. we still need the stackstorm maintainers to help us merge this into the main code to get it all worked out.. @rush-skills is helping us on that but I think he has a bit on his plate now

I would requested you to try it out following the steps at the end of this comment: https://github.com/StackStorm/st2/issues/5625#issuecomment-1192960089

if you need any help setting it up, I would love to help (you could try to point the SSO backend to your company's OKTA setup (provided its saml) to see if the current proposal will help you)

thanks!

[thozook]

@pimguilherme I'm definitely interested in testing out this SSO/SAML implementation. It should be a lot smoother than the solution I hacked together with OAuth2-Proxy.

I should have some time next week to set this up in our development environment and do some thorough testing. Thank you and your team for getting this together.

[pimguilherme]

@pimguilherme I'm definitely interested in testing out this SSO/SAML implementation. It should be a lot smoother than the solution I hacked together with OAuth2-Proxy.

I should have some time next week to set this up in our development environment and do some thorough testing. Thank you and your team for getting this together.

that's great to hear @thozook ! initially we also hacked something together just to get it working and now we were able to dedicate some time to get it into the community :)

your help testing will be of great value, cheers! if you have any questions please let me know \o

[cydergoth]

Hi all, nice to see this has.kicked off some interest as we like ST2 :-)

I am busy dealing with a certain OpenSSL issue but I will try to carve out some time next week to test this. Not sure what ingress controller we will be using, possibly Istio

[arm4b]

:100: More testing and feedback would be highly appreciated.

At this moment the release team is preparing the v3.8 https://github.com/orgs/StackStorm/projects/29 and we definitely should plan this big feature for the next v3.9 and more :eyes: from the @StackStorm/maintainers.

[thozook]

I was able to test this out yesterday. After getting everything setup(had to adjust the web port to 8080 due to things already setup on my machine), I was able to run the SSO authenticating against Keycloak without issues.

I then tried to setup SSO with Azure AD(Adjusted metadata_url but kept entity_id the same), but I get an error page with the message: {"faultstring":"Internal Server Error"}

Watching the logs of the st2auth container, it has the following messages:

2022-11-08 15:31:45,933 INFO [-] 54d661a5-42ef-440e-89b5-54d98b44ada5 - GET /sso/request/web with query={} (method='GET',path='/sso/request/web',remote_addr='192.168.32.19',query={},request_id='54d661a5-42ef-440e-89b5-54d98b44ada5')
2022-11-08 15:31:45,944 INFO [-] destination to provider: https://login.microsoftonline.com/##TENANTID##/saml2
2022-11-08 15:31:45,944 INFO [-] REQUEST: <ns0:AuthnRequest xmlns:ns0="urn:oasis:names:tc:SAML:2.0:protocol" xmlns:ns1="urn:oasis:names:tc:SAML:2.0:assertion" ID="id_7849cb85-6829-41af-b9d3-4cbe256bd0a2" Version="2.0" IssueInstant="2022-11-08T15:31:45Z" Destination="https://login.microsoftonline.com/##TENANTID##/saml2" ProtocolBinding="urn:oasis:names:tc:SAML:2.0:bindings:HTTP-POST" AssertionConsumerServiceURL="http://localhost:8080/auth/sso/callback"><ns1:Issuer Format="urn:oasis:names:tc:SAML:2.0:nameid-format:entity">http://localhost:8080</ns1:Issuer></ns0:AuthnRequest>
2022-11-08 15:31:45,945 INFO [-] AuthNReq: <ns0:AuthnRequest xmlns:ns0="urn:oasis:names:tc:SAML:2.0:protocol" xmlns:ns1="urn:oasis:names:tc:SAML:2.0:assertion" ID="id_7849cb85-6829-41af-b9d3-4cbe256bd0a2" Version="2.0" IssueInstant="2022-11-08T15:31:45Z" Destination="https://login.microsoftonline.com/##TENANTID##/saml2" ProtocolBinding="urn:oasis:names:tc:SAML:2.0:bindings:HTTP-POST" AssertionConsumerServiceURL="http://localhost:8080/auth/sso/callback"><ns1:Issuer Format="urn:oasis:names:tc:SAML:2.0:nameid-format:entity">http://localhost:8080</ns1:Issuer></ns0:AuthnRequest>
2022-11-08 15:31:45,945 INFO [-] HTTP REDIRECT
2022-11-08 15:31:45,946 INFO [-] 54d661a5-42ef-440e-89b5-54d98b44ada5 - 307 0 13.105ms (method='GET',path='/sso/request/web',remote_addr='192.168.32.19',status=307,runtime=13.105,content_length=0,request_id='54d661a5-42ef-440e-89b5-54d98b44ada5')
2022-11-08 15:31:46,176 INFO [-] 670d9388-62f7-4fc2-a1c3-49fe31ff7379 - POST /sso/callback with query={} (method='POST',path='/sso/callback',remote_addr='192.168.32.19',query={},request_id='670d9388-62f7-4fc2-a1c3-49fe31ff7379')
2022-11-08 15:31:46,183 ERROR [-] Signature Error: Signature missing for response
2022-11-08 15:31:46,183 ERROR [-] XML parse error: Signature missing for response
2022-11-08 15:31:46,183 ERROR [-] Failed to call controller function "post" for operation "st2auth.controllers.v1.sso:idp_callback_controller.post": Signature missing for response
Traceback (most recent call last):
  File "/opt/stackstorm/st2/lib/python3.8/site-packages/st2common/router.py", line 632, in __call__
    resp = func(**kw)
  File "/opt/stackstorm/st2/lib/python3.8/site-packages/st2auth/controllers/v1/sso.py", line 145, in post
    raise e
  File "/opt/stackstorm/st2/lib/python3.8/site-packages/st2auth/controllers/v1/sso.py", line 96, in post
    original_sso_request = self._validate_and_delete_sso_request(response)
  File "/opt/stackstorm/st2/lib/python3.8/site-packages/st2auth/controllers/v1/sso.py", line 55, in _validate_and_delete_sso_request
    request_id = SSO_BACKEND.get_request_id_from_response(response)
  File "/opt/stackstorm/st2/lib/python3.8/site-packages/st2auth_sso_saml2/saml.py", line 196, in get_request_id_from_response
    authn_response = self._get_authn_response_from_response(response)
  File "/opt/stackstorm/st2/lib/python3.8/site-packages/st2auth_sso_saml2/saml.py", line 180, in _get_authn_response_from_response
    authn_response = saml_client.parse_authn_request_response(
  File "/opt/stackstorm/st2/lib/python3.8/site-packages/saml2/client_base.py", line 841, in parse_authn_request_response
    resp = self._parse_response(
  File "/opt/stackstorm/st2/lib/python3.8/site-packages/saml2/entity.py", line 1496, in _parse_response
    response = response.loads(xmlstr, False, origxml=xmlstr)
  File "/opt/stackstorm/st2/lib/python3.8/site-packages/saml2/response.py", line 528, in loads
    self._loads(xmldata, decode, origxml)
  File "/opt/stackstorm/st2/lib/python3.8/site-packages/saml2/response.py", line 344, in _loads
    self.response = self.signature_check(
  File "/opt/stackstorm/st2/lib/python3.8/site-packages/saml2/sigver.py", line 1760, in correctly_signed_response
    raise SignatureError('Signature missing for response')
saml2.sigver.SignatureError: Signature missing for response
2022-11-08 15:31:46,186 ERROR [-] API call failed: Signature missing for response
Traceback (most recent call last):
  File "/opt/stackstorm/st2/lib/python3.8/site-packages/st2common/middleware/error_handling.py", line 49, in __call__
    return self.app(environ, start_response)
  File "/opt/stackstorm/st2/lib/python3.8/site-packages/st2common/router.py", line 746, in as_wsgi
    resp = self(req)
  File "/opt/stackstorm/st2/lib/python3.8/site-packages/st2common/router.py", line 644, in __call__
    raise e
  File "/opt/stackstorm/st2/lib/python3.8/site-packages/st2common/router.py", line 632, in __call__
    resp = func(**kw)
  File "/opt/stackstorm/st2/lib/python3.8/site-packages/st2auth/controllers/v1/sso.py", line 145, in post
    raise e
  File "/opt/stackstorm/st2/lib/python3.8/site-packages/st2auth/controllers/v1/sso.py", line 96, in post
    original_sso_request = self._validate_and_delete_sso_request(response)
  File "/opt/stackstorm/st2/lib/python3.8/site-packages/st2auth/controllers/v1/sso.py", line 55, in _validate_and_delete_sso_request
    request_id = SSO_BACKEND.get_request_id_from_response(response)
  File "/opt/stackstorm/st2/lib/python3.8/site-packages/st2auth_sso_saml2/saml.py", line 196, in get_request_id_from_response
    authn_response = self._get_authn_response_from_response(response)
  File "/opt/stackstorm/st2/lib/python3.8/site-packages/st2auth_sso_saml2/saml.py", line 180, in _get_authn_response_from_response
    authn_response = saml_client.parse_authn_request_response(
  File "/opt/stackstorm/st2/lib/python3.8/site-packages/saml2/client_base.py", line 841, in parse_authn_request_response
    resp = self._parse_response(
  File "/opt/stackstorm/st2/lib/python3.8/site-packages/saml2/entity.py", line 1496, in _parse_response
    response = response.loads(xmlstr, False, origxml=xmlstr)
  File "/opt/stackstorm/st2/lib/python3.8/site-packages/saml2/response.py", line 528, in loads
    self._loads(xmldata, decode, origxml)
  File "/opt/stackstorm/st2/lib/python3.8/site-packages/saml2/response.py", line 344, in _loads
    self.response = self.signature_check(
  File "/opt/stackstorm/st2/lib/python3.8/site-packages/saml2/sigver.py", line 1760, in correctly_signed_response
    raise SignatureError('Signature missing for response')
saml2.sigver.SignatureError: Signature missing for response (_exception_class='SignatureError',_exception_message='Signature missing for response',_exception_data={})
2022-11-08 15:31:46,188 INFO [-] 670d9388-62f7-4fc2-a1c3-49fe31ff7379 - 500 39 12.066ms (method='POST',path='/sso/callback',remote_addr='192.168.32.19',status=500,runtime=12.066,content_length=39,request_id='670d9388-62f7-4fc2-a1c3-49fe31ff7379')

(Sanitized company identifying info in the above logs of course) I tried some troubleshooting and attempted adding the following settings to see if they made a difference:

"extra_pysaml2_sp_settings": {
        "want_response_signed": "False",
        "want_assertions_signed": "False"

But the message was the same.

My configs I have setup are as follows: Stackstorm side:

sso_backend_kwargs = {
    "entity_id": "http://localhost:8080",
    "metadata_url": "https://login.microsoftonline.com/##TENANTID##/federationmetadata/2007-06/federationmetadata.xml",
    "extra_pysaml2_sp_settings": {
        "want_response_signed": "False",
        "want_assertions_signed": "False"
    }
 }

Azure Side:

Identifier (Entity ID): http://localhost:8080
Reply URL (Assertion Consumer Service URL): http://localhost:8080/auth/sso/callback

[pimguilherme]

thanks for the very valuable test @thozook !!

could you try with

"extra_pysaml2_sp_settings": {
        "want_response_signed": false,
        "want_assertions_signed": false

as these are supposedly json?

if not, could you give it a second try settingt he AzureAD setting to sign responses?

ideally just setting the pysaml2 settings should resolve this situation.. the implementation of extra settings was done exactly for these kinds of scenario

also didn't plan for the error to come out so generic for the end user as "Internal Server Error" (I remember having the intention of being at least a bit more descriptive), but anyway..

[thozook]

Adjusted "extra_pysaml2_sp_settings" and managed to get a different error this time:

2022-11-09 16:28:01,064 ERROR [-] returncode=1
error=func=xmlSecOpenSSLEvpSignatureVerify:file=evp_signatures.c:line=368:obj=rsa-sha256:subj=unknown:error=18:data do not match:details=EVP_VerifyFinal: signature does not verify
FAIL
SignedInfo References (ok/all): 1/1
Manifests References (ok/all): 0/0
Error: failed to verify file "/tmp/tmpjmkoywfl.xml"

output=
2022-11-09 16:28:01,064 ERROR [-] check_sig: ['/usr/bin/xmlsec1', '--verify', '--enabled-reference-uris', 'empty,same-doc', '--enabled-key-data', 'raw-x509-cert', '--pubkey-cert-pem', '/tmp/tmp9pq7viz0.pem', '--id-attr:ID', 'urn:oasis:names:tc:SAML:2.0:protocol:Response', '--node-id', '_acb7d9d9-4890-4b0e-b493-1d73534d5320', '--output', '/tmp/tmpycudif_f.xml', '/tmp/tmpjmkoywfl.xml']

[pimguilherme]

Thanks @thozook ! that's something I have nevery come across.. I will try to replicate it using AD as well

Would you be able to try the second approach, where you would sign the responses on AD?

Cheers!

[pimguilherme]

Perhaps you could also give this configuration a try?

"extra_pysaml2_sp_settings": {
        "want_response_signed": false,
        "want_assertions_signed": false
        "want_assertions_or_response_signed": true,

or

"extra_pysaml2_sp_settings": {
        "want_response_signed": false,
        "want_assertions_signed": true

I'm wondering if your assertions are signed.. It's strange that it's trying to validate signature because you have defined you want neither signatures

EDIT

I could not reproduce your issue.. I was able to configure AD even setting the same thing as you, and trying any of the possible signing combinations in AD

This is what my config looks like:

Claims: (a single Username claim is what 's used to detect the user.. no namespace)

[thozook]

@pimguilherme Thank you for those screenshots, they helped me find out what was incorrect in my settings.

I had left the Attributes and Claims section to the default settings on Azure. After adjusting it to match what you have setup, the SSO flow worked perfectly for me.

I was even able to remove the settings for "want_response_signed" and "want_assertions_signed" after setting the SAML signing option to sign both the response and assertion.

[pimguilherme]

that's great to hear! :) if you have any other issues please report them \o

[nzlosh]

Do these clarifications need to be added the st2docs PR to avoid such confusion in the future?

[pimguilherme]

Do these clarifications need to be added the st2docs PR to avoid such confusion in the future?

I think it's worth adding as we find out some of these examples, but in the end we provide a bit of a "generic" interface to saml configuration and the tweaking / adjusting of particular setups is more in the hands of the admin (hoping these generic configs are enough to provide any tweaking needed)

Some things are definitely worth documenting like the need of the claim being Username.. I will update the docs!

-- EDIT: done! not all checks are passing due to some readthedocs check, but apparently all PRs are the same.. so letting that aside for now

[sidharth61]

Your https://output.circle-artifacts.com/output/job/9492f18c-c1f6-4232-a3a7-a45a9e5423bc/artifacts/1/packages/focal/st2_3.8dev-1_amd64.deb this link from file st2-dockerfiles/base/Dockerfile is not work same issue in st2web. Can I get updated link?

[pimguilherme]

Hey! I will rebuild it later today and update the ticket!

On Fri, Feb 24, 2023, 11:12 sidharth61 @.***> wrote:

Your https://output.circle-artifacts.com/output/job/9492f18c-c1f6-4232-a3a7-a45a9e5423bc/artifacts/1/packages/focal/st2_3.8dev-1_amd64.deb this link from file st2-dockerfiles/base/Dockerfile is not work same issue in st2web. Can I get updated link?

— Reply to this email directly, view it on GitHub https://github.com/StackStorm/st2/issues/5625#issuecomment-1443738577, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAKXPJJFCDG2CYS6DXMPBHDWZC6V7ANCNFSM5T5JLGXQ . You are receiving this because you were mentioned.Message ID: @.***>

[sidharth61]

@pimguilherme thanks foe quick help. https://output.circle-artifacts.com/output/job/64165511-269a-489e-b3cf-a9e122879408/artifacts/0/packages/focal/st2web_3.8dev-1_amd64.deb link inside st2-dockerfiles/st2web/Dockerfile (feat/saml-test).

[pimguilherme]

sure! I have rebuilt them @sidharth61 , thanks :)

@nzlosh @armab @cognifloyd do you think this is still going to make 3.9.0?

[cognifloyd]

I'm focused on revamping the developer tooling around contributing to StackStorm, so I'm having a hard time switching focus to reviewing other changes in ST2. But, Improving SSO is an awesome thing, so I do want to see you work merged ASAP, preferably in 3.9.

Please ask for PR reviews in #development on slack. If no one responds after a few days to a week (time for people across time zones to see it), post again in the #tsc channel.

[pimguilherme]

Got it! Will do, thanks @cognifloyd !

I will try to engage people on those channels

Cheers!

[rite2hhh]

Is SSO officially supported in st2 v3.8.0? I don't see it specified in the release notes: https://docs.stackstorm.com/changelog.html#november-18-2022

However, I see that it's part of the config file for the tagged v3.8.0 release: https://github.com/StackStorm/st2/blob/v3.8.0/conf/st2.conf.sample

This release note mentions that SSO/SAML support with be part of the next release: https://stackstorm.com/2022/12/v3-8-0-released/

[sidharth61]

@pimguilherme for Azure AD for entity_id I am using my domain name with port number. I am getting error "{"faultstring":"Invalid referer -- it should be either some localhost endpoint or the SSO configured entity"}" I am not localhost as you used in config. Do I also have to use localhost instead of my domain name?

[pimguilherme]

Is SSO officially supported in st2 v3.8.0? I don't see it specified in the release notes: https://docs.stackstorm.com/changelog.html#november-18-2022

However, I see that it's part of the config file for the tagged v3.8.0 release: https://github.com/StackStorm/st2/blob/v3.8.0/conf/st2.conf.sample

This release note mentions that SSO/SAML support with be part of the next release: https://stackstorm.com/2022/12/v3-8-0-released/

Hi @rite2hhh ! SSO was partly implemented in 3.8.0 (it was done so a while ago), but it was never really fully supported.. The foundation for SSO was there but no specific SSO mechanism was fully implemented. This PR aims to implement the SSO backend + SAML fully for usage.

if you are looking for SSO/SAML in stackstorm, it would be very helpful if you could test it out with these steps and see if it works for you :) https://github.com/StackStorm/st2/issues/5625#issuecomment-1192960089

Cheers!

---

@sidharth61, I think you should use the domain/port combination that is used by your users to access stackstorm. Can you please confirm that's the case? And then you should also configure it in Azure AD like this:

Cheers!

[sidharth61]

@pimguilherme I need instruction for enabling ssl. For azure sso I need this ssl enabled. Don't find any documentation. I made some changes but not working for me.i also generated ssl cert using code that is provided to create free certificate and enabled ssl option.

[pimguilherme]

@pimguilherme I need instruction for enabling ssl. For azure sso I need this ssl enabled. Don't find any documentation. I made some changes but not working for me.i also generated ssl cert using code that is provided to create free certificate and enabled ssl option.

Hi @sidharth61 ! You should be able to use SSL normally.. in the example above there is SSL enabled (https://stackstorm.homolog.etc). Can you confirm if you have been able to configure stackstorm to serve using SSL? This depends a lot on how you are deploying Stackstorm

Thanks!

[sidharth61]

@pimguilherme I enabled ssl by changing this ST2WEB_HTTPS: ${ST2WEB_HTTPS:-0} in file st2-docker/docker-compose.yml. Also enabled - "${ST2_EXPOSE_HTTPS:-127.0.0.1:443}:443" this and changed ip address from 127.0.0.1 to 0.0.0.0 , copied ssl certificate in folder /etc/ssl/st2 inside st2web container. After this restarted nginx service and st2web container started showing error.

[sidharth61]

@pimguilherme can you provide me steps how to enable ssl. After I enabled ssl in st2web I am getting ssl issue with st2api baseurl.

[sidharth61]

@cognifloyd need help with ssl. I deployed stackstorm container. Enable ssl for st2web as it is described in s2web dockerfile and generated free ssl. But getting issue with api url redirecting to 308. What other changes should I do to make ssl work . Attest provide some documentation for enabling ssl for docker

[sidharth61]

@pimguilherme I deployed stackstorm like you described in this thread. I changed ip from 127.0.0.1 to 0.0.0.0 . Enabled ssl option st2web as you have described and generated free ssl using given code but getting issue on line 141 in nginx config st2.conf. That has url related to st2auth. Can you tell if I have to enable ssl in other containers also. I need help in this issue.