Authentication error using OIDC

[Kaal-Bhairaav]

I have deployed the app in the docker and set it to use OIDC.

I am not able to get the app working. The app gets the token from okta OIDC successfully and then throws error that the OIDC authentication failed even though logs in okta shows successful attempt.

Is there any deployed demo site that uses OIDC like "https://demo.consolemeoss.com/" uses saml?

[castrapel]

demo.consolemeoss.com uses OIDC, but it uses it through ALB Authentication and not directly through ConsoleMe. So first question: Are you using OIDC directly through ConsoleMe (you have get_user_by_oidc_settings enabled in your ConsoleMe configuration), or are you using ALB Authentication (You have get_user_by_aws_alb_auth enabled)?

2nd, are you setting all of the required parameters? (Please don't share your entire config, as it may contain secrets): OIDC or ALB Auth

3rd question: Can you provide a stack trace?

[Kaal-Bhairaav]

1. I changed value of CONFIG_LOCATION to /apps/consoleme/example_config/example_config_oidc.yaml in docker-compose-dockerhub.yaml file and then deployed the docker using docker-compose -f docker-compose-dockerhub.yaml -f docker-compose-dependencies.yaml up -d command, I hope that is the correct way to use OIDC.

2.1 cat example_config/example_config_oidc.yaml

  - example_config_base.yaml
  - example_secrets.yaml

auth:
  get_user_by_oidc: true
  force_redirect_to_identity_provider: false
  set_auth_cookie: true

get_user_by_oidc_settings:
  resource: ConsoleMeOpenId
  metadata_url: https://dev-24752425.okta.com/oauth2/default/.well-known/oauth-authorization-server
  # If you have a metadata URL and it returns JSON with authorization_endpoint, token_endpoint, and jwks_uri, you do
  # not need to specify those values in the configuration.
  #authorization_endpoint: https://dev-123456.okta.com/oauth2/default/v1/authorize
  #token_endpoint: https://dev-123456.okta.com/oauth2/default/v1/token
  #jwks_uri: https://dev-123456.okta.com/oauth2/default/v1/keys
  jwt_verify: true
  jwt_email_key: email
  jwt_groups_key: groups
  grant_type: authorization_code
  id_token_response_key: id_token
  access_token_response_key: access_token
  access_token_audience: "consoleme"

development: true

url: http://127.0.0.1:8081

2.2 cat example_config/example_secrets.yaml

#sentry:
#  dsn: fakedsn

fakesecret: secret
jwt_secret: secret

oidc_secrets:
  client_id: [ID_from_okta_app]
  secret: [Secret_from_okta_app]
  client_scope:
    - email
    - groups
    - openid

3.1 When I navigate to http://127.0.0.1:8081/, initial login access is provided by the application and I can see upto this much.

3.2Then I keep getting following network requests which does not get any response

3.3following are the logs from docker which seems to be exceptions occurring while the application is generating above requests.

{"asctime": "2021-04-25T13:54:43Z+0000", "name": "consoleme", "processName": "MainProcess", "filename": "base.py", "funcName": "authorization_flow", "levelname": "DEBUG", "lineno": 291, "module": "base", "threadName": "MainThread", "message": "Incoming request", "function": "Basehandler.authorization_flow", "ip": "172.19.0.1", "request_path": "/auth?state=http%3A%2F%2F127.0.0.1%3A8081%2F&error=invalid_scope&error_description=One+or+more+scopes+are+not+configured+for+the+authorization+server+resource.", "user-agent": "Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Firefox/78.0", "request_id": "c054744a-37a9-40e3-80c2-d81151a8e753", "eventTime": "2021-04-25T06:50:51.691071-07:00", "hostname": "14b0708e6322", "timestamp": "2021-04-25T13:54:43Z+0000"}
2021-04-25 13:54:46,361 - INFO - tornado.access - [web.py:2239 - log_request() ] - 302 GET /auth?state=http%3A%2F%2F127.0.0.1%3A8081%2F&error=invalid_scope&error_description=One+or+more+scopes+are+not+configured+for+the+authorization+server+resource. (172.19.0.1) 2594.46ms
2021-04-25 13:54:46,362 - ERROR - tornado.application - [web.py:1789 - log_exception() ] - Uncaught exception GET /auth?state=http%3A%2F%2F127.0.0.1%3A8081%2F&error=invalid_scope&error_description=One+or+more+scopes+are+not+configured+for+the+authorization+server+resource. (172.19.0.1)
HTTPServerRequest(protocol='http', host='127.0.0.1:8081', method='GET', uri='/auth?state=http%3A%2F%2F127.0.0.1%3A8081%2F&error=invalid_scope&error_description=One+or+more+scopes+are+not+configured+for+the+authorization+server+resource.', version='HTTP/1.1', remote_ip='172.19.0.1')
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/tornado/web.py", line 1683, in _execute
    result = await result
  File "/apps/consoleme/consoleme/handlers/auth.py", line 11, in prepare
    await super(AuthHandler, self).prepare()
  File "/apps/consoleme/consoleme/handlers/base.py", line 170, in prepare
    return await self.authorization_flow()
  File "/apps/consoleme/consoleme/handlers/base.py", line 333, in authorization_flow
    res = await authenticate_user_by_oidc(self)
  File "/apps/consoleme/consoleme/lib/oidc.py", line 144, in authenticate_user_by_oidc
    request.finish()
  File "/usr/local/lib/python3.8/site-packages/tornado/web.py", line 1124, in finish
    raise RuntimeError("finish() called twice")
RuntimeError: finish() called twice
{"asctime": "2021-04-25T13:54:46Z+0000", "name": "consoleme", "processName": "MainProcess", "filename": "base.py", "funcName": "authorization_flow", "levelname": "DEBUG", "lineno": 291, "module": "base", "threadName": "MainThread", "message": "Incoming request", "function": "Basehandler.authorization_flow", "ip": "172.19.0.1", "request_path": "/auth?state=http%3A%2F%2F127.0.0.1%3A8081%2F&error=invalid_scope&error_description=One+or+more+scopes+are+not+configured+for+the+authorization+server+resource.", "user-agent": "Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Firefox/78.0", "request_id": "d0c6f9e2-81e6-44c8-9a21-184390b81557", "eventTime": "2021-04-25T06:50:51.691071-07:00", "hostname": "14b0708e6322", "timestamp": "2021-04-25T13:54:46Z+0000"}
2021-04-25 13:54:49,523 - INFO - tornado.access - [web.py:2239 - log_request() ] - 302 GET /auth?state=http%3A%2F%2F127.0.0.1%3A8081%2F&error=invalid_scope&error_description=One+or+more+scopes+are+not+configured+for+the+authorization+server+resource. (172.19.0.1) 2611.15ms
2021-04-25 13:54:49,524 - ERROR - tornado.application - [web.py:1789 - log_exception() ] - Uncaught exception GET /auth?state=http%3A%2F%2F127.0.0.1%3A8081%2F&error=invalid_scope&error_description=One+or+more+scopes+are+not+configured+for+the+authorization+server+resource. (172.19.0.1)
HTTPServerRequest(protocol='http', host='127.0.0.1:8081', method='GET', uri='/auth?state=http%3A%2F%2F127.0.0.1%3A8081%2F&error=invalid_scope&error_description=One+or+more+scopes+are+not+configured+for+the+authorization+server+resource.', version='HTTP/1.1', remote_ip='172.19.0.1')
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/tornado/web.py", line 1683, in _execute
    result = await result
  File "/apps/consoleme/consoleme/handlers/auth.py", line 11, in prepare
    await super(AuthHandler, self).prepare()
  File "/apps/consoleme/consoleme/handlers/base.py", line 170, in prepare
    return await self.authorization_flow()
  File "/apps/consoleme/consoleme/handlers/base.py", line 333, in authorization_flow
    res = await authenticate_user_by_oidc(self)
  File "/apps/consoleme/consoleme/lib/oidc.py", line 144, in authenticate_user_by_oidc
    request.finish()
  File "/usr/local/lib/python3.8/site-packages/tornado/web.py", line 1124, in finish
    raise RuntimeError("finish() called twice")
RuntimeError: finish() called twice

[castrapel]

I see an error from Okta in your chrome inspector (invalid_scope), which might indicate that one of your scopes (either email, groups, or openid) is invalid. Here's the scopes of my consoleme configuration in Okta. Do you have these configured as well?

[Kaal-Bhairaav]

I gave following permission to the app but still the issue is same.

Aslo I see following URL being requested at the end and getting error from the okta server.

https://dev-24752425.okta.com/oauth2/default/v1/authorize?response_type=code&redirect_uri=http%3A%2F%2F127.0.0.1%3A8081%2Fauth&client_id=[ID]&scope=email+groups+openid&state=http%3A%2F%2F127.0.0.1%3A8081%2F

following is the HTTP response:

HTTP/1.1 302 Found
Date: Mon, 26 Apr 2021 08:41:20 GMT
Content-Length: 0
Connection: close
[OTHER HEADERS]
location: http://127.0.0.1:8081/auth?state=http%3A%2F%2F127.0.0.1%3A8081%2F&error=invalid_scope&error_description=One+or+more+scopes+are+not+configured+for+the+authorization+server+resource.
content-language: en
[OTHER HEADERS]

There seems to be some issue with scope in OKTA but I am not sure what to configure, can you please guide me.

[Kaal-Bhairaav]

Also If I remove email and group from the scope parameter in the request and only use openID I am getting okta login page. Hence the request will look like https://dev-24752425.okta.com/oauth2/default/v1/authorize?response_type=code&redirect_uri=http%3A%2F%2F127.0.0.1%3A8081%2Fauth&client_id=[id]&scope=openid&state=http%3A%2F%2F127.0.0.1%3A8081%2F After logging into okta it redirects to http://127.0.0.1:8081/auth?code=[code]&state=http%3A%2F%2F127.0.0.1%3A8081%2F URL and the response is python exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/tornado/web.py", line 1683, in _execute
    result = await result
  File "/apps/consoleme/consoleme/handlers/auth.py", line 11, in prepare
    await super(AuthHandler, self).prepare()
  File "/apps/consoleme/consoleme/handlers/base.py", line 170, in prepare
    return await self.authorization_flow()
  File "/apps/consoleme/consoleme/handlers/base.py", line 335, in authorization_flow
    raise SilentException(
consoleme.exceptions.exceptions.SilentException: Unable to authenticate the user by OIDC. Redirecting to authentication endpoint

while this excpetion occurs I am getting Log in OKTA as following: failure: invalid_authorization_code

Following is Log from OKTA if I use all 3 scopes (email,group,openid)

Following is the Log for only openID as scope

[castrapel]

First, you mentioned you requested the group scope. Note that it is groups, and not group. Give that a shot. If you can't get groups to work, be aware that at least email is required. ConsoleMe needs to know the user's email to determine their level of authorization. If it cannot determine the user's email address, it will not let them load any authenticated page.

If that still doesn't work, try the steps outlined here to add email and groups: https://developer.okta.com/docs/guides/customize-authz-server/create-scopes/ . A screenshot of my settings are below.

[Kaal-Bhairaav]

Hey Man sorry but I tried everything and it does not seems to be working,

Lastly can I ask details on how you have configured you application in okta, Because there are also plenty of settings.

Using these settings you provided I now no longer require to manually remove scope from URL but after OKTA login I am back at python exception which I showed earlier.

It is my last hope if you can help me with configs of application.

Thanks.

[castrapel]

Sure, it'll be good to document this once we figure out which settings are causing the issue:

For login redirect uris, make sure you have these set:

https://YOUR_CONSOLEME_DOMAIN/oauth2/idpresponse and   https://YOUR_CONSOLEME_DOMAIN/auth

for initiate login uri:

https://YOUR_CONSOLEME_DOMAIN/

Under Assignments: Make sure the users you want to be able to log in to ConsoleMe are assigned to your app.

Under scopes, I have: okta.groups.read , okta.users.read.self

[Kaal-Bhairaav]

Made changes accordingly. All that you mentioned are necessary steps.
In addition I found that by default audience name in OKTA is "api://default" and in config file of consoleme it is written as "consoleme" so I just have to change that in file example_config/example_config_oidc.yaml.

  jwt_groups_key: groups
  grant_type: authorization_code
  id_token_response_key: id_token
  access_token_response_key: access_token
  access_token_audience: "api://default"

Also, I can see you are the person who made commit "Verify resource in cache for get_resource_url endpoint (#9064)", the security issue for which the validation was added was reported by me only in the BugCrowd. I found a vulnerability from reviewing code if the server is configured with OIDC auth. That is why I wanted to setup this. Now that I have setup everything I was able to reproduce the bug for POC. If it gets accepted you might have to change few lines again...:D

Thanks for your extreme help.