search

opensearch-project / security-dashboards-plugin

🔐 Manage your internal users, roles, access control, and audit logs from OpenSearch Dashboards
https://opensearch.org/docs/latest/security-plugin/index/
Apache License 2.0
70 stars 152 forks source link

Security Plugin - OpenID Connect Authentication returns 401 Unauthorized since 2.14 #1974

Closed hjannasch closed 3 months ago

hjannasch commented 4 months ago

After upgrading to OpenSearch & OpenSearchDashboards 2.14 the authentication via OpenID Connect against Entra Id (Azure AD) is not working anymore.

To Reproduce Steps to reproduce the behavior:

  1. Configure OIDC
  2. Try to login

Dashboards config

# Enable OpenID authentication
opensearch_security.auth.type: "openid"
opensearch_security.openid.connect_url: "https://login.microsoftonline.com/<TENANT_ID>/v2.0/.well-known/openid-configuration"
opensearch_security.openid.client_id: "<CLIENT_ID>"
opensearch_security.openid.client_secret: <SECRET>
opensearch_security.openid.base_redirect_url: "https://DNS_NAME_PLACEHOLDER"

opensearch_security.multitenancy.enabled: true
opensearch_security.multitenancy.tenants.preferred: ["Private", "Global"]
opensearch_security.readonly_mode.roles: [“kibana_read_only”]

Request flow - NOK GET https://logs-training.itbs.bs.ch/auth/openid/login?nextUrl=%2F&redirectHash=false Response: 302 Found

GET https://login.microsoftonline.com/.../oauth2/v2.0/authorize?client_id= Response: 302 Found

GET https://logs-training.itbs.bs.ch/auth/openid/login?code=... Response: 401 Unauthorized {statusCode: 401, error: "Unauthorized", message: "Unauthorized"}

Expected behavior Not having 401 Unauthorized from /auth/openid/login endpoint, if I revert to 2.13 (without any other configuration change) OIDC is working again.

OpenSearch Version 2.14

Dashboards Version 2.14

Plugins All OpenSearch default plugins.

Screenshots

If applicable, add screenshots to help explain your problem.

Host/Environment (please complete the following information):

Additional context If I revert to 2.13 (without any other configuration change) OIDC is working again.

cwperks commented 4 months ago

@hjannasch Thank you for filing this issue!

@derek-ho Could this be related to MDS changes in 2.14? There was no change related to OpenID in the 2.14 release: https://github.com/opensearch-project/security-dashboards-plugin/blob/main/release-notes/opensearch-security-dashboards-plugin.release-notes-2.14.0.0.md

derek-ho commented 4 months ago

@hjannasch Thank you for filing this issue!

@derek-ho Could this be related to MDS changes in 2.14? There was no change related to OpenID in the 2.14 release: https://github.com/opensearch-project/security-dashboards-plugin/blob/main/release-notes/opensearch-security-dashboards-plugin.release-notes-2.14.0.0.md

I don't think that MDS should have affected this, but I may need to deep dive and try to reproduce. I think another issue that has come up regarding refresh tokens not being used indicates it is working for other users. @hjannasch do you have the flag data_source.enabled set as true?

hjannasch commented 4 months ago

@derek-ho data_source.enabled is not set.

hjannasch commented 4 months ago

@derek-ho Maybe the logs can help? I have removed the long codeand session_stateparams. Can I enable more debug, trace logs for the security plugin?

{"type":"log","@timestamp":"2024-05-30T07:04:03Z","tags":["error","plugins","securityDashboards"],"pid":1,"message":"OpenId authentication failed: Error: Authentication Exception"}
{"type":"response","@timestamp":"2024-05-30T07:04:03Z","tags":[],"pid":1,"method":"get","statusCode":401,"req":{"url":"/auth/openid/login?code=...&session_state=...","method":"get","headers":{"host":"logs-training.itbs.bs.ch","upgrade-insecure-requests":"1","user-agent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/125.0.0.0 Safari/537.36","accept":"text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7","sec-fetch-site":"cross-site","sec-fetch-mode":"navigate","sec-fetch-dest":"document","sec-ch-ua":"\"Google Chrome\";v=\"125\", \"Chromium\";v=\"125\", \"Not.A/Brand\";v=\"24\"","sec-ch-ua-mobile":"?0","sec-ch-ua-platform":"\"Windows\"","referer":"https://logs-training.itbs.bs.ch/","accept-encoding":"gzip, deflate, br, zstd","accept-language":"de-CH,de-DE;q=0.9,de;q=0.8,en-US;q=0.7,en;q=0.6","priority":"u=0, i","x-forwarded-proto":"https"},"remoteAddress":"172.25.3.2","userAgent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/125.0.0.0 Safari/537.36","referer":"https://logs-training.itbs.bs.ch/"},"res":{"statusCode":401,"responseTime":528,"contentLength":9},"message":"GET /auth/openid/login?code=...&session_state=abaa8de2-ab69-4622-9e7e-eff41f811428 401 528ms - 9.0B"}
{"type":"response","@timestamp":"2024-05-30T07:04:03Z","tags":[],"pid":1,"method":"get","statusCode":401,"req":{"url":"/favicon.ico","method":"get","headers":{"host":"logs-training.itbs.bs.ch","sec-ch-ua":"\"Google Chrome\";v=\"125\", \"Chromium\";v=\"125\", \"Not.A/Brand\";v=\"24\"","sec-ch-ua-mobile":"?0","user-agent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/125.0.0.0 Safari/537.36","sec-ch-ua-platform":"\"Windows\"","accept":"image/avif,image/webp,image/apng,image/svg+xml,image/*,*/*;q=0.8","sec-fetch-site":"same-origin","sec-fetch-mode":"no-cors","sec-fetch-dest":"image","referer":"https://logs-training.itbs.bs.ch/auth/openid/login?code=...&session_state=...","accept-encoding":"gzip, deflate, br, zstd","accept-language":"de-CH,de-DE;q=0.9,de;q=0.8,en-US;q=0.7,en;q=0.6","priority":"u=1, i","x-forwarded-proto":"https"},"remoteAddress":"172.25.3.2","userAgent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/125.0.0.0 Safari/537.36","referer":"https://logs-training.itbs.bs.ch/auth/openid/login?code=...&state=46y-bPw20ZvB-B-zAjtUnH&session_state=..."},"res":{"statusCode":401,"responseTime":3,"contentLength":9},"message":"GET /favicon.ico 401 3ms - 9.0B"}
hjannasch commented 4 months ago

@derek-ho With logging.verbose: true there's not much more in the logs (one new debug log).

{"type":"response","@timestamp":"2024-05-31T09:25:54Z","tags":[],"pid":1,"method":"get","statusCode":200,"req":{"url":"/auth/openid/captureUrlFragment.js","method":"get","headers":{"host":"logs-training.itbs.bs.ch","sec-ch-ua":"\"Google Chrome\";v=\"125\", \"Chromium\";v=\"125\", \"Not.A/Brand\";v=\"24\"","sec-ch-ua-mobile":"?0","user-agent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/125.0.0.0 Safari/537.36","sec-ch-ua-platform":"\"Windows\"","accept":"*/*","sec-fetch-site":"same-origin","sec-fetch-mode":"no-cors","sec-fetch-dest":"script","referer":"https://logs-training.itbs.bs.ch/auth/openid/captureUrlFragment?nextUrl=/","accept-encoding":"gzip, deflate, br, zstd","accept-language":"de-CH,de-DE;q=0.9,de;q=0.8,en-US;q=0.7,en;q=0.6","priority":"u=1","x-forwarded-proto":"https"},"remoteAddress":"172.25.3.2","userAgent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/125.0.0.0 Safari/537.36","referer":"https://logs-training.itbs.bs.ch/auth/openid/captureUrlFragment?nextUrl=/"},"res":{"statusCode":200,"responseTime":2,"contentLength":9},"message":"GET /auth/openid/captureUrlFragment.js 200 2ms - 9.0B"}
{"type":"response","@timestamp":"2024-05-31T09:25:54Z","tags":[],"pid":1,"method":"get","statusCode":302,"req":{"url":"/auth/openid/login?nextUrl=%2F&redirectHash=false","method":"get","headers":{"host":"logs-training.itbs.bs.ch","sec-ch-ua":"\"Google Chrome\";v=\"125\", \"Chromium\";v=\"125\", \"Not.A/Brand\";v=\"24\"","sec-ch-ua-mobile":"?0","sec-ch-ua-platform":"\"Windows\"","upgrade-insecure-requests":"1","user-agent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/125.0.0.0 Safari/537.36","accept":"text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7","sec-fetch-site":"same-origin","sec-fetch-mode":"navigate","sec-fetch-dest":"document","referer":"https://logs-training.itbs.bs.ch/auth/openid/captureUrlFragment?nextUrl=/","accept-encoding":"gzip, deflate, br, zstd","accept-language":"de-CH,de-DE;q=0.9,de;q=0.8,en-US;q=0.7,en;q=0.6","priority":"u=0, i","x-forwarded-proto":"https"},"remoteAddress":"172.25.3.2","userAgent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/125.0.0.0 Safari/537.36","referer":"https://logs-training.itbs.bs.ch/auth/openid/captureUrlFragment?nextUrl=/"},"res":{"statusCode":302,"responseTime":4,"contentLength":9},"message":"GET /auth/openid/login?nextUrl=%2F&redirectHash=false 302 4ms - 9.0B"}
{"type":"log","@timestamp":"2024-05-31T09:25:55Z","tags":["debug","opensearch","opendistro_security","query"],"pid":1,"message":"401\nGET /_plugins/_security/authinfo\n"}
{"type":"log","@timestamp":"2024-05-31T09:25:55Z","tags":["error","plugins","securityDashboards"],"pid":1,"message":"OpenId authentication failed: Error: Authentication Exception"}
{"type":"response","@timestamp":"2024-05-31T09:25:54Z","tags":[],"pid":1,"method":"get","statusCode":401,"req":{"url":"/auth/openid/login?code=...&session_state=...","method":"get","headers":{"host":"logs-training.itbs.bs.ch","upgrade-insecure-requests":"1","user-agent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/125.0.0.0 Safari/537.36","accept":"text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7","sec-fetch-site":"cross-site","sec-fetch-mode":"navigate","sec-fetch-dest":"document","sec-ch-ua":"\"Google Chrome\";v=\"125\", \"Chromium\";v=\"125\", \"Not.A/Brand\";v=\"24\"","sec-ch-ua-mobile":"?0","sec-ch-ua-platform":"\"Windows\"","referer":"https://logs-training.itbs.bs.ch/","accept-encoding":"gzip, deflate, br, zstd","accept-language":"de-CH,de-DE;q=0.9,de;q=0.8,en-US;q=0.7,en;q=0.6","priority":"u=0, i","x-forwarded-proto":"https"},"remoteAddress":"172.25.3.2","userAgent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/125.0.0.0 Safari/537.36","referer":"https://logs-training.itbs.bs.ch/"},"res":{"statusCode":401,"responseTime":439,"contentLength":9},"message":"GET /auth/openid/login?code=...&session_state=... 401 439ms - 9.0B"}
{"type":"response","@timestamp":"2024-05-31T09:25:55Z","tags":[],"pid":1,"method":"get","statusCode":401,"req":{"url":"/favicon.ico","method":"get","headers":{"host":"logs-training.itbs.bs.ch","sec-ch-ua":"\"Google Chrome\";v=\"125\", \"Chromium\";v=\"125\", \"Not.A/Brand\";v=\"24\"","sec-ch-ua-mobile":"?0","user-agent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/125.0.0.0 Safari/537.36","sec-ch-ua-platform":"\"Windows\"","accept":"image/avif,image/webp,image/apng,image/svg+xml,image/*,*/*;q=0.8","sec-fetch-site":"same-origin","sec-fetch-mode":"no-cors","sec-fetch-dest":"image","referer":"https://logs-training.itbs.bs.ch/auth/openid/login?code=...&state=...","accept-encoding":"gzip, deflate, br, zstd","accept-language":"de-CH,de-DE;q=0.9,de;q=0.8,en-US;q=0.7,en;q=0.6","priority":"u=1, i","x-forwarded-proto":"https"},"remoteAddress":"172.25.3.2","userAgent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/125.0.0.0 Safari/537.36","referer":"https://logs-training.itbs.bs.ch/auth/openid/login?code=...&state=...&session_state=..."},"res":{"statusCode":401,"responseTime":3,"contentLength":9},"message":"GET /favicon.ico 401 3ms - 9.0B"}
stephen-crawford commented 4 months ago

[Triage] Hi @hjannasch thank you for filing this issue. This sounds like a regression and something we would not want to happen! Someone will go ahead and take a further look. It sounds like @derek-ho may be on the case.

hjannasch commented 3 months ago

@derek-ho Any news concerning this issue?

derek-ho commented 3 months ago

Hi @hjannasch just getting around to looking at this today. I am not able to reproduce this issue:

server  respons [15:00:41.357]  GET /auth/openid/captureUrlFragment?nextUrl=%2F 200 4ms - 9.0B
server  respons [15:00:41.383]  GET /auth/openid/captureUrlFragment.js 200 2ms - 9.0B
server  respons [15:00:41.406]  GET /auth/openid/login?redirectHash=false&nextUrl=%2F 302 3ms - 9.0B
server  respons [15:00:43.486]  GET /auth/openid/login?state=XXX&session_state=XXX 302 67ms - 9.0B

Since in you previous messages I also see no difference in configuration between 2.13 and 2.14 makes me feel like it is a regression in between 2.13 and 2.14 for your specific setup. Can you share your other opensearch_dashboards.yml config? It seems like the only change that was introduced in this timeframe is: https://github.com/opensearch-project/security-dashboards-plugin/pull/1899, which seems related to basepath settings. Did you have any other changes between the base path settings?

hjannasch commented 3 months ago

Hi @derek-ho opensearch_dashboards.yml is as follows. I detected a copy/paste issue where the client_secret is duplicated, but the second one is in comments. But I don't think that's an issue.

server.name: opensearch-dashboards
server.host: "0.0.0.0"
opensearch.hosts: ["ELASTICSEARCH_URL_PLACEHOLDER"]

# Enable OpenID authentication
opensearch_security.auth.type: "openid"
opensearch_security.openid.connect_url: "https://login.microsoftonline.com/<TENANT_ID>/v2.0/.well-known/openid-configuration"
opensearch_security.openid.client_id: "<CLIENT_ID>"
opensearch_security.openid.client_secret: CLIENT_SECRET
opensearch_security.openid.base_redirect_url: "https://logs-training.itbs.bs.ch"

opensearch_security.multitenancy.enabled: true
opensearch_security.multitenancy.tenants.preferred: ["Private", "Global"]
opensearch_security.readonly_mode.roles: [“kibana_read_only”]

# The client secret of the OpenID Connect client
#opensearch_security.openid.client_secret: "CLIENT_SECRET"

# Configure the Kibana internal server user
opensearch.username: kibanaserver
opensearch.password: elastic_kibanaserver_password_placeholder

# Disable SSL verification when using self-signed demo certificates
opensearch.ssl.verificationMode: none

# Whitelist basic headers and multi-tenancy header
opensearch.requestHeadersWhitelist: ["Authorization", "security_tenant"]
hjannasch commented 3 months ago

Hi @derek-ho tried upgrading to 2.15, but same issue detected.

hjannasch commented 3 months ago

Hi @derek-ho , I found the issue. Inside the security backend config I had accidentally set challenge: true on basic auth, which means basic auth took precedence in the backend? In prior OS versions this misconfiguration was not a problem, after setting challenge: false it worked.

    authc:
      basic_internal_auth_domain:
        description: "Authenticate via HTTP Basic against internal users database"
        http_enabled: true
        transport_enabled: true
        order: 0
        http_authenticator:
          type: basic
          challenge: true
        authentication_backend:
          type: intern
      openid_auth_domain:
        http_enabled: true
        transport_enabled: true
        order: 1
        http_authenticator:
          type: openid
          challenge: false
cwperks commented 3 months ago

Thanks for replying back with the fix @hjannasch!