Open kannanvr opened 11 months ago
i got the same problem. did you fix it?
Same problem =/
Working for me. Please see below. Pay attention to strings that are inside<...>
Deploy cluster using Helm Chart Values yaml
opensearchCluster:
enabled: true
general:
version: 2.11.0
serviceName: "opensearch-cluster"
pluginsList:
- repository-s3
drainDataNodes: true
security:
config:
adminCredentialsSecret:
name: admin-credentials-secret
securityConfigSecret:
name: securityconfig-secret
dashboards:
enable: true
replicas: 1
version: 2.11.0
opensearchCredentialsSecret:
name: admin-credentials-secret
additionalConfig:
logging.verbose: "true"
opensearch.requestHeadersWhitelist: |
["authorization", "securitytenant"]
opensearch_security.auth.type: |
["basicauth", "openid"]
opensearch_security.auth.multiple_auth_enabled: "true"
opensearch_security.multitenancy.enabled: "true"
opensearch_security.multitenancy.tenants.preferred: |
[ "Private", "Global" ]
opensearch_security.readonly_mode.roles: |
["kibana_read_only"]
opensearch_security.openid.base_redirect_url: "https://<example.com>"
opensearch_security.multitenancy.tenants.enable_global: "true"
opensearch_security.multitenancy.tenants.enable_private: "true"
opensearch_security.openid.client_id: "opensearch-dashboard"
opensearch_security.openid.client_secret: ref+vault://kv-v2/application/env/opensearch?proto=http#/oidc_client_secret
opensearch_security.openid.connect_url: "https://<keycloaktest.example.com>/auth/realms/<example-realm>/.well-known/openid-configuration"
opensearch_security.openid.scope: "openid"
nodePools:
- component: masters
diskSize: "30Gi"
replicas: 3
roles:
- "master"
- "data"
resources:
requests:
memory: "2Gi"
cpu: "500m"
limits:
memory: "2Gi"
cpu: "500m"
Create a securityconfig-secret
action_groups.yml: |-
_meta:
type: "actiongroups"
config_version: 2
internal_users.yml: ref+vault://kv-v2/application/env/opensearch?proto=http#/internal_users.yml
nodes_dn.yml: |-
_meta:
type: "nodesdn"
config_version: 2
whitelist.yml: |-
_meta:
type: "whitelist"
config_version: 2
tenants.yml: |-
_meta:
type: "tenants"
config_version: 2
roles_mapping.yml: |-
_meta:
type: "rolesmapping"
config_version: 2
all_access:
reserved: false
backend_roles:
- "admin"
description: "Maps admin to all_access"
users:
- "<user@example.com>"
own_index:
reserved: false
users:
- "*"
description: "Allow full access to an index named like the username"
readall:
reserved: false
backend_roles:
- "readall"
manage_snapshots:
reserved: false
backend_roles:
- "snapshotrestore"
dashboard_server:
reserved: true
users:
- "dashboarduser"
roles.yml: |-
_meta:
type: "roles"
config_version: 2
dashboard_read_only:
reserved: true
security_rest_api_access:
reserved: true
# Allows users to view monitors, destinations and alerts
alerting_read_access:
reserved: true
cluster_permissions:
- 'cluster:admin/opendistro/alerting/alerts/get'
- 'cluster:admin/opendistro/alerting/destination/get'
- 'cluster:admin/opendistro/alerting/monitor/get'
- 'cluster:admin/opendistro/alerting/monitor/search'
# Allows users to view and acknowledge alerts
alerting_ack_alerts:
reserved: true
cluster_permissions:
- 'cluster:admin/opendistro/alerting/alerts/*'
# Allows users to use all alerting functionality
alerting_full_access:
reserved: true
cluster_permissions:
- 'cluster_monitor'
- 'cluster:admin/opendistro/alerting/*'
index_permissions:
- index_patterns:
- '*'
allowed_actions:
- 'indices_monitor'
- 'indices:admin/aliases/get'
- 'indices:admin/mappings/get'
# Allow users to read Anomaly Detection detectors and results
anomaly_read_access:
reserved: true
cluster_permissions:
- 'cluster:admin/opendistro/ad/detector/info'
- 'cluster:admin/opendistro/ad/detector/search'
- 'cluster:admin/opendistro/ad/detectors/get'
- 'cluster:admin/opendistro/ad/result/search'
- 'cluster:admin/opendistro/ad/tasks/search'
- 'cluster:admin/opendistro/ad/detector/validate'
- 'cluster:admin/opendistro/ad/result/topAnomalies'
# Allows users to use all Anomaly Detection functionality
anomaly_full_access:
reserved: true
cluster_permissions:
- 'cluster_monitor'
- 'cluster:admin/opendistro/ad/*'
index_permissions:
- index_patterns:
- '*'
allowed_actions:
- 'indices_monitor'
- 'indices:admin/aliases/get'
- 'indices:admin/mappings/get'
# Allows users to read Notebooks
notebooks_read_access:
reserved: true
cluster_permissions:
- 'cluster:admin/opendistro/notebooks/list'
- 'cluster:admin/opendistro/notebooks/get'
# Allows users to all Notebooks functionality
notebooks_full_access:
reserved: true
cluster_permissions:
- 'cluster:admin/opendistro/notebooks/create'
- 'cluster:admin/opendistro/notebooks/update'
- 'cluster:admin/opendistro/notebooks/delete'
- 'cluster:admin/opendistro/notebooks/get'
- 'cluster:admin/opendistro/notebooks/list'
# Allows users to read observability objects
observability_read_access:
reserved: true
cluster_permissions:
- 'cluster:admin/opensearch/observability/get'
# Allows users to all Observability functionality
observability_full_access:
reserved: true
cluster_permissions:
- 'cluster:admin/opensearch/observability/create'
- 'cluster:admin/opensearch/observability/update'
- 'cluster:admin/opensearch/observability/delete'
- 'cluster:admin/opensearch/observability/get'
# Allows users to read and download Reports
reports_instances_read_access:
reserved: true
cluster_permissions:
- 'cluster:admin/opendistro/reports/instance/list'
- 'cluster:admin/opendistro/reports/instance/get'
- 'cluster:admin/opendistro/reports/menu/download'
# Allows users to read and download Reports and Report-definitions
reports_read_access:
reserved: true
cluster_permissions:
- 'cluster:admin/opendistro/reports/definition/get'
- 'cluster:admin/opendistro/reports/definition/list'
- 'cluster:admin/opendistro/reports/instance/list'
- 'cluster:admin/opendistro/reports/instance/get'
- 'cluster:admin/opendistro/reports/menu/download'
# Allows users to all Reports functionality
reports_full_access:
reserved: true
cluster_permissions:
- 'cluster:admin/opendistro/reports/definition/create'
- 'cluster:admin/opendistro/reports/definition/update'
- 'cluster:admin/opendistro/reports/definition/on_demand'
- 'cluster:admin/opendistro/reports/definition/delete'
- 'cluster:admin/opendistro/reports/definition/get'
- 'cluster:admin/opendistro/reports/definition/list'
- 'cluster:admin/opendistro/reports/instance/list'
- 'cluster:admin/opendistro/reports/instance/get'
- 'cluster:admin/opendistro/reports/menu/download'
# Allows users to use all asynchronous-search functionality
asynchronous_search_full_access:
reserved: true
cluster_permissions:
- 'cluster:admin/opendistro/asynchronous_search/*'
index_permissions:
- index_patterns:
- '*'
allowed_actions:
- 'indices:data/read/search*'
# Allows users to read stored asynchronous-search results
asynchronous_search_read_access:
reserved: true
cluster_permissions:
- 'cluster:admin/opendistro/asynchronous_search/get'
# Allows user to use all index_management actions - ism policies, rollups, transforms
index_management_full_access:
reserved: true
cluster_permissions:
- "cluster:admin/opendistro/ism/*"
- "cluster:admin/opendistro/rollup/*"
- "cluster:admin/opendistro/transform/*"
index_permissions:
- index_patterns:
- '*'
allowed_actions:
- 'indices:admin/opensearch/ism/*'
# Allows users to use all cross cluster replication functionality at leader cluster
cross_cluster_replication_leader_full_access:
reserved: true
index_permissions:
- index_patterns:
- '*'
allowed_actions:
- "indices:admin/plugins/replication/index/setup/validate"
- "indices:data/read/plugins/replication/changes"
- "indices:data/read/plugins/replication/file_chunk"
# Allows users to use all cross cluster replication functionality at follower cluster
cross_cluster_replication_follower_full_access:
reserved: true
cluster_permissions:
- "cluster:admin/plugins/replication/autofollow/update"
index_permissions:
- index_patterns:
- '*'
allowed_actions:
- "indices:admin/plugins/replication/index/setup/validate"
- "indices:data/write/plugins/replication/changes"
- "indices:admin/plugins/replication/index/start"
- "indices:admin/plugins/replication/index/pause"
- "indices:admin/plugins/replication/index/resume"
- "indices:admin/plugins/replication/index/stop"
- "indices:admin/plugins/replication/index/update"
- "indices:admin/plugins/replication/index/status_check"
config.yml: |-
_meta:
type: "config"
config_version: "2"
config:
dynamic:
http:
anonymous_auth_enabled: false
xff:
enabled: false
internalProxies: '10\.\d{1-3}\.\d{1-3}\.\d{1-3}' # regex pattern
authc:
basic_internal_auth_domain:
description: "Authenticate via HTTP Basic against internal users database"
http_enabled: true
transport_enabled: true
order: "4"
http_authenticator:
type: "basic"
challenge: false
authentication_backend:
type: "internal"
openid_auth_domain:
http_enabled: true
transport_enabled: true
order: "1"
http_authenticator:
type: "openid"
challenge: "false"
config:
subject_key: "preferred_username"
roles_key: "roles"
openid_connect_url: https://<keycloaktest.example.com>/auth/realms/<example-realm>/.well-known/openid-configuration
authentication_backend:
type: "noop"
opensearch.yaml: |-
plugins.security.audit.type: internal_opensearch
plugins.security.authcz.admin_dn: ["CN=admin,OU=opensearch-cluster"]
plugins.security.check_snapshot_restore_write_privileges: true
plugins.security.enable_snapshot_restore_privilege: true
plugins.security.nodes_dn: ["CN=opensearch-cluster,OU=opensearch-cluster"]
plugins.security.restapi.roles_enabled: ["all_access", "security_rest_api_access"]
plugins.security.ssl.http.enabled: true
plugins.security.ssl.http.pemcert_filepath: tls-http/tls.crt
plugins.security.ssl.http.pemkey_filepath: tls-http/tls.key
plugins.security.ssl.http.pemtrustedcas_filepath: tls-http/ca.crt
plugins.security.ssl.transport.enforce_hostname_verification: false
plugins.security.ssl.transport.pemcert_filepath: tls-transport/tls.crt
plugins.security.ssl.transport.pemkey_filepath: tls-transport/tls.key
plugins.security.ssl.transport.pemtrustedcas_filepath: tls-transport/ca.crt
plugins.security.system_indices.enabled: true
plugins.security.system_indices.indices: [".opendistro-alerting-config",".opendistro-alerting-alert*",".opendistro-anomaly-results*",".opendistro-anomaly-detector*",".opendistro-anomaly-checkpoints",".opendistro-anomaly-detection-state",".opendistro-reports-*",".opendistro-notifications-*",".opendistro-notebooks",".opensearch-observability",".opendistro-asynchronous-search-response*",".replication-metadata-store"]
log4j2.properties: |-
status = error
appender.console.type = Console
appender.console.name = console
appender.console.layout.type = PatternLayout
appender.console.layout.pattern = [%d{ISO8601}][%-5p][%-25c{1.}] [%node_name]%marker %m%n
rootLogger.level = debug
rootLogger.appenderRef.console.ref = console
logger.securityjwt.name = com.amazon.dlic.auth.http.jwt
logger.securityjwt.level = trace
Example of internal users that is referenced above as stored in vault
---
# This is the internal user database
# The hash value is a bcrypt hash and can be generated with plugin/tools/hash.sh
_meta:
type: "internalusers"
config_version: 2
# Define your internal users here
## Demo users
admin:
hash: "$salt."
reserved: true
backend_roles:
- "admin"
description: "Demo admin user"
anomalyadmin:
hash: "$salt."
reserved: false
opendistro_security_roles:
- "anomaly_full_access"
description: "Demo anomaly admin user, using internal role"
kibanaserver:
hash: "$salt."
reserved: true
description: "Demo OpenSearch Dashboards user"
kibanaro:
hash: "$salt."
reserved: false
backend_roles:
- "kibanauser"
- "readall"
attributes:
attribute1: "value1"
attribute2: "value2"
attribute3: "value3"
description: "Demo OpenSearch Dashboards read only user, using external role mapping"
logstash:
hash: "$salt."
reserved: false
backend_roles:
- "logstash"
description: "Demo logstash user, using external role mapping"
readall:
hash: "$salt."
reserved: false
backend_roles:
- "readall"
description: "Demo readall user, using external role mapping"
snapshotrestore:
hash: "$salt."
reserved: false
backend_roles:
- "snapshotrestore"
description: "Demo snapshotrestore user, using external role mapping"
finally the json for keycloak oidc client
{
"clientId": "opensearch-dashboard",
"name": "",
"description": "",
"rootUrl": "",
"adminUrl": "",
"baseUrl": "https://example.com/",
"surrogateAuthRequired": false,
"enabled": true,
"alwaysDisplayInConsole": false,
"clientAuthenticatorType": "client-secret",
"secret": "<redacted>",
"redirectUris": [
"https://example.com/*"
],
"webOrigins": [
"https://example.com"
],
"notBefore": 0,
"bearerOnly": false,
"consentRequired": false,
"standardFlowEnabled": true,
"implicitFlowEnabled": true,
"directAccessGrantsEnabled": false,
"serviceAccountsEnabled": true,
"publicClient": false,
"frontchannelLogout": true,
"protocol": "openid-connect",
"attributes": {
"client.secret.creation.time": "1718778859",
"oauth2.device.authorization.grant.enabled": "false",
"backchannel.logout.revoke.offline.tokens": "false",
"use.refresh.tokens": "true",
"oidc.ciba.grant.enabled": "false",
"backchannel.logout.session.required": "true",
"client_credentials.use_refresh_token": "false",
"acr.loa.map": "{}",
"require.pushed.authorization.requests": "false",
"tls.client.certificate.bound.access.tokens": "false",
"display.on.consent.screen": "false",
"token.response.type.bearer.lower-case": "false"
},
"authenticationFlowBindingOverrides": {},
"fullScopeAllowed": true,
"nodeReRegistrationTimeout": -1,
"protocolMappers": [
{
"name": "Client Host",
"protocol": "openid-connect",
"protocolMapper": "oidc-usersessionmodel-note-mapper",
"consentRequired": false,
"config": {
"user.session.note": "clientHost",
"introspection.token.claim": "true",
"id.token.claim": "true",
"access.token.claim": "true",
"claim.name": "clientHost",
"jsonType.label": "String"
}
},
{
"name": "Client IP Address",
"protocol": "openid-connect",
"protocolMapper": "oidc-usersessionmodel-note-mapper",
"consentRequired": false,
"config": {
"user.session.note": "clientAddress",
"introspection.token.claim": "true",
"id.token.claim": "true",
"access.token.claim": "true",
"claim.name": "clientAddress",
"jsonType.label": "String"
}
},
{
"name": "Client ID",
"protocol": "openid-connect",
"protocolMapper": "oidc-usersessionmodel-note-mapper",
"consentRequired": false,
"config": {
"user.session.note": "client_id",
"introspection.token.claim": "true",
"id.token.claim": "true",
"access.token.claim": "true",
"claim.name": "client_id",
"jsonType.label": "String"
}
}
],
"defaultClientScopes": [
"web-origins",
"acr",
"opensearch",
"roles",
"profile",
"email"
],
"optionalClientScopes": [
"address",
"phone",
"offline_access",
"microprofile-jwt"
],
"access": {
"view": true,
"configure": true,
"manage": true
}
}
Hope this helps
I found this while trying to troubleshoot OpenID not working. The log4j2.properties
changes allowed me to identify that java was not able to validate the certificate for my KeyCloak instance correctly. I had to reissue the certificate without any nameConstraints
(see bug report here)
I found this while trying to troubleshoot OpenID not working. The
log4j2.properties
changes allowed me to identify that java was not able to validate the certificate for my KeyCloak instance correctly. I had to reissue the certificate without anynameConstraints
(see bug report here)
how did you manage to change log4j2.properties? i put in the security-config secret but the changes has no effects
I found this while trying to troubleshoot OpenID not working. The
log4j2.properties
changes allowed me to identify that java was not able to validate the certificate for my KeyCloak instance correctly. I had to reissue the certificate without anynameConstraints
(see bug report here)how did you manage to change log4j2.properties? i put in the security-config secret but the changes has no effects
I copied the post above, setting opensearchCluster.security.config.securityConfigSecret.name
to the secret that had log4j2.properties
as a key and the value exactly from that post. I can't remember if I needed to restart the pods or not for it to take effect.
What is the bug?
Keycloak with opensearch is not working.
How can one reproduce the bug?
Hi All, We have deployed the below configuration file
Following is the security config.
We have configured the keycloak configuration correctly. Also, the session id also provided to opensearch from keycloak. But we are getting the authorization error.
What is the expected behavior?
opensearch should login with keycloak
What is your host/environment?
Operating system, version.
Do you have any screenshots?
If applicable, add screenshots to help explain your problem.
Do you have any additional context?
Add any other context about the problem.