search

OpenUnison / openunison-k8s-login-oidc

Kubernetes login portal for both kubectl and the dashboard using OpenID Connect. Use groups from your assertion in RBAC policies to control access to your cluster. Supports impersonation and OpenID Connect integration with your API server.
https://www.tremolosecurity.com/kubernetes/
Apache License 2.0
12 stars 5 forks source link

Not able to access Openunison UI #38

Open prashanthkumar122 opened 3 years ago

prashanthkumar122 commented 3 years ago

Hi Tremolo team, Not able to access Openunison portal connected with keycloak. After entering https://IP:port, It will redirect to keycloak. and will not redirect back to Openunison portal. Below is my values.yaml file network: openunison_host: "Master-node-IP" dashboard_host: "https://Master-node-IP:30050" api_server_host: "https://Master-node-IP:6443" session_inactivity_timeout_seconds: 900 k8s_url: https://Master-node-IP:6443 createIngressCertificate: true ingress_type: nginx ingress_annotations: kubernetes.io/ingress.class: nginx

cert_template: ou: "Master-node-IP" o: "MyOrg" l: "Kubernetes" st: "State of Cluster" c: "MyCountry"

image: "docker.io/tremolosecurity/openunison-k8s-login-oidc:latest" myvd_config_path: "WEB-INF/myvd.conf" k8s_cluster_name: kubernetes enable_impersonation: false

hosts:

dashboard: namespace: "kubernetes-dashboard" cert_name: "kubernetes-dashboard-certs" label: "k8s-app=kubernetes-dashboard" service_name: kubernetes-dashboard certs: use_k8s_cm: false

trusted_certs: []

monitoring: prometheus_service_account: system:serviceaccount:monitoring:prometheus-k8s

oidc: client_id: test-kubernetes auth_url: https://Ingress-IP.nip.io/auth/ token_url: https://Ingress-IP.nip.io/auth/realms/master/protocol/openid-connect/token user_in_idtoken: false userinfo_url: https://Ingress-IP.nip.io/auth/realms/master/protocol/openid-connect/userinfo domain: "" scopes: openid email profile groups claims: sub: sub email: email given_name: given_name family_name: family_name display_name: name groups: groups

network_policies: enabled: false ingress: enabled: true labels: app.kubernetes.io/name: ingress-nginx namespace: openunison monitoring: enabled: true labels: app.kubernetes.io/name: monitoring apiserver: enabled: false labels: app.kubernetes.io/name: kube-system

services: enable_tokenrequest: false token_request_audience: api token_request_expiration_seconds: 600 node_selectors: [] pullSecret: ""

openunison: replicas: 1 non_secret_data: K8S_DB_SSO: oidc secrets: []

mlbiam commented 3 years ago

openunison_host: "Master-node-IP" dashboard_host: "https://Master-node-IP:30050"

The first issue is your openunison_host and dashboard_host need to be the URLs you want to connect to from your browser. see this graphic: openunison_k8s_network

auth_url: https://Ingress-IP.nip.io/auth/

is your keycloak running in Kubernetes? Your URLs for this section should come from your discovery document. For Keycloak you can get that from https://host/<root>/auth/realms/{realm-name}/.well-known/openid-configuration

prashanthkumar122 commented 3 years ago

openunison_host: "Master-node-IP" dashboard_host: "https://Master-node-IP:30050"

The first issue is your openunison_host and dashboard_host need to be the URLs you want to connect to from your browser. see this graphic:

As Openunison-orchestra is not picking up Ingress LB IP address as openunison_host, so i provied master node IP for openunison_host. I have also tried to deploy on aws eks. One more issue is Kubernetes dashboard will go off if I start orchestra template. Below is the values.yml which I tried to deploy on aws-eks, with gitlab

network: openunison_host: "3.21.205.145" dashboard_host: "https://a4feb48c357b24d9ca5475bad1bfc926-1184502140.us-east-2.elb.amazonaws.com" api_server_host: "D508007F6E6D1FA97E81F4F2F86911D5.gr7.us-east-2.eks.amazonaws.com" session_inactivity_timeout_seconds: 900 k8s_url: https://D508007F6E6D1FA97E81F4F2F86911D5.gr7.us-east-2.eks.amazonaws.com createIngressCertificate: true ingress_type: nginx ingress_annotations: kubernetes.io/ingress.class: nginx

cert_template: ou: "Kubernetes" o: "MyOrg" l: "My Cluster" st: "State of Cluster" c: "MyCountry"

image: "docker.io/tremolosecurity/openunison-k8s-login-oidc:latest" myvd_config_path: "WEB-INF/myvd.conf" k8s_cluster_name: kubernetes enable_impersonation: false

dashboard: namespace: "kubernetes-dashboard" cert_name: "kubernetes-dashboard-certs" label: "k8s-app=kubernetes-dashboard" service_name: kubernetes-dashboard certs: use_k8s_cm: false

trusted_certs: []

- name: idp

#    pem_b64: SDFGSDFGHDFHSDFGSDGSDFGDS

monitoring: prometheus_service_account: system:serviceaccount:monitoring:prometheus-k8s

oidc: client_id: 71ff2a89ced5bfdd09a61ea54004a19b6242b16e092e6c609ae86a612898e207 auth_url: http://3.142.244.2:8080/oauth/authorize token_url: http://3.142.244.2:8080/oauth/token user_in_idtoken: false userinfo_url: https://www.myoidc.com/oauth/userinfo domain: "" scopes: openid email profile groups claims: sub: sub email: email given_name: given_name family_name: family_name display_name: name groups: groups

network_policies: enabled: false ingress: enabled: true labels: app.kubernetes.io/name: ingress-nginx monitoring: enabled: true labels: app.kubernetes.io/name: monitoring apiserver: enabled: false labels: app.kubernetes.io/name: kube-system

services: enable_tokenrequest: false token_request_audience: api token_request_expiration_seconds: 600 node_selectors: [] pullSecret: ""

openunison: replicas: 1 non_secret_data: {} secrets: []

openunison_k8s_network

auth_url: https://Ingress-IP.nip.io/auth/

is your keycloak running in Kubernetes? Your URLs for this section should come from your discovery document. For Keycloak you can get that from https://host/<root>/auth/realms/{realm-name}/.well-known/openid-configuration

mlbiam commented 3 years ago

As Openunison-orchestra is not picking up Ingress LB IP address as openunison_host, so i provied master node IP for openunison_host. I have also tried to deploy on aws eks. One more issue is Kubernetes dashboard will go off if I start orchestra template. Below is the values.yml which I tried to deploy on aws-eks, with gitlab

The openunison_host and dashboard_host settings are what you want your browser to point to. They are the host names that will point to your load ballancer that is sitting in front of your cluster's ingress controller.

prashanthkumar122 commented 3 years ago

Hi Tremolo team, Still facing redirection issue, After entering openunison portal url, redirecting to keycloak, and even after authenticating to keycloak, it is not redirecting to openunison portal. I have provided dns names for both k8s-db and openunison. Below is the values.yml, and Ingress resource also creating. Please help to fix the issue.

network: openunison_host: "openunison.Load-balancer-ip.nip.io" dashboard_host: "k8sdb.Load-balancer-ip.nip.io" api_server_host: "master-node:6443" session_inactivity_timeout_seconds: 900 k8s_url: https://master-node:6443 createIngressCertificate: true ingress_type: nginx ingress_annotations: kubernetes.io/ingress.class: nginx

cert_template: ou: "openunison.Load-balancer-ip.nip.io" o: "MyOrg" l: "Kubernetes" st: "State of Cluster" c: "MyCountry"

image: "docker.io/tremolosecurity/openunison-k8s-login-oidc:latest" myvd_config_path: "WEB-INF/myvd.conf" k8s_cluster_name: kubernetes enable_impersonation: false

hosts:

dashboard: namespace: "kubernetes-dashboard" cert_name: "kubernetes-dashboard-certs" label: "k8s-app=kubernetes-dashboard" service_name: kubernetes-dashboard certs: use_k8s_cm: false

trusted_certs: []

monitoring: prometheus_service_account: system:serviceaccount:monitoring:prometheus-k8s

oidc: client_id: openunison auth_url: https://Load-balancer-ip.nip.io/auth/ token_url: https://Load-balancer-ip.nip.io/auth/realms/master/protocol/openid-connect/token user_in_idtoken: false userinfo_url: https://Load-balancer-ip.nip.io/auth/realms/master/protocol/openid-connect/userinfo domain: "" scopes: openid email profile groups claims: sub: sub email: email given_name: given_name family_name: family_name display_name: name groups: groups

network_policies: enabled: false ingress: enabled: true labels: app.kubernetes.io/name: ingress-nginx namespace: openunison monitoring: enabled: true labels: app.kubernetes.io/name: monitoring apiserver: enabled: false labels: app.kubernetes.io/name: kube-system

services: enable_tokenrequest: false token_request_audience: api token_request_expiration_seconds: 600 node_selectors: [] pullSecret: ""

openunison: replicas: 1 non_secret_data: K8S_DB_SSO: oidc secrets: []

prashanthkumar122 commented 3 years ago

Hi Tremolo team, Got any solution for above mentioned issue.??

mlbiam commented 3 years ago

your auth_url doesn't look right. What does your https://Load-balancer-ip.nip.io/auth/realms/master/protocol/openid-connect/.well-known/openid-configuration look like?

prashanthkumar122 commented 3 years ago

your auth_url doesn't look right. What does your https://Load-balancer-ip.nip.io/auth/realms/master/protocol/openid-connect/.well-known/openid-configuration look like?

After changing auth_url to this, https://LB-IP.nip.io/auth/realms/master/protocol/openid-connect/auth below is the result, and this is the redirect url specified in keycloak, 'https://openunison.LB-IP.nip.io/auth/oidc' image

mlbiam commented 3 years ago

It looks like you haven't configured keycloak with your correct redirect_uri. What did you use for your configuration in keycloak? It should be https://openunison.Load-balancer-ip.nip.io/auth/oidc

prashanthkumar122 commented 3 years ago

It looks like you haven't configured keycloak with your correct redirect_uri. What did you use for your configuration in keycloak? It should be https://openunison.Load-balancer-ip.nip.io/auth/oidc

yes that is what I mentioned in redirect url of keycloak 'https://openunison.Load-balancer-ip.nip.io/auth/oidc'

mlbiam commented 3 years ago

in your browser enable developer tools and turn on Network so you can see the individual urls being requested. you'll see one that looks like https://LB-IP.nip.io/auth/realms/master/protocol/openid-connect/auth?... one of the paramters will be redirect_uri, what is it?

prashanthkumar122 commented 3 years ago

in your browser enable developer tools and turn on Network so you can see the individual urls being requested. you'll see one that looks like https://LB-IP.nip.io/auth/realms/master/protocol/openid-connect/auth?... one of the paramters will be redirect_uri, what is it?

Below is the details from keycloak: https://Load-balancer-ip.nip.io/auth/realms/master/.well-known/openid-configuration

{"issuer":"https://Load-balancer-ip.nip.io/auth/realms/master","authorization_endpoint":"https://Load-balancer-ip.nip.io/auth/realms/master/protocol/openid-connect/auth","token_endpoint":"https://Load-balancer-ip.nip.io/auth/realms/master/protocol/openid-connect/token","introspection_endpoint":"https://Load-balancer-ip.nip.io/auth/realms/master/protocol/openid-connect/token/introspect","userinfo_endpoint":"https://Load-balancer-ip.nip.io/auth/realms/master/protocol/openid-connect/userinfo","end_session_endpoint":"https://Load-balancer-ip.nip.io/auth/realms/master/protocol/openid-connect/logout","jwks_uri":"https://Load-balancer-ip.nip.io/auth/realms/master/protocol/openid-connect/certs","check_session_iframe":"https://Load-balancer-ip.nip.io/auth/realms/master/protocol/openid-connect/login-status-iframe.html","grant_types_supported":["authorization_code","implicit","refresh_token","password","client_credentials"],"response_types_supported":["code","none","id_token","token","id_token token","code id_token","code token","code id_token token"],"subject_types_supported":["public","pairwise"],"id_token_signing_alg_values_supported":["PS384","ES384","RS384","HS256","HS512","ES256","RS256","HS384","ES512","PS256","PS512","RS512"],"id_token_encryption_alg_values_supported":["RSA-OAEP","RSA-OAEP-256","RSA1_5"],"id_token_encryption_enc_values_supported":["A256GCM","A192GCM","A128GCM","A128CBC-HS256","A192CBC-HS384","A256CBC-HS512"],"userinfo_signing_alg_values_supported":["PS384","ES384","RS384","HS256","HS512","ES256","RS256","HS384","ES512","PS256","PS512","RS512","none"],"request_object_signing_alg_values_supported":["PS384","ES384","RS384","HS256","HS512","ES256","RS256","HS384","ES512","PS256","PS512","RS512","none"],"response_modes_supported":["query","fragment","form_post"],"registration_endpoint":"https://Load-balancer-ip.nip.io/auth/realms/master/clients-registrations/openid-connect","token_endpoint_auth_methods_supported":["private_key_jwt","client_secret_basic","client_secret_post","tls_client_auth","client_secret_jwt"],"token_endpoint_auth_signing_alg_values_supported":["PS384","ES384","RS384","HS256","HS512","ES256","RS256","HS384","ES512","PS256","PS512","RS512"],"claims_supported":["aud","sub","iss","auth_time","name","given_name","family_name","preferred_username","email","acr"],"claim_types_supported":["normal"],"claims_parameter_supported":true,"scopes_supported":["openid","address","email","microprofile-jwt","offline_access","phone","profile","roles","web-origins"],"request_parameter_supported":true,"request_uri_parameter_supported":true,"require_request_uri_registration":true,"code_challenge_methods_supported":["plain","S256"],"tls_client_certificate_bound_access_tokens":true,"revocation_endpoint":"https://Load-balancer-ip.nip.io/auth/realms/master/protocol/openid-connect/revoke","revocation_endpoint_auth_methods_supported":["private_key_jwt","client_secret_basic","client_secret_post","tls_client_auth","client_secret_jwt"],"revocation_endpoint_auth_signing_alg_values_supported":["PS384","ES384","RS384","HS256","HS512","ES256","RS256","HS384","ES512","PS256","PS512","RS512"],"backchannel_logout_supported":true,"backchannel_logout_session_supported":true}

prashanthkumar122 commented 3 years ago

now if i enter url: https://openunison.Load-balancer-ip.nip.io, it is is directly displaying error page

image

mlbiam commented 3 years ago

Can you please provide the logs from your openunison-orchestra pod?

prashanthkumar122 commented 3 years ago

Can you please provide the logs from your openunison-orchestra pod?

Hi Tremolo team, It is working for gitlab and okta, If i configure for keycloak, i am getting below error

[2021-03-30 11:11:52,033][XNIO-1 task-1] ERROR OpenIDConnectAuthMech - Could not retrieve token : 400 / Bad Request [2021-03-30 11:11:52,033][XNIO-1 task-1] INFO AccessLog - [AuFail] - scale - https://openunison.LoadBalancer-IP.nip.io/auth/oidc - cn=none - enterprise_idp [10.0.14.203] - [f6e0b22e853fd4370d2640c7031a836c38e258ea0] [2021-03-30 11:11:52,097][XNIO-1 task-1] INFO AccessLog - [Error] - scale - https://openunison.LoadBalancer-IP.nip.io/auth/oidc - uid=Anonymous,o=Tremolo - NONE [10.0.14.203] - [f6e0b22e853fd4370d2640c7031a836c38e258ea0] [2021-03-30 11:11:52,097][XNIO-1 task-1] ERROR ConfigSys - Could not process request javax.servlet.ServletException: Could not load user data at com.tremolosecurity.unison.proxy.auth.openidconnect.OpenIDConnectAuthMech.doGet(OpenIDConnectAuthMech.java:248) ~[unison-auth-openidconnect-1.0.21.jar:?] at com.tremolosecurity.proxy.auth.AuthMgrSys.doAuthMgr(AuthMgrSys.java:191) ~[unison-server-core-1.0.21.jar:?] at com.tremolosecurity.embedd.NextEmbSys.nextSys(NextEmbSys.java:126) ~[unison-server-core-1.0.21.jar:?] at com.tremolosecurity.proxy.auth.AzSys.doAz(AzSys.java:89) ~[unison-sdk-1.0.21.jar:?] at com.tremolosecurity.embedd.NextEmbSys.nextSys(NextEmbSys.java:111) ~[unison-server-core-1.0.21.jar:?] at com.tremolosecurity.proxy.auth.AuthSys.doAuth(AuthSys.java:118) ~[unison-server-core-1.0.21.jar:?] at com.tremolosecurity.embedd.NextEmbSys.nextSys(NextEmbSys.java:105) ~[unison-server-core-1.0.21.jar:?] at com.tremolosecurity.proxy.ConfigSys.doConfig(ConfigSys.java:296) [unison-server-core-1.0.21.jar:?] at com.tremolosecurity.embedd.NextEmbSys.nextSys(NextEmbSys.java:93) [unison-server-core-1.0.21.jar:?] at com.tremolosecurity.filter.UnisonServletFilter.doFilter(UnisonServletFilter.java:290) [unison-server-core-1.0.21.jar:?] at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61) [undertow-servlet-2.2.5.Final.jar:2.2.5.Final] at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131) [undertow-servlet-2.2.5.Final.jar:2.2.5.Final] at io.undertow.servlet.handlers.FilterHandler.handleRequest(FilterHandler.java:84) [undertow-servlet-2.2.5.Final.jar:2.2.5.Final] at io.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:62) [undertow-servlet-2.2.5.Final.jar:2.2.5.Final] at io.undertow.servlet.handlers.ServletChain$1.handleRequest(ServletChain.java:68) [undertow-servlet-2.2.5.Final.jar:2.2.5.Final] at io.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36) [undertow-servlet-2.2.5.Final.jar:2.2.5.Final] at io.undertow.servlet.handlers.RedirectDirHandler.handleRequest(RedirectDirHandler.java:68) [undertow-servlet-2.2.5.Final.jar:2.2.5.Final] at io.undertow.servlet.handlers.security.SSLInformationAssociationHandler.handleRequest(SSLInformationAssociationHandler.java:117) [undertow-servlet-2.2.5.Final.jar:2.2.5.Final] at io.undertow.servlet.handlers.security.ServletAuthenticationCallHandler.handleRequest(ServletAuthenticationCallHandler.java:57) [undertow-servlet-2.2.5.Final.jar:2.2.5.Final] at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) [undertow-core-2.2.5.Final.jar:2.2.5.Final] at io.undertow.security.handlers.AbstractConfidentialityHandler.handleRequest(AbstractConfidentialityHandler.java:46) [undertow-core-2.2.5.Final.jar:2.2.5.Final] at io.undertow.servlet.handlers.security.ServletConfidentialityConstraintHandler.handleRequest(ServletConfidentialityConstraintHandler.java:64) [undertow-servlet-2.2.5.Final.jar:2.2.5.Final] at io.undertow.security.handlers.AuthenticationMechanismsHandler.handleRequest(AuthenticationMechanismsHandler.java:60) [undertow-core-2.2.5.Final.jar:2.2.5.Final] at io.undertow.servlet.handlers.security.CachedAuthenticatedSessionHandler.handleRequest(CachedAuthenticatedSessionHandler.java:77) [undertow-servlet-2.2.5.Final.jar:2.2.5.Final] at io.undertow.security.handlers.AbstractSecurityContextAssociationHandler.handleRequest(AbstractSecurityContextAssociationHandler.java:43) [undertow-core-2.2.5.Final.jar:2.2.5.Final] at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) [undertow-core-2.2.5.Final.jar:2.2.5.Final] at io.undertow.servlet.handlers.SendErrorPageHandler.handleRequest(SendErrorPageHandler.java:52) [undertow-servlet-2.2.5.Final.jar:2.2.5.Final] at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) [undertow-core-2.2.5.Final.jar:2.2.5.Final] at io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:269) [undertow-servlet-2.2.5.Final.jar:2.2.5.Final] at io.undertow.servlet.handlers.ServletInitialHandler.access$100(ServletInitialHandler.java:78) [undertow-servlet-2.2.5.Final.jar:2.2.5.Final] at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:133) [undertow-servlet-2.2.5.Final.jar:2.2.5.Final] at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:130) [undertow-servlet-2.2.5.Final.jar:2.2.5.Final] at io.undertow.servlet.core.ServletRequestContextThreadSetupAction$1.call(ServletRequestContextThreadSetupAction.java:48) [undertow-servlet-2.2.5.Final.jar:2.2.5.Final] at io.undertow.servlet.core.ContextClassLoaderSetupAction$1.call(ContextClassLoaderSetupAction.java:43) [undertow-servlet-2.2.5.Final.jar:2.2.5.Final] at io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:249) [undertow-servlet-2.2.5.Final.jar:2.2.5.Final] at io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:78) [undertow-servlet-2.2.5.Final.jar:2.2.5.Final] at io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:99) [undertow-servlet-2.2.5.Final.jar:2.2.5.Final] at io.undertow.server.Connectors.executeRootHandler(Connectors.java:387) [undertow-core-2.2.5.Final.jar:2.2.5.Final] at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:841) [undertow-core-2.2.5.Final.jar:2.2.5.Final] at org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35) [jboss-threads-2.3.3.Final.jar:2.3.3.Final] at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1982) [jboss-threads-2.3.3.Final.jar:2.3.3.Final] at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1486) [jboss-threads-2.3.3.Final.jar:2.3.3.Final] at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1377) [jboss-threads-2.3.3.Final.jar:2.3.3.Final] at org.xnio.XnioWorker$WorkerThreadFactory$1$1.run(XnioWorker.java:1280) [xnio-api-3.8.4.Final.jar:3.8.4.Final] at java.lang.Thread.run(Thread.java:834) [?:?] Caused by: java.lang.Exception: Could not retrieve token : 401 / Unauthorized at com.tremolosecurity.unison.proxy.auth.openidconnect.loadUser.LoadAttributesFromWS.loadUserAttributesFromIdP(LoadAttributesFromWS.java:55) ~[unison-auth-openidconnect-1.0.21.jar:?] at com.tremolosecurity.unison.proxy.auth.openidconnect.OpenIDConnectAuthMech.doGet(OpenIDConnectAuthMech.java:246) ~[unison-auth-openidconnect-1.0.21.jar:?] ... 44 more

mlbiam commented 3 years ago

Take a look at the keycloak logs, does it complain about invalid scopes?

prashanthkumar122 commented 3 years ago

Take a look at the keycloak logs, does it complain about invalid scopes?

yes it has error logs of invalid scopes.

06:43:31,686 INFO [org.jboss.as] (Controller Boot Thread) WFLYSRV0051: Admin console listening on http://127.0.0.1:9990 06:43:33,880 INFO [org.keycloak.keys.DefaultKeyManager] (default task-2) No keys found for realm=master and algorithm=RS256 for use=SIG. Generating keys. 06:43:34,192 INFO [org.keycloak.keys.DefaultKeyManager] (default task-1) No keys found for realm=master and algorithm=RS256 for use=SIG. Generating keys. 06:43:34,305 INFO [org.keycloak.keys.DefaultKeyManager] (default task-3) No keys found for realm=master and algorithm=RS256 for use=SIG. Generating keys. 04:50:22,720 WARN [org.jboss.as.domain.management.security] (default I/O-2) WFLYDM0113: Generated self signed certificate at /opt/jboss/keycloak/standalone/configuration/application.keystore. Please note that self signed certificates are not secure, and should only be used for testing purposes. Do not use this self signed certificate in production. SHA-1 fingerprint of the generated key is 73:a5:f4:5a:8f:c6:90:f6:16:e2:27:1a:18:1e:62:e4:33:64:a4:bc SHA-256 fingerprint of the generated key is 77:14:54:24:c2:87:f8:fe:0b:7a:80:ba:6e:ed:4d:f5:6b:e5:11:b0:d2:15:5f:d4:bf:70:5d:bd:c3:df:56:59 04:51:20,743 INFO [org.keycloak.keys.DefaultKeyManager] (default task-10) No keys found for realm=master and algorithm=HS256 for use=SIG. Generating keys. 09:40:23,236 ERROR [org.keycloak.services] (default task-30) KC-SERVICES0093: Invalid parameter value for: scope 09:40:23,237 WARN [org.keycloak.events] (default task-30) type=LOGIN_ERROR, realmId=master, clientId=open-kube, userId=null, ipAddress=10.81.232.30, error=invalid_request, response_type=code, redirect_uri=https://openunison.10.0.14.251.nip.io/auth/oidc, response_mode=query 09:40:29,241 WARN [org.keycloak.protocol.oidc.utils.OAuth2CodeParser] (default task-30) Invalid format of the code 09:40:29,254 WARN [org.keycloak.events] (default task-30) type=CODE_TO_TOKEN_ERROR, realmId=master, clientId=open-kube, userId=null, ipAddress=192.168.33.22, error=invalid_code, grant_type=authorization_code, client_auth_method=client-secret 09:40:29,316 WARN [org.keycloak.events] (default task-30) type=USER_INFO_REQUEST_ERROR, realmId=master, clientId=null, userId=null, ipAddress=192.168.33.22, error=invalid_token, auth_method=validate_access_token

After this i removed groups in scope, login works, but it shows unauthourized while executing get pods kubectl get pods error: You must be logged in to the server (Unauthorized)

mlbiam commented 3 years ago

It looks like Keycloak doesn't support the groups scope. To support keycloak:

  1. In values.yaml change oidc.scopes from openid email profile groups to openid email profile
  2. Update your helm deployment - helm upgrade orchestra tremolo/openunison-k8s-login-oidc --namespace openunison -f ~/path/to/values-k8s-login-oidc-kc.yaml
  3. In Keycloak, if you want to add groups to your id_token, click on Mappers for your client and click on the Create button. Fill in per the screenshot: keycloakmapper

Out of the box, keycloak assigns group names as the name of the group after a "/". So in your RBAC bindings you'll use that instead of just the name of the group. For instance if you have a group called group1 your RBAC binding would look like:

kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: activedirectory-cluster-admins
subjects:
- kind: Group
  name: /group1
roleRef:
  kind: ClusterRole
  name: cluster-admin
  apiGroup: rbac.authorization.k8s.io
prashanthkumar122 commented 3 years ago

It looks like Keycloak doesn't support the groups scope. To support keycloak:

  1. In values.yaml change oidc.scopes from openid email profile groups to openid email profile
  2. Update your helm deployment - helm upgrade orchestra tremolo/openunison-k8s-login-oidc --namespace openunison -f ~/path/to/values-k8s-login-oidc-kc.yaml
  3. In Keycloak, if you want to add groups to your id_token, click on Mappers for your client and click on the Create button. Fill in per the screenshot: keycloakmapper

Out of the box, keycloak assigns group names as the name of the group after a "/". So in your RBAC bindings you'll use that instead of just the name of the group. For instance if you have a group called group1 your RBAC binding would look like:

kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: activedirectory-cluster-admins
subjects:
- kind: Group
  name: /group1
roleRef:
  kind: ClusterRole
  name: cluster-admin
  apiGroup: rbac.authorization.k8s.io

Hi , I followed instructions, copied the token from openunison UI, but I am getting below error if I execute 'kubectl get pods' kubectl get pods error: You must be logged in to the server (Unauthorized)

mlbiam commented 3 years ago

error: You must be logged in to the server (Unauthorized)

this generally means you either didn't complete the integration between the API server and OpenUnison or it's misconfigured. Take a look at the API server logs for what the issue is.

prashanthkumar122 commented 3 years ago

error: You must be logged in to the server (Unauthorized)

this generally means you either didn't complete the integration between the API server and OpenUnison or it's misconfigured. Take a look at the API server logs for what the issue is.

hi, already added this in kube-api-serv

and also ou-ca.pem i got it from openunison gui

mlbiam commented 3 years ago

What do the API server logs say?

prashanthkumar122 commented 3 years ago

What do the API server logs say?

I0413 18:01:34.275908 1 client.go:360] parsed scheme: "passthrough" I0413 18:01:34.275952 1 passthrough.go:48] ccResolverWrapper: sending update to cc: {[{https://127.0.0.1:2379 0 }] } I0413 18:01:34.275964 1 clientconn.go:948] ClientConn switching balancer to "pick_first" E0413 18:01:37.367907 1 oidc.go:224] oidc authenticator: initializing plugin: Get "https://openunison.Load-Balancer-IP.nip.io/auth/idp/k8sIdp/.well-known/openid-configuration": x509: certificate signed by unknown authority (possibly because of "x509: invalid signature: parent certificate cannot sign this kind of certificate" while trying to verify candidate authority certificate "openunison.Load-Balancer-IP.nip.io")

mlbiam commented 3 years ago

E0413 18:01:37.367907 1 oidc.go:224] oidc authenticator: initializing plugin: Get "https://openunison.Load-Balancer-IP.nip.io/auth/idp/k8sIdp/.well-known/openid-configuration": x509: certificate signed by unknown authority (possibly because of "x509: invalid signature: parent certificate cannot sign this kind of certificate" while trying to verify candidate authority certificate "openunison.Load-Balancer-IP.nip.io")

does your load balancer have it's own certificate?

prashanthkumar122 commented 3 years ago

No, it doesnot have.

mlbiam commented 3 years ago

When you login to the openunison UI and you look at the certificate in your browser, what does it say?

prashanthkumar122 commented 3 years ago

When you login to the openunison UI and you look at the certificate in your browser, what does it say?

image

And below is the openunison-orcheshtra logs

[2021-04-20 11:12:41,362][XNIO-1 task-10] INFO AccessLog - [AzSuccess] - k8sIdp - https://openunison.10.0.14.251.nip.io/auth/idp/k8sIdp/token - uid=x-56-xx-51-xdex-52-xx-56-xfd-x-49-xcx-50-xx-48-x-x-52-xx-51-xx-49-xa-x-57-xx-51-xx-57-xx-57-x-x-57-xx-55-xx-50-xx-53-xcx-53-xcdx-52-xx-48-xcx-48-x,ou=shadow,o=Tremolo - NONE [10.0.14.203] - [fa812ecf9b7db09230d7bf0e16b10e56dc4e2bd18]

and kube-apiserver logs

E0420 11:42:17.367198 1 oidc.go:224] oidc authenticator: initializing plugin: Get "https://openunison.10.0.14.251.nip.io/auth/idp/k8sIdp/.well-known/openid-configuration": x509: certificate signed by unknown authority (possibly because of "x509: invalid signature: parent certificate cannot sign this kind of certificate" while trying to verify candidate authority certificate "openunison.LoadBalancer-IP.nip.io")

But it was working fine when i connect with gitlab on http

prashanthkumar122 commented 3 years ago

HI any update on the above issue

mlbiam commented 3 years ago

is the certificate hash for the certificate you're displaying the same as what your API server is trusting? If there have been multiple redeployments the wrong cert may be trusted now.

prashanthkumar122 commented 3 years ago

is the certificate hash for the certificate you're displaying the same as what your API server is trusting? If there have been multiple redeployments the wrong cert may be trusted now.

I have redeployed by deleting the 'openunison' namespace. I have update the ou-ca.pem also Still same error. error: You must be logged in to the server (Unauthorized)

prashanthkumar122 commented 3 years ago

hi , I have upgraded the cluster to 21.0.0, and redeployed, but still same output, error: You must be logged in to the server (Unauthorized)

But kubeapi server error logs got changed E0427 10:54:43.584868 1 authentication.go:63] "Unable to authenticate the request" err="[invalid bearer token, Token has been invalidated]" E0427 10:54:45.511535 1 authentication.go:63] "Unable to authenticate the request" err="[invalid bearer token, Token has