OpenUnison / openunison-k8s-activedirectory

Self service portal for Kubernetes. Automate provisioning and access of namespaces, authenticate users using Active Directory or LDAP.
https://www.tremolosecurity.com/kubernetes/
Apache License 2.0
15 stars 6 forks source link

PKIX path building failed: #14

Open devopstales opened 4 years ago

devopstales commented 4 years ago

When I try to use the Dashboard I get this error:

Error
An error occurred while processing this request. Please see the system administrator for assistance.

Log for Error:

ERROR ConfigSys - Could not process request
javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
    at sun.security.ssl.Alerts.getSSLException(Alerts.java:192) ~[?:1.8.0_222]
    at sun.security.ssl.SSLSocketImpl.fatal(SSLSocketImpl.java:1946) ~[?:1.8.0_222]
    at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:316) ~[?:1.8.0_222]
    at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:310) ~[?:1.8.0_222]
    at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1639) ~[?:1.8.0_222]
    at sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:223) ~[?:1.8.0_222]
    at sun.security.ssl.Handshaker.processLoop(Handshaker.java:1037) ~[?:1.8.0_222]
    at sun.security.ssl.Handshaker.process_record(Handshaker.java:965) ~[?:1.8.0_222]
    at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:1064) ~[?:1.8.0_222]
    at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1367) ~[?:1.8.0_222]
    at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1395) ~[?:1.8.0_222]
    at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1379) ~[?:1.8.0_222]
    at org.apache.http.conn.ssl.SSLConnectionSocketFactory.createLayeredSocket(SSLConnectionSocketFactory.java:436) ~[httpclient-4.5.9.jar:4.5.9]
    at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:384) ~[httpclient-4.5.9.jar:4.5.9]
    at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ~[httpclient-4.5.9.jar:4.5.9]
    at org.apache.http.impl.conn.BasicHttpClientConnectionManager.connect(BasicHttpClientConnectionManager.java:313) ~[httpclient-4.5.9.jar:4.5.9]
    at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) ~[httpclient-4.5.9.jar:4.5.9]
    at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) ~[httpclient-4.5.9.jar:4.5.9]
    at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) ~[httpclient-4.5.9.jar:4.5.9]
    at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) ~[httpclient-4.5.9.jar:4.5.9]
    at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) ~[httpclient-4.5.9.jar:4.5.9]
    at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) ~[httpclient-4.5.9.jar:4.5.9]
    at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) ~[httpclient-4.5.9.jar:4.5.9]
    at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:108) ~[httpclient-4.5.9.jar:4.5.9]
    at com.tremolosecurity.unison.proxy.auth.openidconnect.OpenIDConnectAuthMech.doGet(OpenIDConnectAuthMech.java:206) ~[unison-auth-openidconnect-1.0.17.jar:?]
    at com.tremolosecurity.proxy.auth.AuthMgrSys.doAuthMgr(AuthMgrSys.java:191) ~[unison-server-core-1.0.17.jar:?]
    at com.tremolosecurity.embedd.NextEmbSys.nextSys(NextEmbSys.java:126) ~[unison-server-core-1.0.17.jar:?]
    at com.tremolosecurity.proxy.auth.AzSys.doAz(AzSys.java:89) ~[unison-sdk-1.0.17.jar:?]
    at com.tremolosecurity.embedd.NextEmbSys.nextSys(NextEmbSys.java:111) ~[unison-server-core-1.0.17.jar:?]
    at com.tremolosecurity.proxy.auth.AuthSys.doAuth(AuthSys.java:118) ~[unison-server-core-1.0.17.jar:?]
    at com.tremolosecurity.embedd.NextEmbSys.nextSys(NextEmbSys.java:105) ~[unison-server-core-1.0.17.jar:?]
    at com.tremolosecurity.proxy.ConfigSys.doConfig(ConfigSys.java:293) [unison-server-core-1.0.17.jar:?]
    at com.tremolosecurity.embedd.NextEmbSys.nextSys(NextEmbSys.java:93) [unison-server-core-1.0.17.jar:?]
    at com.tremolosecurity.filter.UnisonServletFilter.doFilter(UnisonServletFilter.java:290) [unison-server-core-1.0.17.jar:?]
    at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61) [undertow-servlet-2.0.22.Final.jar:2.0.22.Final]
    at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131) [undertow-servlet-2.0.22.Final.jar:2.0.22.Final]
    at io.undertow.servlet.handlers.FilterHandler.handleRequest(FilterHandler.java:84) [undertow-servlet-2.0.22.Final.jar:2.0.22.Final]
    at io.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:62) [undertow-servlet-2.0.22.Final.jar:2.0.22.Final]
    at io.undertow.servlet.handlers.ServletChain$1.handleRequest(ServletChain.java:68) [undertow-servlet-2.0.22.Final.jar:2.0.22.Final]
    at io.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36) [undertow-servlet-2.0.22.Final.jar:2.0.22.Final]
    at io.undertow.servlet.handlers.security.SSLInformationAssociationHandler.handleRequest(SSLInformationAssociationHandler.java:132) [undertow-servlet-2.0.22.Final.jar:2.0.22.Final]
    at io.undertow.servlet.handlers.security.ServletAuthenticationCallHandler.handleRequest(ServletAuthenticationCallHandler.java:57) [undertow-servlet-2.0.22.Final.jar:2.0.22.Final]
    at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) [undertow-core-2.0.22.Final.jar:2.0.22.Final]
    at io.undertow.security.handlers.AbstractConfidentialityHandler.handleRequest(AbstractConfidentialityHandler.java:46) [undertow-core-2.0.22.Final.jar:2.0.22.Final]
    at io.undertow.servlet.handlers.security.ServletConfidentialityConstraintHandler.handleRequest(ServletConfidentialityConstraintHandler.java:64) [undertow-servlet-2.0.22.Final.jar:2.0.22.Final]
    at io.undertow.security.handlers.AuthenticationMechanismsHandler.handleRequest(AuthenticationMechanismsHandler.java:60) [undertow-core-2.0.22.Final.jar:2.0.22.Final]
    at io.undertow.servlet.handlers.security.CachedAuthenticatedSessionHandler.handleRequest(CachedAuthenticatedSessionHandler.java:77) [undertow-servlet-2.0.22.Final.jar:2.0.22.Final]
    at io.undertow.security.handlers.AbstractSecurityContextAssociationHandler.handleRequest(AbstractSecurityContextAssociationHandler.java:43) [undertow-core-2.0.22.Final.jar:2.0.22.Final]
    at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) [undertow-core-2.0.22.Final.jar:2.0.22.Final]
    at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) [undertow-core-2.0.22.Final.jar:2.0.22.Final]
    at io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:292) [undertow-servlet-2.0.22.Final.jar:2.0.22.Final]
    at io.undertow.servlet.handlers.ServletInitialHandler.access$100(ServletInitialHandler.java:81) [undertow-servlet-2.0.22.Final.jar:2.0.22.Final]
    at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:138) [undertow-servlet-2.0.22.Final.jar:2.0.22.Final]
    at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:135) [undertow-servlet-2.0.22.Final.jar:2.0.22.Final]
    at io.undertow.servlet.core.ServletRequestContextThreadSetupAction$1.call(ServletRequestContextThreadSetupAction.java:48) [undertow-servlet-2.0.22.Final.jar:2.0.22.Final]
    at io.undertow.servlet.core.ContextClassLoaderSetupAction$1.call(ContextClassLoaderSetupAction.java:43) [undertow-servlet-2.0.22.Final.jar:2.0.22.Final]
    at io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:272) [undertow-servlet-2.0.22.Final.jar:2.0.22.Final]
    at io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:81) [undertow-servlet-2.0.22.Final.jar:2.0.22.Final]
    at io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:104) [undertow-servlet-2.0.22.Final.jar:2.0.22.Final]
    at io.undertow.server.Connectors.executeRootHandler(Connectors.java:376) [undertow-core-2.0.22.Final.jar:2.0.22.Final]
    at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:830) [undertow-core-2.0.22.Final.jar:2.0.22.Final]
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_222]
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_222]
    at java.lang.Thread.run(Thread.java:748) [?:1.8.0_222]
Caused by: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
    at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:397) ~[?:1.8.0_222]
    at sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:302) ~[?:1.8.0_222]
    at sun.security.validator.Validator.validate(Validator.java:262) ~[?:1.8.0_222]
    at sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:330) ~[?:1.8.0_222]
    at sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:237) ~[?:1.8.0_222]
    at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:132) ~[?:1.8.0_222]
    at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1621) ~[?:1.8.0_222]
    ... 59 more
Caused by: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
    at sun.security.provider.certpath.SunCertPathBuilder.build(SunCertPathBuilder.java:141) ~[?:1.8.0_222]
    at sun.security.provider.certpath.SunCertPathBuilder.engineBuild(SunCertPathBuilder.java:126) ~[?:1.8.0_222]
    at java.security.cert.CertPathBuilder.build(CertPathBuilder.java:280) ~[?:1.8.0_222]
    at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:392) ~[?:1.8.0_222]
    at sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:302) ~[?:1.8.0_222]
    at sun.security.validator.Validator.validate(Validator.java:262) ~[?:1.8.0_222]
    at sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:330) ~[?:1.8.0_222]
    at sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:237) ~[?:1.8.0_222]
    at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:132) ~[?:1.8.0_222]
    at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1621) ~[?:1.8.0_222]
    ... 59 more
mlbiam commented 4 years ago

What version of the dashboard are you using and what distribution of Kubernetes are you using?

mlbiam commented 4 years ago

Also, what secrets are there in your dashboard namspace?

devopstales commented 4 years ago

Kubernetes: v1.16.1 Dashboard: dashboard:v2.0.0-beta8 secrets in namespace:

NAME                               TYPE                                  DATA   AGE
default-token-prbmb                kubernetes.io/service-account-token   3      3m
kubernetes-dashboard-certs         Opaque                                0      3m
kubernetes-dashboard-csrf          Opaque                                1      3m
kubernetes-dashboard-key-holder    Opaque                                2      3m
kubernetes-dashboard-tls           kubernetes.io/tls                     3      103s
kubernetes-dashboard-token-k78jh   kubernetes.io/service-account-token   3      3m
mlbiam commented 4 years ago

take a look at your orchestra cr (kubectl edit openunison orchestra -n openunison), look for a block like this in `spec.key_store.keys[2]:

- create_data:
          ca_cert: true
          delete_pods_labels:
          - k8s-app=kubernetes-dashboard
          key_size: 2048
          secret_info:
            cert_name: dashboard.crt
            key_name: dashboard.key
            type_of_secret: Opaque
          server_name: kubernetes-dashboard.kubernetes-dashboard.svc.cluster.local
          sign_by_k8s_ca: false
          subject_alternative_names: []
          target_namespace: kubernetes-dashboard
        import_into_ks: certificate
        name: kubernetes-dashboard
        replace_if_exists: true
        tls_secret_name: kubernetes-dashboard-certs

does target_namespace say kubernetes-dashboard or kube-system?

devopstales commented 4 years ago

My dashboard is in a custom namespace cald kubernetes-dashboard-system and this is my config:

     - create_data:
          ca_cert: true
          delete_pods_labels:
          - k8s-app=kubernetes-dashboard
          key_size: 2048
          secret_info:
            cert_name: dashboard.crt
            key_name: dashboard.key
            type_of_secret: Opaque
          server_name: kubernetes-dashboard.kubernetes-dashboard-system.svc.cluster.local
          sign_by_k8s_ca: false
          subject_alternative_names: []
          target_namespace: kubernetes-dashboard-system
        import_into_ks: certificate
        name: kubernetes-dashboard
        replace_if_exists: true
        tls_secret_name: kubernetes-dashboard-certs
mlbiam commented 4 years ago

hm, should have updated the secret. Take a look at the kubernetes-dashboard-certs secret. Does it have any annotations on it? Are you using a custom secret name for your dashboard cert? What other certs are in the namespace?

if you exec into the shell for openunison you can check the cert:

kubectl exec -ti openunison-orchestra-857bd56b97-kwn7t -n openunison  -- openssl s_client -connect 'kubernetes-dashboard.kubernetes-dashboard-system.svc.cluster.local:443' --showcerts

Whats the subject say? Should be C = US, ST = Virginia, L = Alexandria, O = Tremolo Security, OU = k8s, CN = kubernetes-dashboard-sytem.kubernetes-dashboard.svc.cluster.local

devopstales commented 4 years ago

I did't change the name of the dashboard cert. It is kubernetes-dashboard-certs as you can see:

NAME                               TYPE                                  DATA   AGE
default-token-kchdt                kubernetes.io/service-account-token   3      47h
kubernetes-dashboard-certs         Opaque                                2      47h
kubernetes-dashboard-csrf          Opaque                                1      47h
kubernetes-dashboard-key-holder    Opaque                                2      47h
kubernetes-dashboard-tls           kubernetes.io/tls                     3      47h
kubernetes-dashboard-token-z7k8h   kubernetes.io/service-account-token   3      47h

Thi is my secret:

apiVersion: v1
data:
  dashboard.crt: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUVZRENDQTBpZ0F3SUJBZ0lHQVhGNVp4d2lNQTBHQ1NxR1NJYjNEUUVCQ3dVQU1JR3JNUXN3Q1FZRFZRUUcNCkV3SlZVekVSTUE4R0ExVUVDQk1JVm1seVoybHVhV0V4RXpBUkJnTlZCQWNUQ2tGc1pYaGhibVJ5YVdFeEdUQVgNCkJnTlZCQW9URUZSeVpXMXZiRzhnVTJWamRYSnBkSGt4RERBS0JnTlZCQXNUQTJzNGN6RkxNRWtHQTFVRUF4TkMNCmEzVmlaWEp1WlhSbGN5MWtZWE5vWW05aGNtUXVhM1ZpWlhKdVpYUmxjeTFrWVhOb1ltOWhjbVF0YzNsemRHVnQNCkxuTjJZeTVqYkhWemRHVnlMbXh2WTJGc01CNFhEVEl3TURReE5ERTFOVFV6TWxvWERUSXhNRFF4TkRFMU5UVXoNCk1sb3dnYXN4Q3pBSkJnTlZCQVlUQWxWVE1SRXdEd1lEVlFRSUV3aFdhWEpuYVc1cFlURVRNQkVHQTFVRUJ4TUsNClFXeGxlR0Z1WkhKcFlURVpNQmNHQTFVRUNoTVFWSEpsYlc5c2J5QlRaV04xY21sMGVURU1NQW9HQTFVRUN4TUQNCmF6aHpNVXN3U1FZRFZRUURFMEpyZFdKbGNtNWxkR1Z6TFdSaGMyaGliMkZ5WkM1cmRXSmxjbTVsZEdWekxXUmgNCmMyaGliMkZ5WkMxemVYTjBaVzB1YzNaakxtTnNkWE4wWlhJdWJHOWpZV3d3Z2dFaU1BMEdDU3FHU0liM0RRRUINCkFRVUFBNElCRHdBd2dnRUtBb0lCQVFDUWZSVng1QU5RUm5xSWx1REV6NWwzUUZKZWRaYU5qaU1jQUpZRGpCZWYNClRHQzdpZDFZNkMydE5HcThRNnVpYlZlejgrcjJndjg2NU95N0tmUjVxbTBQL2ZEREVTa1J5RWhhL1BqSkVzTVANClg2elhLaHpJdnNaNzR5eDlnSE9mN3p0VUhwSHRCeVJxQmQ3ZjUxOWxzYWFKTFRKVEdqNzhiQWFBNkRORktXQzcNClh3bEdxWXZRZ2p1RUVybWIvUm5nL2lNKy9YakJKbEdlTGo1bXNWdzRlaEcySVh0K0VNQzRiZis1UkZaN1hETTINClBiZFdEZHRYL1dIdVJPU01RWEloZUpnbEN0SjQ0MVpNYnZVOGhrYTJZMmx2MUJ5ZGMyUTRVQmUwcUFVclBnanQNCjUzVUNVSHNZeXNmNDdMSWQ2WXgrYWNmY1c5NGtGTWlUczJOb0d1S1J3RUkvQWdNQkFBR2pnWWN3Z1lRd0R3WUQNClZSMFRBUUgvQkFVd0F3RUIvekFPQmdOVkhROEJBZjhFQkFNQ0FnUXdFZ1lEVlIwbEFRSC9CQWd3QmdZRVZSMGwNCkFEQk5CZ05WSFJFRVJqQkVna0pyZFdKbGNtNWxkR1Z6TFdSaGMyaGliMkZ5WkM1cmRXSmxjbTVsZEdWekxXUmgNCmMyaGliMkZ5WkMxemVYTjBaVzB1YzNaakxtTnNkWE4wWlhJdWJHOWpZV3d3RFFZSktvWklodmNOQVFFTEJRQUQNCmdnRUJBQmdpd3RPM1dJZGYwbzdaUkMxQXdBVjNHa3cyNjFucDcxcUJDd0xKZ0Z6T242dUxWNW9YWmNXYWd5OEwNCjlLRVhWZzQ3Q0cwc2cyNDJIRzJCZUhkQkRlTFJNek15cWhvbEdGTWY2S1kybTBEUHRvbVpiOHZVYjlvVmFCSDENCkMvaDR0R0JObDAveC9mRTFiand2N2ZNN2VSQU1CcGtEbSt6SnJraXZ3VjFLc2drTlZoVjRMVU4rWW9VdDhtNHoNCmRYbUk4VzBjclgwSTNyVmx0eEp3Z2xVQ05QRVNSQkI1VXpISXhhL3JNcXp0NVQ0NDlNam1YbDdkZzFwNWZRdVcNCnRVQUFEZG5hYzJ4RXhsUWovWkJJN0tqbzZHVGRCY3FTcVo5dE5GS3NzekF1ejRnbHh5ZTlYTEoxQVV4QkYycUMNCjhzSC9aQnV2dk5wdUpuM1VXWmNpc1hsa2pjdz0NCi0tLS0tRU5EIENFUlRJRklDQVRFLS0tLS0K
  dashboard.key: LS0tLS1CRUdJTiBQUklWQVRFIEtFWS0tLS0tCk1JSUV2UUlCQURBTkJna3Foa2lHOXcwQkFRRUZBQVNDQktjd2dnU2pBZ0VBQW9JQkFRQ1FmUlZ4NUFOUVJucUkKbHVERXo1bDNRRkplZFphTmppTWNBSllEakJlZlRHQzdpZDFZNkMydE5HcThRNnVpYlZlejgrcjJndjg2NU95NwpLZlI1cW0wUC9mRERFU2tSeUVoYS9QakpFc01QWDZ6WEtoekl2c1o3NHl4OWdIT2Y3enRVSHBIdEJ5UnFCZDdmCjUxOWxzYWFKTFRKVEdqNzhiQWFBNkRORktXQzdYd2xHcVl2UWdqdUVFcm1iL1JuZy9pTSsvWGpCSmxHZUxqNW0Kc1Z3NGVoRzJJWHQrRU1DNGJmKzVSRlo3WERNMlBiZFdEZHRYL1dIdVJPU01RWEloZUpnbEN0SjQ0MVpNYnZVOApoa2EyWTJsdjFCeWRjMlE0VUJlMHFBVXJQZ2p0NTNVQ1VIc1l5c2Y0N0xJZDZZeCthY2ZjVzk0a0ZNaVRzMk5vCkd1S1J3RUkvQWdNQkFBRUNnZ0VBQmZtR0ozMThQWmVNZlpWdm1mUFRyc0MwSnovSmgvVDJzck83Z0U0TXRBT2UKaEdMQkNtb2ZhV2dmZWNWNmxXN2NENDdvaW1VRVZOVWVqT2JCT3o2MjM5QnFZa2FWWWl6Y0FSY3cybSt5TFRLUgpuUndhc0pwaTdzQ0lHSEIxVjRtMjlGV0hSenZpWDVmNmpWdXQ4RENXYUZtOEdQK2UxeVZJWjVBeWZvdWxlbG4xCndzZjFkb2Y1V205VG53ZWxSdTVPTlVyLzF5ZFc3c0N6QlBXR084VDFiV2lSWEhnMlNwQzBFdjlPcHVEUjAyeHMKYy9WRXFKYkFkQlB1c0x0YzRSVm83MlRhWm9yZ056RXlmWE0rZFNwbGtUSldzSzZGQWV3eVBSUXdrYllZczBWMQp0L3FJYmUxOTJEZ0xtcGMwUFlvRjRXdXJqbytIL01LM3AxSW9qajU0c1FLQmdRRE1hUi9KNW54d2RrZGZYbzVvCnNueEgycWhtdEdQTW1oS0pRU3ZkRTgySlJ1SE5DZmhWZXI2L09OZ3ZTQ3BvTU9EUEs1bWd2UVYrRWRTOHc0cWUKMFlRK1BOOHpTaDdicWw1N2YzTnVOUStTQTFveHBYb3Y0VS9IdmhTcUY3Zm9LT3pFRGdXQU5zZlM3dU9yVEh2cgo2Wlp1cHdudWM0em1laW4xK205OW0zcFgwd0tCZ1FDMDlHelVBVkJ1Y0lwYTlrWnJ5am1VQTZ6UTJzVllhM3pXCnp2Q3lPSmFCQ0JmdStES2J1UFk2ckZBSkpJSUZrUFZpam15ZERTTSsyTS9leTFxcUZWdmlES2RqNm5GM21wQUkKZ2ljUjVyTy9xQy9lTnNYWVNncFBkd0hwdWtsQThuOEpuWXpneEZ0bDI1R2p3anpPQ3hJZ3hWTzd0a2pxODY4cwpueS9ZakpCMFpRS0JnRjFpR1RldnFHSG9aeXJXazlDWTJkWXB1bzFSRTliQS9IN2YyeDRna2VHSkZGY0xtdnZTCkc1cFlhY3NQK08rb2hNdWdhNHJYVDg4TWJsZmVMVkhvL2JWQk1lYklrZGhndU5GS3BIMVFtRkJvWDZnV3BQK0MKdTJQUkhyZW1Ka3BWaC9UNVF1SVVKSzNrblBZQTl3d0xwc0hlWmRjZHl0endLb05KYWUzRVBkd3BBb0dCQUxLMwo0YkExcUJHYmR4cEZ0K0dIdGZ4WThOcko1elRmM0VsdjAwdkNGZXRVYWdici9WZk5ndXNKK25ZOUoybUY5TVVsCk0yS2RjaDBGRjdETW1UdjRvbW9NNDFvTVY0cXdEZWRKWEhzU3BwNzQ1bVlMSkxYczBtZ1Blc1V6NWx1TjVTaEQKQWJ1UVUxQ3pYaEVscXRZNUd1dGcyTVZCVUowMGxlTjFTVlVQNXkvOUFvR0FUZGZkby92VVBNQ3VxSU9HWlladApEVDNrMkxrdTJySnRveXRweXRaYkw4cUptOWRKdU5yRzYySVlTMXVvbkI1OHJ2SG5mUW5LLzRzRHlYTTNHRWhSCjBvTDArVU5GMlllcEdqNStCS0MxZC96a293RE1CYUdtOTB2ekFwWkk0eHdkNi9VVnA3Q0NWcGZpYXdUK0RTZFMKbDRNMXg0Y2JCSm8wdFM1RllGdkVUS0U9Ci0tLS0tRU5EIFBSSVZBVEUgS0VZLS0tLS0K
kind: Secret
metadata:
  creationTimestamp: "2020-04-14T15:55:32Z"
  labels:
    operated-by: openunison-operator
    tremolo_operator_created: "true"
  name: kubernetes-dashboard-certs
  namespace: kubernetes-dashboard-system
  resourceVersion: "30258"
  selfLink: /api/v1/namespaces/kubernetes-dashboard-system/secrets/kubernetes-dashboard-certs
  uid: 95cd58fc-7ab7-425a-8292-bf524a43b72e
type: Opaque

In the and I get this:

kubectl exec -ti openunison-orchestra-74d7697d85-rfzvl -n openunison  -- openssl s_client -connect 'kubernetes-dashboard.kubernetes-dashboard-system:443' --showcerts
CONNECTED(00000005)
depth=0 
verify error:num=18:self signed certificate
verify return:1
depth=0 
verify error:num=10:certificate has expired
notAfter=Jan  1 00:00:00 1 GMT
verify return:1
depth=0 
notAfter=Jan  1 00:00:00 1 GMT
verify return:1
---
Certificate chain
 0 s:
   i:
-----BEGIN CERTIFICATE-----
MIIBAzCBqqADAgECAhB2cR/vIPLwHsXFaLfj75LZMAoGCCqGSM49BAMCMAAwIhgP
MDAwMTAxMDEwMDAwMDBaGA8wMDAxMDEwMTAwMDAwMFowADBZMBMGByqGSM49AgEG
CCqGSM49AwEHA0IABCZ+NLx/krVaBjmtgyNhkY4YVXVk2LUcjbZy3bmsu6idQx/Q
/FzylgRkmF5Th8RFj8ILyHgyjuBCylww2N74glujAjAAMAoGCCqGSM49BAMCA0gA
MEUCID3mH6NCBOWEOQZm2CJ2ZpTyAgzK8CIsYarz+2E2JnOEAiEApC8GhsDRbqsm
/x9fFm+jO2kgtWbC371F1fEp9h2lRGA=
-----END CERTIFICATE-----
---
Server certificate
subject=

issuer=

---
No client certificate CA names sent
Peer signing digest: SHA256
Peer signature type: ECDSA
Server Temp Key: X25519, 253 bits
---
SSL handshake has read 651 bytes and written 443 bytes
Verification error: certificate has expired
---
New, TLSv1.2, Cipher is ECDHE-ECDSA-AES128-GCM-SHA256
Server public key is 256 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
    Protocol  : TLSv1.2
    Cipher    : ECDHE-ECDSA-AES128-GCM-SHA256
    Session-ID: B4C199F9C1EF64293DE5A0C76195C08B730078507693D03832BF88BDC3193946
    Session-ID-ctx: 
    Master-Key: 4A41E2651E03E00B95A1639687AC57A76251D5EB15175F101EA9ADB68CED8CE9FEA2DE5D0E0B5361A7984A8E20D14534
    PSK identity: None
    PSK identity hint: None
    SRP username: None
    TLS session ticket:
    0000 - c7 b2 84 56 08 0d 4e c1-85 1a 52 03 28 9b a6 09   ...V..N...R.(...
    0010 - e0 90 cf c8 df 9c 0c 98-66 46 8e ae cd 12 d9 ff   ........fF......
    0020 - 98 41 ee 63 1a 31 5a 74-12 a6 4f 36 05 a1 b9 3f   .A.c.1Zt..O6...?
    0030 - 86 83 d9 98 8e 6d fd 4c-cc 98 55 32 3e 4b 38 49   .....m.L..U2>K8I
    0040 - c0 b3 ab d5 fe bb 21 d0-9f 45 13 a3 0c bd b2 53   ......!..E.....S
    0050 - 79 be f4 95 ea ae 13 1f-38 a4 20 ed cc be 84 ca   y.......8. .....
    0060 - 6d 9f 59 76 82 1e 41 91-f4 e1 fc e7 f3 dc 2b 4e   m.Yv..A.......+N
    0070 - 58 d0 bb 0a 7c e7 35 21-                          X...|.5!

    Start Time: 1587052237
    Timeout   : 7200 (sec)
    Verify return code: 10 (certificate has expired)
    Extended master secret: no
---
mlbiam commented 4 years ago

how old is the dashboard pod? did it get restarted after openunison was deployed? (it should have been). It looks like the dashboard is still using an auto-generated cert. What do do the logs say in the dashboard? It should say something about whether or not it can find a cert

devopstales commented 4 years ago

I restartid now it is 9sec old:

2020/04/16 16:06:55 Using namespace: kubernetes-dashboard-system
2020/04/16 16:06:55 Using in-cluster config to connect to apiserver
2020/04/16 16:06:55 Starting overwatch
2020/04/16 16:06:55 Using secret token for csrf signing
2020/04/16 16:06:55 Initializing csrf token from kubernetes-dashboard-csrf secret
2020/04/16 16:06:55 Successful initial request to the apiserver, version: v1.16.1
2020/04/16 16:06:55 Generating JWE encryption key
2020/04/16 16:06:55 New synchronizer has been registered: kubernetes-dashboard-key-holder-kubernetes-dashboard-system. Starting
2020/04/16 16:06:55 Starting secret synchronizer for kubernetes-dashboard-key-holder in namespace kubernetes-dashboard-system
2020/04/16 16:06:55 Initializing JWE encryption key from synchronized object
2020/04/16 16:06:55 Creating in-cluster Sidecar client
2020/04/16 16:06:55 Auto-generating certificates
2020/04/16 16:06:56 Successfully created certificates
2020/04/16 16:06:56 Serving securely on HTTPS port: 8443
2020/04/16 16:06:56 Successful request to sidecar

Nothing changed there is no issuer in cert.

mlbiam commented 4 years ago

2020/04/16 16:06:56 Successfully created certificates

Did you make changes to the dashboard Deployment? it doesn't look like it sees the certificate secret. This line should say Certificate already exists. Returning

devopstales commented 4 years ago

I didn't change the Deployment just delete the pod.

mlbiam commented 4 years ago

can you share your Deployment for the dashboard?

devopstales commented 4 years ago

This is my deployment:

apiVersion: apps/v1
kind: Deployment
metadata:
  annotations:
    deployment.kubernetes.io/revision: "1"
    kubectl.kubernetes.io/last-applied-configuration: |
      {"apiVersion":"apps/v1","kind":"Deployment","metadata":{"annotations":{},"labels":{"k8s-app":"kubernetes-dashboard"},"name":"kubernetes-dashboard","namespace":"kubernetes-dashboard-system"},"spec":{"replicas":1,"revisionHistoryLimit":10,"selector":{"matchLabels":{"k8s-app":"kubernetes-dashboard"}},"template":{"metadata":{"labels":{"k8s-app":"kubernetes-dashboard"}},"spec":{"containers":[{"args":["--auto-generate-certificates","--namespace=kubernetes-dashboard-system"],"image":"kubernetesui/dashboard:v2.0.0-beta8","imagePullPolicy":"Always","livenessProbe":{"httpGet":{"path":"/","port":8443,"scheme":"HTTPS"},"initialDelaySeconds":30,"timeoutSeconds":30},"name":"kubernetes-dashboard","ports":[{"containerPort":8443,"protocol":"TCP"}],"securityContext":{"allowPrivilegeEscalation":false,"readOnlyRootFilesystem":true,"runAsGroup":2001,"runAsUser":1001},"volumeMounts":[{"mountPath":"/certs","name":"kubernetes-dashboard-certs"},{"mountPath":"/tmp","name":"tmp-volume"}]}],"nodeSelector":{"beta.kubernetes.io/os":"linux"},"serviceAccountName":"kubernetes-dashboard","tolerations":[{"effect":"NoSchedule","key":"node-role.kubernetes.io/master"}],"volumes":[{"name":"kubernetes-dashboard-certs","secret":{"secretName":"kubernetes-dashboard-certs"}},{"emptyDir":{},"name":"tmp-volume"}]}}}}
  creationTimestamp: "2020-04-16T15:48:18Z"
  generation: 1
  labels:
    k8s-app: kubernetes-dashboard
  name: kubernetes-dashboard
  namespace: kubernetes-dashboard-system
  resourceVersion: "55902"
  selfLink: /apis/apps/v1/namespaces/kubernetes-dashboard-system/deployments/kubernetes-dashboard
  uid: 15704a57-8865-4f9c-863a-707059c0e1e4
spec:
  progressDeadlineSeconds: 600
  replicas: 1
  revisionHistoryLimit: 10
  selector:
    matchLabels:
      k8s-app: kubernetes-dashboard
  strategy:
    rollingUpdate:
      maxSurge: 25%
      maxUnavailable: 25%
    type: RollingUpdate
  template:
    metadata:
      creationTimestamp: null
      labels:
        k8s-app: kubernetes-dashboard
    spec:
      containers:
      - args:
        - --auto-generate-certificates
        - --namespace=kubernetes-dashboard-system
        image: kubernetesui/dashboard:v2.0.0-beta8
        imagePullPolicy: Always
        livenessProbe:
          failureThreshold: 3
          httpGet:
            path: /
            port: 8443
            scheme: HTTPS
          initialDelaySeconds: 30
          periodSeconds: 10
          successThreshold: 1
          timeoutSeconds: 30
        name: kubernetes-dashboard
        ports:
        - containerPort: 8443
          protocol: TCP
        resources: {}
        securityContext:
          allowPrivilegeEscalation: false
          readOnlyRootFilesystem: true
          runAsGroup: 2001
          runAsUser: 1001
        terminationMessagePath: /dev/termination-log
        terminationMessagePolicy: File
        volumeMounts:
        - mountPath: /certs
          name: kubernetes-dashboard-certs
        - mountPath: /tmp
          name: tmp-volume
      dnsPolicy: ClusterFirst
      nodeSelector:
        beta.kubernetes.io/os: linux
      restartPolicy: Always
      schedulerName: default-scheduler
      securityContext: {}
      serviceAccount: kubernetes-dashboard
      serviceAccountName: kubernetes-dashboard
      terminationGracePeriodSeconds: 30
      tolerations:
      - effect: NoSchedule
        key: node-role.kubernetes.io/master
      volumes:
      - name: kubernetes-dashboard-certs
        secret:
          defaultMode: 420
          secretName: kubernetes-dashboard-certs
      - emptyDir: {}
        name: tmp-volume
status:
  availableReplicas: 1
  conditions:
  - lastTransitionTime: "2020-04-16T15:48:18Z"
    lastUpdateTime: "2020-04-16T15:48:29Z"
    message: ReplicaSet "kubernetes-dashboard-69fb5c48b" has successfully progressed.
    reason: NewReplicaSetAvailable
    status: "True"
    type: Progressing
  - lastTransitionTime: "2020-04-16T16:37:25Z"
    lastUpdateTime: "2020-04-16T16:37:25Z"
    message: Deployment has minimum availability.
    reason: MinimumReplicasAvailable
    status: "True"
    type: Available
  observedGeneration: 1
  readyReplicas: 1
  replicas: 1
  updatedReplicas: 1
mlbiam commented 4 years ago

that all looks right. so why isn't the container pulling in the cert? going to see if we can get some help from the dashboard team...

mlbiam commented 4 years ago

can't reproduce this issue. Can you try updating the arguments to look like:

        - --namespace=kubernetes-dashboard-system
        - --tls-cert-file=/dashboard.crt
        - --tls-key-file=/dashboard.key

?

devopstales commented 4 years ago

I recreated the custer now I get hist in the certificate:

kubectl exec -ti openunison-orchestra-74d7697d85-x6dhr -n openunison  -- openssl s_client -connect 'kubernetes-dashboard.kubernetes-dashboard-system:443' --showcerts
CONNECTED(00000005)
depth=0 C = US, ST = Virginia, L = Alexandria, O = Tremolo Security, OU = k8s, CN = kubernetes-dashboard.kubernetes-dashboard-system.svc.cluster.local
verify error:num=18:self signed certificate
verify return:1
depth=0 C = US, ST = Virginia, L = Alexandria, O = Tremolo Security, OU = k8s, CN = kubernetes-dashboard.kubernetes-dashboard-system.svc.cluster.local
verify error:num=26:unsupported certificate purpose
verify return:1
depth=0 C = US, ST = Virginia, L = Alexandria, O = Tremolo Security, OU = k8s, CN = kubernetes-dashboard.kubernetes-dashboard-system.svc.cluster.local
verify return:1
---
Certificate chain
 0 s:C = US, ST = Virginia, L = Alexandria, O = Tremolo Security, OU = k8s, CN = kubernetes-dashboard.kubernetes-dashboard-system.svc.cluster.local
   i:C = US, ST = Virginia, L = Alexandria, O = Tremolo Security, OU = k8s, CN = kubernetes-dashboard.kubernetes-dashboard-system.svc.cluster.local
-----BEGIN CERTIFICATE-----
MIIEYDCCA0igAwIBAgIGAXGWrPAaMA0GCSqGSIb3DQEBCwUAMIGrMQswCQYDVQQG
EwJVUzERMA8GA1UECBMIVmlyZ2luaWExEzARBgNVBAcTCkFsZXhhbmRyaWExGTAX
BgNVBAoTEFRyZW1vbG8gU2VjdXJpdHkxDDAKBgNVBAsTA2s4czFLMEkGA1UEAxNC
a3ViZXJuZXRlcy1kYXNoYm9hcmQua3ViZXJuZXRlcy1kYXNoYm9hcmQtc3lzdGVt
LnN2Yy5jbHVzdGVyLmxvY2FsMB4XDTIwMDQyMDA4MjA0OFoXDTIxMDQyMDA4MjA0
OFowgasxCzAJBgNVBAYTAlVTMREwDwYDVQQIEwhWaXJnaW5pYTETMBEGA1UEBxMK
QWxleGFuZHJpYTEZMBcGA1UEChMQVHJlbW9sbyBTZWN1cml0eTEMMAoGA1UECxMD
azhzMUswSQYDVQQDE0JrdWJlcm5ldGVzLWRhc2hib2FyZC5rdWJlcm5ldGVzLWRh
c2hib2FyZC1zeXN0ZW0uc3ZjLmNsdXN0ZXIubG9jYWwwggEiMA0GCSqGSIb3DQEB
AQUAA4IBDwAwggEKAoIBAQCmvwRVnYQaWYhs1CjDw0YRLAeyCWuNecT3ZR66B5Tl
RnFFCpwwDfoZfO+bH1Pdeqg35Kkh49/wiEdn3ExUcR/D0zHaPrA73ovLg5oavTNp
5BmZf23W63EitZ0TP+Svq/Qtf9DW9dVVKuy0gZ8P4KZOihuMfbGM1lJWgq+I/m3t
rQVsVON4Fy9UsXd27tZrLymXMuv9nYmLZZrQQXVZ42bIRBsn4vW4/R/Td6roCa6L
hQhRIgEy+rZY3qwvpP+32v3Gp9KdIXZDHWM2bKT30IdQD3AOivWNvwPP0mfkOKhQ
VfdgZgqsWiXG7kQOXirFKP/Q5bAjKASNlJEh/9Eghc+XAgMBAAGjgYcwgYQwDwYD
VR0TAQH/BAUwAwEB/zAOBgNVHQ8BAf8EBAMCAgQwEgYDVR0lAQH/BAgwBgYEVR0l
ADBNBgNVHREERjBEgkJrdWJlcm5ldGVzLWRhc2hib2FyZC5rdWJlcm5ldGVzLWRh
c2hib2FyZC1zeXN0ZW0uc3ZjLmNsdXN0ZXIubG9jYWwwDQYJKoZIhvcNAQELBQAD
ggEBAAnVL5kLcozpPVag/uZiEP7ncj7XV62pwp52tN/JJup4rdobOk+mh+iyXRsz
kdvHFtR+wco35Y9PaupYsw18Kfgeg7fLbjGCPY/FdoSVQGTTMR6o+OnMkJaVatTK
vh/HpkmVK++Z+q9sT8EKBCrUMSHpkthfoQhJcapA0hFL3zRIDPShG5SE7cdeA1Tq
IXl3myqLAQTyVRh8th7vI0dSw0DlZxO5ZKckrSYqyD9myIE7KzILFyTL42THzX3l
O3rGPpOkIz9KDD6cObl8QLOxOkIZDuzboFhqKVu2K7mM7lICYsmtkD1Cc5EVeKos
kslYDnqdcaJgbkYNOwwJu8Vwfvk=
-----END CERTIFICATE-----
---
Server certificate
subject=C = US, ST = Virginia, L = Alexandria, O = Tremolo Security, OU = k8s, CN = kubernetes-dashboard.kubernetes-dashboard-system.svc.cluster.local

issuer=C = US, ST = Virginia, L = Alexandria, O = Tremolo Security, OU = k8s, CN = kubernetes-dashboard.kubernetes-dashboard-system.svc.cluster.local

---
No client certificate CA names sent
Peer signing digest: SHA256
Peer signature type: RSA
Server Temp Key: X25519, 253 bits
---
SSL handshake has read 1697 bytes and written 443 bytes
Verification error: unsupported certificate purpose
---
New, TLSv1.2, Cipher is ECDHE-RSA-AES128-GCM-SHA256
Server public key is 2048 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
    Protocol  : TLSv1.2
    Cipher    : ECDHE-RSA-AES128-GCM-SHA256
    Session-ID: 65A465DD660504E2C190562EE3AEBA0058E164A95E139D5651E9E3CFDCD14DC1
    Session-ID-ctx: 
    Master-Key: 6730AB341B052EC04F32E2107D0C1B9389C1B22C8F5D3E685ED2B23083200D5D43046E87A8E91D436615306B8980D161
    PSK identity: None
    PSK identity hint: None
    SRP username: None
    TLS session ticket:
    0000 - ed 3d 32 c8 59 32 4b 11-a3 f1 6d c4 6f 25 c5 36   .=2.Y2K...m.o%.6
    0010 - 4a dd 66 e4 41 74 76 95-32 64 b5 08 13 dc 1e dd   J.f.Atv.2d......
    0020 - 49 48 de a0 bb ff 46 14-58 a0 57 90 8c c6 bd 49   IH....F.X.W....I
    0030 - a1 96 9e c9 79 ce 9c 7c-48 cf 2d 1e 7d af 2b cc   ....y..|H.-.}.+.
    0040 - ce 65 29 45 df 18 75 21-82 50 0f 81 ca 3d 43 32   .e)E..u!.P...=C2
    0050 - c0 39 70 e5 e7 a9 06 f3-94 1f df 6c 4f 7e f5 4a   .9p........lO~.J
    0060 - 5f 4e ca 78 7d 70 5f 7b-fc cc f4 07 c1 1e 83 dc   _N.x}p_{........
    0070 - db 78 0d 8d 53 38 d2 b5-                          .x..S8..

    Start Time: 1587373078
    Timeout   : 7200 (sec)
    Verify return code: 26 (unsupported certificate purpose)
    Extended master secret: no
---

The I deleted the dashboard pod:

2020/04/20 09:01:40 Starting overwatch
2020/04/20 09:01:40 Using namespace: kubernetes-dashboard-system
2020/04/20 09:01:40 Using in-cluster config to connect to apiserver
2020/04/20 09:01:40 Using secret token for csrf signing
2020/04/20 09:01:40 Initializing csrf token from kubernetes-dashboard-csrf secret
2020/04/20 09:01:40 Successful initial request to the apiserver, version: v1.16.1
2020/04/20 09:01:40 Generating JWE encryption key
2020/04/20 09:01:40 New synchronizer has been registered: kubernetes-dashboard-key-holder-kubernetes-dashboard-system. Starting
2020/04/20 09:01:40 Starting secret synchronizer for kubernetes-dashboard-key-holder in namespace kubernetes-dashboard-system
2020/04/20 09:01:40 Initializing JWE encryption key from synchronized object
2020/04/20 09:01:40 Creating in-cluster Sidecar client
2020/04/20 09:01:40 Auto-generating certificates
2020/04/20 09:01:40 Certificates already exist. Returning.
2020/04/20 09:01:40 Serving securely on HTTPS port: 8443
2020/04/20 09:01:40 Successful request to sidecar

But same error, I tried to change the arguments in the dashboard pod too as you sad:

[2020-04-20 09:17:16,601][XNIO-1 task-3] INFO  AccessLog - [AzSuccess] - k8sIdp - https://127.0.0.1:8443/auth/idp/k8sIdp/.well-known/openid-configuration - uid=Anonymous,o=Tremolo - NONE [127.0.0.1] - [fc4e28fffd132159649087d0b620fc294426fa14e]
[2020-04-20 09:17:17,807][XNIO-1 task-4] INFO  AccessLog - [AzSuccess] - k8sIdp - https://openunison.k8s.intra/auth/idp/k8sIdp/completefed - uid=tester,ou=users,ou=shadow,o=Tremolo -  [10.244.1.1] - [f670a977e8f35a761081df33e6833251fbd95d65e]
[2020-04-20 09:17:17,846][XNIO-1 task-11] INFO  AccessLog - [Error] - k8s - https://k8sdb.k8s.intra/auth/oidc - uid=Anonymous,o=Tremolo - NONE [10.244.1.1] - [fe67eb7a67e5b29e0ab866a2643448a9a9be2c629]
[2020-04-20 09:17:17,847][XNIO-1 task-11] ERROR ConfigSys - Could not process request
javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
    at sun.security.ssl.Alerts.getSSLException(Alerts.java:192) ~[?:1.8.0_222]
    at sun.security.ssl.SSLSocketImpl.fatal(SSLSocketImpl.java:1946) ~[?:1.8.0_222]
    at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:316) ~[?:1.8.0_222]
    at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:310) ~[?:1.8.0_222]
    at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1639) ~[?:1.8.0_222]
    at sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:223) ~[?:1.8.0_222]
    at sun.security.ssl.Handshaker.processLoop(Handshaker.java:1037) ~[?:1.8.0_222]
    at sun.security.ssl.Handshaker.process_record(Handshaker.java:965) ~[?:1.8.0_222]
    at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:1064) ~[?:1.8.0_222]
    at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1367) ~[?:1.8.0_222]
    at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1395) ~[?:1.8.0_222]
    at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1379) ~[?:1.8.0_222]
    at org.apache.http.conn.ssl.SSLConnectionSocketFactory.createLayeredSocket(SSLConnectionSocketFactory.java:436) ~[httpclient-4.5.9.jar:4.5.9]
    at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:384) ~[httpclient-4.5.9.jar:4.5.9]
    at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ~[httpclient-4.5.9.jar:4.5.9]
    at org.apache.http.impl.conn.BasicHttpClientConnectionManager.connect(BasicHttpClientConnectionManager.java:313) ~[httpclient-4.5.9.jar:4.5.9]
    at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) ~[httpclient-4.5.9.jar:4.5.9]
    at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) ~[httpclient-4.5.9.jar:4.5.9]
    at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) ~[httpclient-4.5.9.jar:4.5.9]
    at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) ~[httpclient-4.5.9.jar:4.5.9]
    at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) ~[httpclient-4.5.9.jar:4.5.9]
    at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) ~[httpclient-4.5.9.jar:4.5.9]
    at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) ~[httpclient-4.5.9.jar:4.5.9]
    at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:108) ~[httpclient-4.5.9.jar:4.5.9]
    at com.tremolosecurity.unison.proxy.auth.openidconnect.OpenIDConnectAuthMech.doGet(OpenIDConnectAuthMech.java:206) ~[unison-auth-openidconnect-1.0.17.jar:?]
    at com.tremolosecurity.proxy.auth.AuthMgrSys.doAuthMgr(AuthMgrSys.java:191) ~[unison-server-core-1.0.17.jar:?]
    at com.tremolosecurity.embedd.NextEmbSys.nextSys(NextEmbSys.java:126) ~[unison-server-core-1.0.17.jar:?]
    at com.tremolosecurity.proxy.auth.AzSys.doAz(AzSys.java:89) ~[unison-sdk-1.0.17.jar:?]
    at com.tremolosecurity.embedd.NextEmbSys.nextSys(NextEmbSys.java:111) ~[unison-server-core-1.0.17.jar:?]
    at com.tremolosecurity.proxy.auth.AuthSys.doAuth(AuthSys.java:118) ~[unison-server-core-1.0.17.jar:?]
    at com.tremolosecurity.embedd.NextEmbSys.nextSys(NextEmbSys.java:105) ~[unison-server-core-1.0.17.jar:?]
    at com.tremolosecurity.proxy.ConfigSys.doConfig(ConfigSys.java:293) [unison-server-core-1.0.17.jar:?]
    at com.tremolosecurity.embedd.NextEmbSys.nextSys(NextEmbSys.java:93) [unison-server-core-1.0.17.jar:?]
    at com.tremolosecurity.filter.UnisonServletFilter.doFilter(UnisonServletFilter.java:290) [unison-server-core-1.0.17.jar:?]
    at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61) [undertow-servlet-2.0.22.Final.jar:2.0.22.Final]
    at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131) [undertow-servlet-2.0.22.Final.jar:2.0.22.Final]
    at io.undertow.servlet.handlers.FilterHandler.handleRequest(FilterHandler.java:84) [undertow-servlet-2.0.22.Final.jar:2.0.22.Final]
    at io.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:62) [undertow-servlet-2.0.22.Final.jar:2.0.22.Final]
    at io.undertow.servlet.handlers.ServletChain$1.handleRequest(ServletChain.java:68) [undertow-servlet-2.0.22.Final.jar:2.0.22.Final]
    at io.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36) [undertow-servlet-2.0.22.Final.jar:2.0.22.Final]
    at io.undertow.servlet.handlers.security.SSLInformationAssociationHandler.handleRequest(SSLInformationAssociationHandler.java:132) [undertow-servlet-2.0.22.Final.jar:2.0.22.Final]
    at io.undertow.servlet.handlers.security.ServletAuthenticationCallHandler.handleRequest(ServletAuthenticationCallHandler.java:57) [undertow-servlet-2.0.22.Final.jar:2.0.22.Final]
    at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) [undertow-core-2.0.22.Final.jar:2.0.22.Final]
    at io.undertow.security.handlers.AbstractConfidentialityHandler.handleRequest(AbstractConfidentialityHandler.java:46) [undertow-core-2.0.22.Final.jar:2.0.22.Final]
    at io.undertow.servlet.handlers.security.ServletConfidentialityConstraintHandler.handleRequest(ServletConfidentialityConstraintHandler.java:64) [undertow-servlet-2.0.22.Final.jar:2.0.22.Final]
    at io.undertow.security.handlers.AuthenticationMechanismsHandler.handleRequest(AuthenticationMechanismsHandler.java:60) [undertow-core-2.0.22.Final.jar:2.0.22.Final]
    at io.undertow.servlet.handlers.security.CachedAuthenticatedSessionHandler.handleRequest(CachedAuthenticatedSessionHandler.java:77) [undertow-servlet-2.0.22.Final.jar:2.0.22.Final]
    at io.undertow.security.handlers.AbstractSecurityContextAssociationHandler.handleRequest(AbstractSecurityContextAssociationHandler.java:43) [undertow-core-2.0.22.Final.jar:2.0.22.Final]
    at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) [undertow-core-2.0.22.Final.jar:2.0.22.Final]
    at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) [undertow-core-2.0.22.Final.jar:2.0.22.Final]
    at io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:292) [undertow-servlet-2.0.22.Final.jar:2.0.22.Final]
    at io.undertow.servlet.handlers.ServletInitialHandler.access$100(ServletInitialHandler.java:81) [undertow-servlet-2.0.22.Final.jar:2.0.22.Final]
    at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:138) [undertow-servlet-2.0.22.Final.jar:2.0.22.Final]
    at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:135) [undertow-servlet-2.0.22.Final.jar:2.0.22.Final]
    at io.undertow.servlet.core.ServletRequestContextThreadSetupAction$1.call(ServletRequestContextThreadSetupAction.java:48) [undertow-servlet-2.0.22.Final.jar:2.0.22.Final]
    at io.undertow.servlet.core.ContextClassLoaderSetupAction$1.call(ContextClassLoaderSetupAction.java:43) [undertow-servlet-2.0.22.Final.jar:2.0.22.Final]
    at io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:272) [undertow-servlet-2.0.22.Final.jar:2.0.22.Final]
    at io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:81) [undertow-servlet-2.0.22.Final.jar:2.0.22.Final]
    at io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:104) [undertow-servlet-2.0.22.Final.jar:2.0.22.Final]
    at io.undertow.server.Connectors.executeRootHandler(Connectors.java:376) [undertow-core-2.0.22.Final.jar:2.0.22.Final]
    at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:830) [undertow-core-2.0.22.Final.jar:2.0.22.Final]
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_222]
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_222]
    at java.lang.Thread.run(Thread.java:748) [?:1.8.0_222]
Caused by: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
    at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:397) ~[?:1.8.0_222]
    at sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:302) ~[?:1.8.0_222]
    at sun.security.validator.Validator.validate(Validator.java:262) ~[?:1.8.0_222]
    at sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:330) ~[?:1.8.0_222]
    at sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:237) ~[?:1.8.0_222]
    at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:132) ~[?:1.8.0_222]
    at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1621) ~[?:1.8.0_222]
    ... 59 more
Caused by: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
    at sun.security.provider.certpath.SunCertPathBuilder.build(SunCertPathBuilder.java:141) ~[?:1.8.0_222]
    at sun.security.provider.certpath.SunCertPathBuilder.engineBuild(SunCertPathBuilder.java:126) ~[?:1.8.0_222]
    at java.security.cert.CertPathBuilder.build(CertPathBuilder.java:280) ~[?:1.8.0_222]
    at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:392) ~[?:1.8.0_222]
    at sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:302) ~[?:1.8.0_222]
    at sun.security.validator.Validator.validate(Validator.java:262) ~[?:1.8.0_222]
    at sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:330) ~[?:1.8.0_222]
    at sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:237) ~[?:1.8.0_222]
    at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:132) ~[?:1.8.0_222]
    at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1621) ~[?:1.8.0_222]
    ... 59 more
[2020-04-20 09:17:19,557][XNIO-1 task-1] INFO  AccessLog - [AzSuccess] - CheckAlive - https://127.0.0.1:8443/check_alive - uid=Anonymous,o=Tremolo - 
         [127.0.0.1] - [fef3b41e8e96d17b1fdd7f75a24f34be53aacdbb7]
[2020-04-20 09:17:21,888][XNIO-1 task-7] INFO  AccessLog - [AzSuccess] - ScaleCheckSession - https://openunison.k8s.intra/scale/sessioncheck - uid=Anonymous,o=Tremolo - 
         [10.244.1.1] - [f11809fb1d94f9c3130a22cfce282f40467057bcf]
[2020-04-20 09:17:26,639][XNIO-1 task-16] INFO  AccessLog - [AzSuccess] - k8sIdp - https://127.0.0.1:8443/auth/idp/k8sIdp/.well-known/openid-configuration - uid=Anonymous,o=Tremolo - NONE [127.0.0.1] - [f29f848746c71169288c3cb319bc6e75fc61899ee]
mlbiam commented 4 years ago

this is different

at com.tremolosecurity.unison.proxy.auth.openidconnect.OpenIDConnectAuthMech.doGet(OpenIDConnectAuthMech.java:206) ~[unison-auth-openidconnect-1.0.17.jar:?]

This is openunison not being able to varify the certificate for openunison.k8s.intra. Since their on different domains and could be different servers OpenUnison sso's with its self across domains. To make this work, OpenUnison must trust its own eternal facing certificate. I'm guessing you have an ingress certificate that you created?

devopstales commented 4 years ago

Yes I use a self signed rootCA and certs. This rootCA is trusted in my browser. So somehow I need to tell OpenUnison to trust this CA right? Is ther a config for this?

mlbiam commented 4 years ago

Yes. take a look at https://github.com/TremoloSecurity/OpenUnison/wiki/troubleshooting#how-do-i-change-openunisons-certificates. Once the root ca certificate is added to trusted_certs you should get past this issue.

devopstales commented 4 years ago

OK. Now I can see the kubernetes dashboard but I got Unauthorized for everything on it:

2020/04/22 17:23:39 [2020-04-22T17:23:39Z] Incoming HTTP/1.1 GET /api/v1/settings/global/cani request from 10.244.2.83:40876: 
2020/04/22 17:23:39 [2020-04-22T17:23:39Z] Incoming HTTP/1.1 GET /api/v1/settings/pinner request from 10.244.2.83:40880: 
2020/04/22 17:23:39 [2020-04-22T17:23:39Z] Incoming HTTP/1.1 GET /api/v1/plugin/config request from 10.244.2.83:40882: 
2020/04/22 17:23:39 Getting application global configuration
2020/04/22 17:23:39 Application configuration {"serverTime":1587576219905}
2020/04/22 17:23:39 Unauthorized
2020/04/22 17:23:39 [2020-04-22T17:23:39Z] Outcoming response to 10.244.2.83:40876 with 200 status code
2020/04/22 17:23:39 Non-critical error occurred during resource retrieval: Unauthorized
2020/04/22 17:23:39 [2020-04-22T17:23:39Z] Outcoming response to 10.244.2.83:40882 with 200 status code
2020/04/22 17:23:39 Cannot find settings config map: Unauthorized
2020/04/22 17:23:39 Cannot restore settings config map: Unauthorized
2020/04/22 17:23:39 [2020-04-22T17:23:39Z] Outcoming response to 10.244.2.83:40880 with 200 status code
2020/04/22 17:23:40 [2020-04-22T17:23:40Z] Incoming HTTP/1.1 GET /api/v1/settings/global/cani request from 10.244.2.83:40902: 
2020/04/22 17:23:40 Unauthorized
2020/04/22 17:23:40 [2020-04-22T17:23:40Z] Outcoming response to 10.244.2.83:40902 with 200 status code
2020/04/22 17:23:40 [2020-04-22T17:23:40Z] Incoming HTTP/1.1 GET /api/v1/login/status request from 10.244.2.83:40914: 
2020/04/22 17:23:40 [2020-04-22T17:23:40Z] Outcoming response to 10.244.2.83:40914 with 200 status code
2020/04/22 17:23:40 [2020-04-22T17:23:40Z] Incoming HTTP/1.1 GET /api/v1/login/status request from 10.244.2.83:40924: 
2020/04/22 17:23:40 [2020-04-22T17:23:40Z] Outcoming response to 10.244.2.83:40924 with 200 status code
2020/04/22 17:23:40 [2020-04-22T17:23:40Z] Incoming HTTP/1.1 GET /api/v1/systembanner request from 10.244.2.83:40920: 
2020/04/22 17:23:40 [2020-04-22T17:23:40Z] Outcoming response to 10.244.2.83:40920 with 200 status code
2020/04/22 17:23:40 [2020-04-22T17:23:40Z] Incoming HTTP/1.1 GET /api/v1/login/status request from 10.244.2.83:40932: 
2020/04/22 17:23:40 [2020-04-22T17:23:40Z] Outcoming response to 10.244.2.83:40932 with 200 status code
2020/04/22 17:23:40 [2020-04-22T17:23:40Z] Incoming HTTP/1.1 GET /api/v1/namespace request from 10.244.2.83:40946: 
2020/04/22 17:23:41 Getting list of namespaces
2020/04/22 17:23:41 Non-critical error occurred during resource retrieval: Unauthorized
2020/04/22 17:23:41 [2020-04-22T17:23:41Z] Outcoming response to 10.244.2.83:40946 with 200 status code
2020/04/22 17:23:41 [2020-04-22T17:23:41Z] Incoming HTTP/1.1 GET /api/v1/cronjob/default?itemsPerPage=10&page=1&sortBy=d%!C(MISSING)creationTimestamp request from 10.244.2.83:40950: 
2020/04/22 17:23:41 [2020-04-22T17:23:41Z] Incoming HTTP/1.1 GET /api/v1/daemonset/default?itemsPerPage=10&page=1&sortBy=d%!C(MISSING)creationTimestamp request from 10.244.2.83:40948: 
2020/04/22 17:23:41 Getting list of all cron jobs in the cluster
2020/04/22 17:23:41 Non-critical error occurred during resource retrieval: Unauthorized
2020/04/22 17:23:41 Non-critical error occurred during resource retrieval: Unauthorized
2020/04/22 17:23:41 Non-critical error occurred during resource retrieval: Unauthorized
2020/04/22 17:23:41 [2020-04-22T17:23:41Z] Outcoming response to 10.244.2.83:40948 with 200 status code
2020/04/22 17:23:41 Non-critical error occurred during resource retrieval: Unauthorized
2020/04/22 17:23:41 [2020-04-22T17:23:41Z] Outcoming response to 10.244.2.83:40950 with 200 status code
2020/04/22 17:23:41 [2020-04-22T17:23:41Z] Incoming HTTP/1.1 GET /api/v1/statefulset/default?itemsPerPage=10&page=1&sortBy=d%!C(MISSING)creationTimestamp request from 10.244.2.83:40952: 
2020/04/22 17:23:41 Getting list of all pet sets in the cluster
2020/04/22 17:23:41 Non-critical error occurred during resource retrieval: Unauthorized
2020/04/22 17:23:41 Non-critical error occurred during resource retrieval: Unauthorized
2020/04/22 17:23:41 Non-critical error occurred during resource retrieval: Unauthorized
2020/04/22 17:23:41 [2020-04-22T17:23:41Z] Outcoming response to 10.244.2.83:40952 with 200 status code
2020/04/22 17:23:41 [2020-04-22T17:23:41Z] Incoming HTTP/1.1 GET /api/v1/secret/default?itemsPerPage=10&page=1&sortBy=d%!C(MISSING)creationTimestamp request from 10.244.2.83:40958: 
2020/04/22 17:23:41 Getting list of secrets in &{[default]} namespace
2020/04/22 17:23:41 Non-critical error occurred during resource retrieval: Unauthorized
2020/04/22 17:23:41 [2020-04-22T17:23:41Z] Outcoming response to 10.244.2.83:40958 with 200 status code
2020/04/22 17:23:41 [2020-04-22T17:23:41Z] Incoming HTTP/1.1 GET /api/v1/persistentvolumeclaim/default?itemsPerPage=10&page=1&sortBy=d%!C(MISSING)creationTimestamp request from 10.244.2.83:40956: 
2020/04/22 17:23:41 [2020-04-22T17:23:41Z] Incoming HTTP/1.1 GET /api/v1/configmap/default?itemsPerPage=10&page=1&sortBy=d%!C(MISSING)creationTimestamp request from 10.244.2.83:40954: 
2020/04/22 17:23:41 Getting list persistent volumes claims
2020/04/22 17:23:41 Getting list config maps in the namespace default
2020/04/22 17:23:41 Non-critical error occurred during resource retrieval: Unauthorized
2020/04/22 17:23:41 [2020-04-22T17:23:41Z] Outcoming response to 10.244.2.83:40956 with 200 status code
2020/04/22 17:23:41 Non-critical error occurred during resource retrieval: Unauthorized
2020/04/22 17:23:41 [2020-04-22T17:23:41Z] Outcoming response to 10.244.2.83:40954 with 200 status code
2020/04/22 17:23:41 [2020-04-22T17:23:41Z] Incoming HTTP/1.1 GET /api/v1/deployment/default?itemsPerPage=10&page=1&sortBy=d%!C(MISSING)creationTimestamp request from 10.244.2.83:40966: 
2020/04/22 17:23:41 [2020-04-22T17:23:41Z] Incoming HTTP/1.1 GET /api/v1/pod/default?itemsPerPage=10&page=1&sortBy=d%!C(MISSING)creationTimestamp request from 10.244.2.83:40962: 
mlbiam commented 4 years ago

in the error on the dashboard, does it just say "unauthorized" or does it say something like "user https://yourhost#youruser doesn't have access to ..."?

mlbiam commented 4 years ago

if the error just says "Unauthorized", no additional information then the issue is that the API server isn't integrated into OpenUnison (https://github.com/OpenUnison/openunison-k8s-activedirectory#complete-sso-integration-with-kubernetes). If you're in a multi-api server environment you need to make sure that the integration is done on all of your api servers.

if the error says "User https://yourhost#user doesn't have access to..." is an RBAC question issue.