OpenUnison / openunison-k8s

Access portal for Kubernetes
Apache License 2.0
105 stars 5 forks source link

SAML2 deployment using ouctl," certificate signed by unknown authority " error at login portal stage #70

Closed eyupdzhanY1 closed 1 year ago

eyupdzhanY1 commented 1 year ago

I tested with ouctl and helm

helm upgrade orchestra-login-portal tremolo/orchestra-login-portal -f values.yaml -n openunison -i
Release "orchestra-login-portal" does not exist. Installing it now.
Error: Internal error occurred: failed calling webhook "applications-openunison.tremolo.io": failed to call webhook: Post "https://openunison-orchestra.openunison.svc:443/k8s/webhooks/v1/applications?timeout=5s": x509: certificate signed by unknown authority
mlbiam commented 1 year ago

Probably still an issue in the operator. The operator is responsible for generating certs for openunison and then patching the webhook yaml. My guess is the operator is failing before it hits that point. What you can do is delete the the openunson-operator pod, rerun ouctl, then capture the entire log from the operator (I know there's a ton of output that can be difficult to decipher). Post that output and it should give us the answer.

eyupdzhanY1 commented 1 year ago

Is there an option for ouctl to state ingress not to use tls secret of its own ?

    ingress:
      - ip: 11.11.222.2111
spec:
  tls:
    - hosts:
        - k8sou.com
        - k8sdb.com
  rules:
    - host:

So ingress nginx handles the tls connection

mlbiam commented 1 year ago

Is there an option for ouctl to state ingress not to use tls secret of its own ?

Yes, in your values.yaml set network.createIngressCertificate to false - Here are the TLS customization docs - https://openunison.github.io/knowledgebase/certificates/

eyupdzhanY1 commented 1 year ago

Log for operator

Using version 'openunison.tremolo.io/v6'
https://10.0.0.1:443/apis/openunison.tremolo.io/v6/namespaces/openunison/openunisons?watch=true&timeoutSeconds=30&allowWatchBookmarks=true&resourceVersion=61034055
Warning: Nashorn engine is planned to be removed from a future JDK release
Loading Script : '/usr/local/openunison/js/globals.js'
Loading Script : '/usr/local/openunison/js/deploy-upstream-k8s.js'
Loading Script : '/usr/local/openunison/js/deploy-objs.js'
Loading Script : '/usr/local/openunison/js/operator.js'
Loading Script : '/usr/local/openunison/js/deploy-openshift.js'
Loading Script : '/usr/local/openunison/js/helpers.js'
Invoking javascript
in js : {"type":"MODIFIED","object":{"metadata":{"generation":1,"uid":"fc89707c-f553-48ff-8cf1-9835ee610cbb","managedFields":[{"apiVersion":"openunison.tremolo.io\/v6","fieldsV1":{"f:status":{"f:conditions":{"f:lastTransitionTime":{},"f:type":{},"f:status":{},".":{}},"f:digest":{},"f:idpCertificateFingerprints":{"f:http:\/\/www.okta.com\/":{},".":{}},".":{}}},"manager":"Apache-HttpClient","subresource":"status","time":"2023-03-20T20:43:52Z","operation":"Update","fieldsType":"FieldsV1"},{"apiVersion":"openunison.tremolo.io\/v6","fieldsV1":{"f:metadata":{"f:annotations":{"f:meta.helm.sh\/release-namespace":{},"f:meta.helm.sh\/release-name":{},"f:argocd.argoproj.io\/sync-wave":{},"f:helm-update":{},".":{}},"f:labels":{"f:app.kubernetes.io\/instance":{},"f:app.kubernetes.io\/name":{},"f:app.kubernetes.io\/component":{},"f:app.kubernetes.io\/part-of":{},".":{},"f:app.kubernetes.io\/managed-by":{}}},"f:spec":{"f:key_store":{"f:static_keys":{},"f:update_controller":{"f:days_to_expire":{},"f:schedule":{},"f:image":{},".":{}},"f:trusted_certificates":{},".":{},"f:key_pairs":{"f:create_keypair_template":{},"f:keys":{},".":{}}},"f:source_secret":{},"f:deployment_data":{"f:node_selectors":{},"f:tokenrequest_api":{"f:audience":{},"f:enabled":{},"f:expirationSeconds":{},".":{}},"f:readiness_probe_command":{},"f:pull_secret":{},"f:liveness_probe_command":{},".":{}},"f:image":{},"f:dest_secret":{},".":{},"f:replicas":{},"f:hosts":{},"f:enable_activemq":{},"f:non_secret_data":{},"f:activemq_image":{},"f:openunison_network_configuration":{"f:activemq_dir":{},"f:quartz_dir":{},"f:secure_key_alias":{},"f:path_to_env_file":{},"f:secure_port":{},"f:ciphers":{},".":{},"f:allowed_client_names":{},"f:open_port":{},"f:path_to_deployment":{},"f:client_auth":{},"f:open_external_port":{},"f:force_to_secure":{},"f:secure_external_port":{}},"f:secret_data":{},"f:myvd_configmap":{},"f:saml_remote_idp":{}}},"manager":"ouctl","time":"2023-03-20T20:50:28Z","operation":"Update","fieldsType":"FieldsV1"}],"resourceVersion":"61036268","creationTimestamp":"2023-03-20T20:43:44Z","name":"orchestra","namespace":"openunison","annotations":{"helm-update":"Mar 20 22:50:03 2023 EET","meta.helm.sh\/release-name":"orchestra","meta.helm.sh\/release-namespace":"openunison","argocd.argoproj.io\/sync-wave":"20"},"labels":{"app.kubernetes.io\/managed-by":"Helm","app.kubernetes.io\/name":"openunison","app.kubernetes.io\/part-of":"openunison","app.kubernetes.io\/instance":"openunison-orchestra","app.kubernetes.io\/component":"openunison"}},"apiVersion":"openunison.tremolo.io\/v6","kind":"OpenUnison","spec":{"image":"docker.io\/tremolosecurity\/openunison-k8s","source_secret":"orchestra-secrets-source","openunison_network_configuration":{"client_auth":"none","secure_external_port":443,"activemq_dir":"\/tmp\/amq","force_to_secure":true,"quartz_dir":"\/tmp\/quartz","allowed_client_names":[],"secure_port":8443,"secure_key_alias":"unison-tls","open_external_port":80,"path_to_deployment":"\/usr\/local\/openunison\/work","ciphers":["TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384","TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384","TLS_RSA_WITH_AES_256_GCM_SHA384","TLS_ECDH_ECDSA_WITH_AES_256_GCM_SHA384","TLS_ECDH_RSA_WITH_AES_256_GCM_SHA384","TLS_DHE_RSA_WITH_AES_256_GCM_SHA384"],"path_to_env_file":"\/etc\/openunison\/ou.env","open_port":8080},"saml_remote_idp":[{"mapping":{"logout_url":"IDP_LOGOUT","signing_cert_alias":"idp-saml2-sig","post_url":"IDP_POST","encryption_cert_alias":"idp-saml2-enc","entity_id":"IDP_ENTITY_ID","redirect_url":"IDP_REDIR"},"source":{"url":"https:\/\/.okta.com\/app\/\/sso\/saml\/metadata"}}],"hosts":[{"secret_name":"ou-tls-certificate","ingress_name":"openunison","names":[{"env_var":"OU_HOST","name":"k8sou.com"},{"env_var":"K8S_DASHBOARD_HOST","name":"k8sdb.com"}],"annotations":[],"ingress_type":"nginx"}],"replicas":1,"non_secret_data":[{"name":"K8S_URL","value":"https:\/\/vf11-vos-aks-prod-dns-a6211d47.hcp.eastus2.azmk8s.io:443"},{"name":"SESSION_INACTIVITY_TIMEOUT_SECONDS","value":"900"},{"name":"K8S_DASHBOARD_NAMESPACE","value":"kubernetes-dashboard"},{"name":"K8S_DASHBOARD_SERVICE","value":"kubernetes-dashboard"},{"name":"K8S_CLUSTER_NAME","value":"openunison-cp"},{"name":"OPENUNISON_PROVISIONING_ENABLED","value":"false"},{"name":"K8S_IMPERSONATION","value":"false"},{"name":"PROMETHEUS_SERVICE_ACCOUNT","value":"system:serviceaccount:monitoring:prometheus-k8s"},{"name":"OU_SVC_NAME","value":"openunison-orchestra.openunison.svc"},{"name":"K8S_TOKEN_TYPE","value":"legacy"},{"name":"K8S_DB_SSO","value":"saml2"},{"name":"PROMETHEUS_SERVICE_ACCOUNT","value":"system:serviceaccount:monitoring:prometheus-k8s"},{"name":"SHOW_PORTAL_ORGS","value":"false"}],"key_store":{"trusted_certificates"cert"}],"static_keys":[{"name":"session-unison","version":1},{"name":"lastmile-oidc","version":1}],"update_controller":{"days_to_expire":10,"image":"docker.io\/tremolosecurity\/kubernetes-artifact-deployment:1.1.0","schedule":"0 2 * * *"},"key_pairs":{"create_keypair_template":[{"name":"ou","value":"Kubernetes"},{"name":"o","value":""},{"name":"l","value":"Production Cluster"},{"name":"st","value":"East US 2"},{"name":"c","value":"USA"}],"keys":[{"create_data":{"server_name":"openunison-orchestra.openunison.svc","subject_alternative_names":[],"ca_cert":true,"sign_by_k8s_ca":false,"key_size":2048},"name":"unison-tls","import_into_ks":"keypair"},{"create_data":{"server_name":"kubernetes-dashboard.kubernetes-dashboard.svc","subject_alternative_names":[],"secret_info":{"key_name":"dashboard.key","cert_name":"dashboard.crt","type_of_secret":"Opaque"},"ca_cert":true,"delete_pods_labels":["k8s-app=kubernetes-dashboard"],"sign_by_k8s_ca":false,"key_size":2048,"target_namespace":"kubernetes-dashboard"},"replace_if_exists":true,"name":"kubernetes-dashboard","tls_secret_name":"kubernetes-dashboard-certs","import_into_ks":"certificate"},{"create_data":{"server_name":"unison-saml2-rp-sig","subject_alternative_names":[],"ca_cert":true,"sign_by_k8s_ca":false,"key_size":2048},"name":"unison-saml2-rp-sig","import_into_ks":"keypair"},{"create_data":{"server_name":"remote-k8s-idp-sig","subject_alternative_names":[],"ca_cert":false,"sign_by_k8s_ca":false,"key_size":2048},"name":"remote-k8s-idp-sig","import_into_ks":"keypair"}]}},"enable_activemq":false,"dest_secret":"orchestra","activemq_image":"docker.io\/tremolosecurity\/activemq-docker:latest","myvd_configmap":"myvd","deployment_data":{"node_selectors":[],"tokenrequest_api":{"audience":"api","expirationSeconds":600,"enabled":false},"liveness_probe_command":["\/usr\/local\/openunison\/bin\/check_alive.sh","https:\/\/127.0.0.1:8443\/auth\/idp\/k8sIdp\/.well-known\/openid-configuration","issuer","https:\/\/127.0.0.1:8443\/check_alive","alive"],"readiness_probe_command":["\/usr\/local\/openunison\/bin\/check_alive.sh","https:\/\/127.0.0.1:8443\/auth\/idp\/k8sIdp\/.well-known\/openid-configuration","issuer","https:\/\/127.0.0.1:8443\/check_alive","alive"],"pull_secret":""},"secret_data":["AD_BIND_PASSWORD","K8S_DB_SECRET","unisonKeystorePassword"]},"status":{"idpCertificateFingerprints":{"http:\/\/www.okta.com\/":"QD8WSqMyA+sbTLlZtRKI+KyIqyeX01z9vMxaEN5Oz5Y="},"digest":"nMEwJy2o7KZqS72Q3bg1P9yAXmEJ3Bk1eID+1IwpAwk=","conditions":{"lastTransitionTime":"2023-03-20 08:43:51UTC","type":"Completed","status":"True"}}}}
Getting host variable names
Host  #0
Name #0
OU_HOST
k8sou.com
Name #1
K8S_DASHBOARD_HOST
k8sdb.com
Done adding host variables
Creating openunison keystore
Storing k8s certificate
Storing trusted certificates
Processing keypairs
Number of keys : '4'
0

Processing key 'unison-tls'
Checking if kubernetes secret exists
Secret exists
Adding existing secret to keystore
Storing to keystore
0
1

Processing key 'kubernetes-dashboard'
Checking if kubernetes secret exists
Secret exists
Adding existing secret to keystore
Storing just the certificate3
1
2

Processing key 'unison-saml2-rp-sig'
Checking if kubernetes secret exists
Secret exists
Adding existing secret to keystore
Storing to keystore
2
3

Processing key 'remote-k8s-idp-sig'
Checking if kubernetes secret exists
Secret exists
Adding existing secret to keystore
Storing to keystore
3
loading static secrets from /api/v1/namespaces/openunison/secrets/orchestra-static-keys
Secret exists, deleting
importing 'lastmile-oidc' from secret
importing 'session-unison' from secret
Checking static key :'session-unison'
import key from secret
Checking static key :'lastmile-oidc'
import key from secret
Posting secret
Remote Identity Providers : [object Object]
Downloading metadata from : https://.okta.com/app//sso/saml/metadata'
XML Metadata :
--------------
<?xml version="1.0" encoding="UTF-8"?><md:EntityDescriptor entityID="http://www.okta.com/" xmlns:md="urn:oasis:names:tc:SAML:2.0:metadata"><md:IDPSSODescriptor WantAuthnRequestsSigned="false" protocolSupportEnumeration="urn:oasis:names:tc:SAML:2.0:protocol"><md:KeyDescriptor use="signing"><ds:KeyInfo xmlns:ds="http://www.w3.org/2000/09/xmldsig#"><ds:X509Data><ds:X509Certificate>MIIDoDCCAoigAwIBAgIGAYb+OcMcMA0GCSqGSIb3DQEBCwUAMIGQMQswCQYDVQQGEwJVUzETMBEG
A1UECAwKQ2FsaWZvcm5pYTEWMBQGA1UEBwwNU2FuIEZyYW5jaXNjbzENMAsGA1UECgwET2t0YTEU
MBIGA1UECwwLU1NPUHJvdmlkZXIxETAPBgNVBAMMCHZlcmlmb25lMRwwGgYJKoZIhvcNAQkBFg1p
bmZvQG9rdGEuY29tMB4XDTIzMDMyMDA4NTMwM1oXDTMzMDMyMDA4NTQwM1owgZAxCzAJBgNVBAYT
AlVTMRMwEQYDVQQIDApDYWxpZm9ybmlhMRYwFAYDVQQHDA1TYW4gRnJhbmNpc2NvMQ0wCwYDVQQK
DARPa3RhMRQwEgYDVQQLDAtTU09Qcm92aWRlcjERMA8GA1UEAwwIdmVyaWZvbmUxHDAaBgkqhkiG
9w0BCQEWDWluZm9Ab2t0IwXRdLSfX3eTyEOoIORttAsjwqS8M3vN
zQLJBGRvTxc1INqetYwhdnUIa4ZtUo28aMTiRSdiJiX9iFho/ICLsM/VOGAjGKQ+0RW8LXvf/Tj3
lD1nP5Qs86e9H1OTsNIeCVsRgpU=</ds:X509Certificate></ds:X509Data></ds:KeyInfo></md:KeyDescriptor><md:NameIDFormat>urn:oasis:names:tc:SAML:1.1:nameid-format:unspecified</md:NameIDFormat><md:NameIDFormat>urn:oasis:names:tc:SAML:1.1:nameid-format:emailAddress</md:NameIDFormat><md:SingleSignOnService Binding="urn:oasis:names:tc:SAML:2.0:bindings:HTTP-POST" Location="https://.okta.com/app/_vosaks_1//sso/saml"/><md:SingleSignOnService Binding="urn:oasis:names:tc:SAML:2.0:bindings:HTTP-Redirect" Location="https://.okta.com/app/_vosaks_1//sso/saml"/></md:IDPSSODescriptor></md:EntityDescriptor>
--------------
Downloaded
Saving fingerprints
{idpCertificateFingerprints=[object Object]}
Importing CACerts
New cacerts generated : java.security.KeyStore@2d834691
DIGEST : EKpEduBSNKBAj7H5k7VC0VAaGcZST+cnJ2+a94wAOC4=
No secret data has changed, not updating the secret
Done
Problem calling '/api/v1/namespaces/openunison/secrets/amq-secrets-orchestra' - 404
{"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"secrets \"amq-secrets-orchestra\" not found","reason":"NotFound","details":{"name":"amq-secrets-orchestra","kind":"secrets"},"code":404}

Obj '/api/v1/namespaces/openunison/secrets/amq-secrets-orchestra' doesn't exist, skipping
Problem calling '/api/v1/namespaces/openunison/secrets/amq-env-secrets-orchestra' - 404
{"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"secrets \"amq-env-secrets-orchestra\" not found","reason":"NotFound","details":{"name":"amq-env-secrets-orchestra","kind":"secrets"},"code":404}

Obj '/api/v1/namespaces/openunison/secrets/amq-env-secrets-orchestra' doesn't exist, skipping
Problem calling '/api/v1/namespaces/openunison/services/amq' - 404
{"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"services \"amq\" not found","reason":"NotFound","details":{"name":"amq","kind":"services"},"code":404}

Obj '/api/v1/namespaces/openunison/services/amq' doesn't exist, skipping
Problem calling '/apis/apps/v1/namespaces/openunison/deployments/amq-orchestra' - 404
{"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"deployments.apps \"amq-orchestra\" not found","reason":"NotFound","details":{"name":"amq-orchestra","group":"apps","kind":"deployments"},"code":404}

Obj '/apis/apps/v1/namespaces/openunison/deployments/amq-orchestra' doesn't exist, skipping
Ingress already exists, not creating
Problem calling '/apis/batch/v1beta1/namespaces/openunison/cronjobs' - 404
{"apiVersion":"batch/v1beta1","kind":"CronJob","metadata":{"labels":{"app":"openunison-orchestra","operated-by":"openunison-operator"},"name":"check-certs-orchestra","namespace":"openunison"},"spec":{"schedule":"0 2 * * *","jobTemplate":{"spec":{"template":{"spec":{"containers":[{"name":"check-certs-orchestra","image":"docker.io/tremolosecurity/kubernetes-artifact-deployment:1.1.0","env":[{"name":"CERT_DAYS_EXPIRE","value":"10"}],"command":["java","-jar","/usr/local/artifactdeploy/artifact-deploy.jar","-extraCertsPath","/etc/extracerts","-installScriptURL","file:///etc/input-maps/cert-check.js","-kubernetesURL","https://kubernetes.default.svc.cluster.local","-rootCaPath","/var/run/secrets/kubernetes.io/serviceaccount/ca.crt","-secretsPath","/etc/input-maps/input.props","-tokenPath","/var/run/secrets/kubernetes.io/serviceaccount/token","-deploymentTemplate","file:///etc/input-maps/deployment.yaml"],"volumeMounts":[{"name":"extra-certs-dir","mountPath":"/etc/extracerts","readOnly":true},{"name":"input-maps","mountPath":"/etc/input-maps","readOnly":true}]}],"restartPolicy":"Never","serviceAccount":"openunison-operator","serviceAccountName":"openunison-operator","volumes":[{"name":"extra-certs-dir","configMap":{"name":"cert-controller-js-orchestra"}},{"name":"input-maps","configMap":{"name":"cert-controller-js-orchestra"}}]}},"backoffLimit":1}}}}
{"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"the server could not find the requested resource","reason":"NotFound","details":{},"code":404}

Problem calling '/apis/batch/v1beta1/namespaces/openunison/cronjobs' - 404
{"apiVersion":"batch/v1beta1","kind":"CronJob","metadata":{"labels":{"app":"openunison-orchestra","operated-by":"openunison-operator"},"name":"check-certs-orchestra","namespace":"openunison"},"spec":{"schedule":"0 2 * * *","jobTemplate":{"spec":{"template":{"spec":{"containers":[{"name":"check-certs-orchestra","image":"docker.io/tremolosecurity/kubernetes-artifact-deployment:1.1.0","env":[{"name":"CERT_DAYS_EXPIRE","value":"10"}],"command":["java","-jar","/usr/local/artifactdeploy/artifact-deploy.jar","-extraCertsPath","/etc/extracerts","-installScriptURL","file:///etc/input-maps/cert-check.js","-kubernetesURL","https://kubernetes.default.svc.cluster.local","-rootCaPath","/var/run/secrets/kubernetes.io/serviceaccount/ca.crt","-secretsPath","/etc/input-maps/input.props","-tokenPath","/var/run/secrets/kubernetes.io/serviceaccount/token","-deploymentTemplate","file:///etc/input-maps/deployment.yaml"],"volumeMounts":[{"name":"extra-certs-dir","mountPath":"/etc/extracerts","readOnly":true},{"name":"input-maps","mountPath":"/etc/input-maps","readOnly":true}]}],"restartPolicy":"Never","serviceAccount":"openunison-operator","serviceAccountName":"openunison-operator","volumes":[{"name":"extra-certs-dir","configMap":{"name":"cert-controller-js-orchestra"}},{"name":"input-maps","configMap":{"name":"cert-controller-js-orchestra"}}]}},"backoffLimit":1}}}}
{"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"the server could not find the requested resource","reason":"NotFound","details":{},"code":404}

looking up '/apis/admissionregistration.k8s.io/v1/validatingwebhookconfigurations/openunison-workflow-validation-orchestra'
{code=200, data={"kind":"ValidatingWebhookConfiguration","apiVersion":"admissionregistration.k8s.io/v1","metadata":{"name":"openunison-workflow-validation-orchestra","uid":"974bf337-71db-48cf-abaf-670c8c4c568b","resourceVersion":"61036274","generation":30,"creationTimestamp":"2023-03-20T14:31:56Z","labels":{"app.kubernetes.io/component":"webhooks","app.kubernetes.io/instance":"openunison-orchestra","app.kubernetes.io/managed-by":"Helm","app.kubernetes.io/name":"openunison","app.kubernetes.io/part-of":"openunison"},"annotations":{"argocd.argoproj.io/sync-wave":"15","meta.helm.sh/release-name":"orchestra","meta.helm.sh/release-namespace":"openunison"},"managedFields":[{"manager":"admissionsenforcer","operation":"Update","apiVersion":"admissionregistration.k8s.io/v1","time":"2023-03-20T20:50:29Z","fieldsType":"FieldsV1","fieldsV1":{"f:webhooks":{"k:{\"name\":\"applications-openunison.tremolo.io\"}":{"f:namespaceSelector":{}},"k:{\"name\":\"authchains-openunison.tremolo.io\"}":{"f:namespaceSelector":{}},"k:{\"name\":\"authmechs-openunison.tremolo.io\"}":{"f:namespaceSelector":{}},"k:{\"name\":\"customazs-openunison.tremolo.io\"}":{"f:namespaceSelector":{}},"k:{\"name\":\"workflows-openunison.tremolo.io\"}":{"f:namespaceSelector":{}}}}},{"manager":"ouctl","operation":"Update","apiVersion":"admissionregistration.k8s.io/v1","time":"2023-03-20T20:50:29Z","fieldsType":"FieldsV1","fieldsV1":{"f:metadata":{"f:annotations":{".":{},"f:argocd.argoproj.io/sync-wave":{},"f:meta.helm.sh/release-name":{},"f:meta.helm.sh/release-namespace":{}},"f:labels":{".":{},"f:app.kubernetes.io/component":{},"f:app.kubernetes.io/instance":{},"f:app.kubernetes.io/managed-by":{},"f:app.kubernetes.io/name":{},"f:app.kubernetes.io/part-of":{}}},"f:webhooks":{".":{},"k:{\"name\":\"applications-openunison.tremolo.io\"}":{".":{},"f:admissionReviewVersions":{},"f:clientConfig":{".":{},"f:caBundle":{},"f:service":{".":{},"f:name":{},"f:namespace":{},"f:path":{},"f:port":{}}},"f:failurePolicy":{},"f:matchPolicy":{},"f:name":{},"f:objectSelector":{},"f:rules":{},"f:sideEffects":{},"f:timeoutSeconds":{}},"k:{\"name\":\"authchains-openunison.tremolo.io\"}":{".":{},"f:admissionReviewVersions":{},"f:clientConfig":{".":{},"f:caBundle":{},"f:service":{".":{},"f:name":{},"f:namespace":{},"f:path":{},"f:port":{}}},"f:failurePolicy":{},"f:matchPolicy":{},"f:name":{},"f:objectSelector":{},"f:rules":{},"f:sideEffects":{},"f:timeoutSeconds":{}},"k:{\"name\":\"authmechs-openunison.tremolo.io\"}":{".":{},"f:admissionReviewVersions":{},"f:clientConfig":{".":{},"f:caBundle":{},"f:service":{".":{},"f:name":{},"f:namespace":{},"f:path":{},"f:port":{}}},"f:failurePolicy":{},"f:matchPolicy":{},"f:name":{},"f:objectSelector":{},"f:rules":{},"f:sideEffects":{},"f:timeoutSeconds":{}},"k:{\"name\":\"customazs-openunison.tremolo.io\"}":{".":{},"f:admissionReviewVersions":{},"f:clientConfig":{".":{},"f:caBundle":{},"f:service":{".":{},"f:name":{},"f:namespace":{},"f:path":{},"f:port":{}}},"f:failurePolicy":{},"f:matchPolicy":{},"f:name":{},"f:objectSelector":{},"f:rules":{},"f:sideEffects":{},"f:timeoutSeconds":{}},"k:{\"name\":\"workflows-openunison.tremolo.io\"}":{".":{},"f:admissionReviewVersions":{},"f:clientConfig":{".":{},"f:caBundle":{},"f:service":{".":{},"f:name":{},"f:namespace":{},"f:path":{},"f:port":{}}},"f:failurePolicy":{},"f:matchPolicy":{},"f:name":{},"f:objectSelector":{},"f:rules":{},"f:sideEffects":{},"f:timeoutSeconds":{}}}}}]},"webhooks":[{"name":"workflows-openunison.tremolo.io","clientConfig":{"service":{"namespace":"openunison","name":"openunison-orchestra","path":"/k8s/webhooks/v1/workflows","port":443},"caBundle":"LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUVGekNDQXYrZ0F3SUJBZ0lHQVhWWU5KTXhNQTBHQ1NxR1NJYjNEUUVCQ3dVQU1JR1hNUkl3RUFZRFZRUUcNCkV3bE5lVU52ZFc1MGNua3hHVEFYQmdOVkJBZ1RFRk4wWVhSbElHOW1JRU5zZFhOMFpYSXhFekFSQmdOVkJBY1QNCkNrMTVJRU5zZFhOMFpYSXhEakFNQmdOVkJBb1RCVTE1VDNKbk1STXdFUVlEVlFRTEV3cExkV0psY201bGRHVnoNCk1Td3dLZ1lEVlFRREV5TnZjR1Z1ZFc1cGMyOXVMVzl5WTJobGMzUnlZUzV2Y0dWdWRXNXBjMjl1TG5OMll6QWUNCkZ3MHlNREV3TWpRd01USTBNREZhRncweU1URXdNalF3TVRJME1ERmFNSUdYTVJJd0VBWURWUVFHRXdsTmVVTnYNCmRXNTBjbmt4R1RBWEJnTlZCQWdURUZOMFlYUmxJRzltSUVOc2RYTjBaWEl4RXpBUkJnTlZCQWNUQ2sxNUlFTnMNCmRYTjBaWEl4RGpBTUJnTlZCQW9UQlUxNVQzSm5NUk13RVFZRFZRUUxFd3BMZFdKbGNtNWxkR1Z6TVN3d0tnWUQNClZRUURFeU52Y0dWdWRXNXBjMjl1TFc5eVkyaGxjM1J5WVM1dmNHVnVkVzVwYzI5dUxuTjJZekNDQVNJd0RRWUoNCktvWklodmNOQVFFQkJRQURnZ0VQQURDQ0FRb0NnZ0VCQUt0WW5Pd05wejhyNXNBQWVDRVcwaUM1WmcyNGoxMDINCjlvcFRMSFE2S3RzK0xVZlI4ekV4VFJncG54V1hIbjRTbitKcHl5d01tdktRU3hmT3lCak1nNmlvcDlNZXQ2b1QNCkhWNTlLb0lzM2t6bitFeHFvd0JoeVRyOGFmL3NZb1pXdTlrcXkxWmErcEgzQXpiSWdIM2JjNllGQkQrak8yWDENCmNtSXdCOXFFbytRU2IvOURQVE1tU0tvbGZCdndha1ZVME1lYUwwbGlYUVNlZG5yNHZvZ20xRWR3Y1hRWkxheXANCkVadUVmTDBySVY1bTJiMzhncWYrcnNTSU9QT0RNVWN2VFpmSnQxczZWK2xTTXA2ZVp6TG9GMlF5MVViOHhBQUkNCldYb3hWOXVMdXFFb1FxZEpKTWI3b0ZGVUUvTmpodkVoV0E0c3ljQ3A5Yk1qVi9YMUp2QXlzQTBDQXdFQUFhTm4NCk1HVXdEd1lEVlIwVEFRSC9CQVV3QXdFQi96QU9CZ05WSFE4QkFmOEVCQU1DQWdRd0VnWURWUjBsQVFIL0JBZ3cNCkJnWUVWUjBsQURBdUJnTlZIUkVFSnpBbGdpTnZjR1Z1ZFc1cGMyOXVMVzl5WTJobGMzUnlZUzV2Y0dWdWRXNXANCmMyOXVMbk4yWXpBTkJna3Foa2lHOXcwQkFRc0ZBQU9DQVFFQWJ6dzlqV09tci9LMmdJdGdpVkd3UGFYRmNJZHMNCjNiamdvVkxYREJkVTFKK3lidm5EVnJEa2owNTlPRzVBOXhRTzEzWVEvbGIreVJNckEwZTdCWEVSR3oxZVlwZmsNCkZoczhCb3QrcENSRVlNMTU3cEtvOUpiN09FTVA4S0FET2xxMFpHYitqN2Zac0NSWlhSbzk5QUpmMms4R0s2WVoNCnFGZE43c1BXUnZXcDRBK2RRbVdPa1RKUVFhdVp5N1lvU1JROUFnSkxvdUxqbzd4QjdLM1hwTXBkNFl0UzBYd1kNClJxV0dreGh6eGtDaWJTZFhrSUFGSkZxc0pZVHFTdHQ4ZXdZaUQyL1Zob3ozcjJjZC8vcXZIWXljL1U5Zm9zM3UNCmR5R1NPWGhnS2pJSTRQblovOS9kemJDWFc0dlFFUVJEaENTTHpmSk5KS3NmMkt2aUNmM3BHa2tHNmc9PQ0KLS0tLS1FTkQgQ0VSVElGSUNBVEUtLS0tLQo="},"rules":[{"operations":["CREATE","UPDATE"],"apiGroups":["openunison.tremolo.io"],"apiVersions":["v1"],"resources":["workflows"],"scope":"Namespaced"}],"failurePolicy":"Fail","matchPolicy":"Equivalent","namespaceSelector":{"matchExpressions":[{"key":"kubernetes.io/metadata.name","operator":"In","values":["openunison"]},{"key":"control-plane","operator":"DoesNotExist"}]},"objectSelector":{},"sideEffects":"None","timeoutSeconds":5,"admissionReviewVersions":["v1"]},{"name":"customazs-openunison.tremolo.io","clientConfig":{"service":{"namespace":"openunison","name":"openunison-orchestra","path":"/k8s/webhooks/v1/customazs","port":443},"caBundle":"LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUVGekNDQXYrZ0F3SUJBZ0lHQVhWWU5KTXhNQTBHQ1NxR1NJYjNEUUVCQ3dVQU1JR1hNUkl3RUFZRFZRUUcNCkV3bE5lVU52ZFc1MGNua3hHVEFYQmdOVkJBZ1RFRk4wWVhSbElHOW1JRU5zZFhOMFpYSXhFekFSQmdOVkJBY1QNCkNrMTVJRU5zZFhOMFpYSXhEakFNQmdOVkJBb1RCVTE1VDNKbk1STXdFUVlEVlFRTEV3cExkV0psY201bGRHVnoNCk1Td3dLZ1lEVlFRREV5TnZjR1Z1ZFc1cGMyOXVMVzl5WTJobGMzUnlZUzV2Y0dWdWRXNXBjMjl1TG5OMll6QWUNCkZ3MHlNREV3TWpRd01USTBNREZhRncweU1URXdNalF3TVRJME1ERmFNSUdYTVJJd0VBWURWUVFHRXdsTmVVTnYNCmRXNTBjbmt4R1RBWEJnTlZCQWdURUZOMFlYUmxJRzltSUVOc2RYTjBaWEl4RXpBUkJnTlZCQWNUQ2sxNUlFTnMNCmRYTjBaWEl4RGpBTUJnTlZCQW9UQlUxNVQzSm5NUk13RVFZRFZRUUxFd3BMZFdKbGNtNWxkR1Z6TVN3d0tnWUQNClZRUURFeU52Y0dWdWRXNXBjMjl1TFc5eVkyaGxjM1J5WVM1dmNHVnVkVzVwYzI5dUxuTjJZekNDQVNJd0RRWUoNCktvWklodmNOQVFFQkJRQURnZ0VQQURDQ0FRb0NnZ0VCQUt0WW5Pd05wejhyNXNBQWVDRVcwaUM1WmcyNGoxMDINCjlvcFRMSFE2S3RzK0xVZlI4ekV4VFJncG54V1hIbjRTbitKcHl5d01tdktRU3hmT3lCak1nNmlvcDlNZXQ2b1QNCkhWNTlLb0lzM2t6bitFeHFvd0JoeVRyOGFmL3NZb1pXdTlrcXkxWmErcEgzQXpiSWdIM2JjNllGQkQrak8yWDENCmNtSXdCOXFFbytRU2IvOURQVE1tU0tvbGZCdndha1ZVME1lYUwwbGlYUVNlZG5yNHZvZ20xRWR3Y1hRWkxheXANCkVadUVmTDBySVY1bTJiMzhncWYrcnNTSU9QT0RNVWN2VFpmSnQxczZWK2xTTXA2ZVp6TG9GMlF5MVViOHhBQUkNCldYb3hWOXVMdXFFb1FxZEpKTWI3b0ZGVUUvTmpodkVoV0E0c3ljQ3A5Yk1qVi9YMUp2QXlzQTBDQXdFQUFhTm4NCk1HVXdEd1lEVlIwVEFRSC9CQVV3QXdFQi96QU9CZ05WSFE4QkFmOEVCQU1DQWdRd0VnWURWUjBsQVFIL0JBZ3cNCkJnWUVWUjBsQURBdUJnTlZIUkVFSnpBbGdpTnZjR1Z1ZFc1cGMyOXVMVzl5WTJobGMzUnlZUzV2Y0dWdWRXNXANCmMyOXVMbk4yWXpBTkJna3Foa2lHOXcwQkFRc0ZBQU9DQVFFQWJ6dzlqV09tci9LMmdJdGdpVkd3UGFYRmNJZHMNCjNiamdvVkxYREJkVTFKK3lidm5EVnJEa2owNTlPRzVBOXhRTzEzWVEvbGIreVJNckEwZTdCWEVSR3oxZVlwZmsNCkZoczhCb3QrcENSRVlNMTU3cEtvOUpiN09FTVA4S0FET2xxMFpHYitqN2Zac0NSWlhSbzk5QUpmMms4R0s2WVoNCnFGZE43c1BXUnZXcDRBK2RRbVdPa1RKUVFhdVp5N1lvU1JROUFnSkxvdUxqbzd4QjdLM1hwTXBkNFl0UzBYd1kNClJxV0dreGh6eGtDaWJTZFhrSUFGSkZxc0pZVHFTdHQ4ZXdZaUQyL1Zob3ozcjJjZC8vcXZIWXljL1U5Zm9zM3UNCmR5R1NPWGhnS2pJSTRQblovOS9kemJDWFc0dlFFUVJEaENTTHpmSk5KS3NmMkt2aUNmM3BHa2tHNmc9PQ0KLS0tLS1FTkQgQ0VSVElGSUNBVEUtLS0tLQo="},"rules":[{"operations":["CREATE","UPDATE"],"apiGroups":["openunison.tremolo.io"],"apiVersions":["v1"],"resources":["customazs"],"scope":"Namespaced"}],"failurePolicy":"Fail","matchPolicy":"Equivalent","namespaceSelector":{"matchExpressions":[{"key":"kubernetes.io/metadata.name","operator":"In","values":["openunison"]},{"key":"control-plane","operator":"DoesNotExist"}]},"objectSelector":{},"sideEffects":"None","timeoutSeconds":5,"admissionReviewVersions":["v1"]},{"name":"authmechs-openunison.tremolo.io","clientConfig":{"service":{"namespace":"openunison","name":"openunison-orchestra","path":"/k8s/webhooks/v1/authmechs","port":443},"caBundle":"LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUVGekNDQXYrZ0F3SUJBZ0lHQVhWWU5KTXhNQTBHQ1NxR1NJYjNEUUVCQ3dVQU1JR1hNUkl3RUFZRFZRUUcNCkV3bE5lVU52ZFc1MGNua3hHVEFYQmdOVkJBZ1RFRk4wWVhSbElHOW1JRU5zZFhOMFpYSXhFekFSQmdOVkJBY1QNCkNrMTVJRU5zZFhOMFpYSXhEakFNQmdOVkJBb1RCVTE1VDNKbk1STXdFUVlEVlFRTEV3cExkV0psY201bGRHVnoNCk1Td3dLZ1lEVlFRREV5TnZjR1Z1ZFc1cGMyOXVMVzl5WTJobGMzUnlZUzV2Y0dWdWRXNXBjMjl1TG5OMll6QWUNCkZ3MHlNREV3TWpRd01USTBNREZhRncweU1URXdNalF3TVRJME1ERmFNSUdYTVJJd0VBWURWUVFHRXdsTmVVTnYNCmRXNTBjbmt4R1RBWEJnTlZCQWdURUZOMFlYUmxJRzltSUVOc2RYTjBaWEl4RXpBUkJnTlZCQWNUQ2sxNUlFTnMNCmRYTjBaWEl4RGpBTUJnTlZCQW9UQlUxNVQzSm5NUk13RVFZRFZRUUxFd3BMZFdKbGNtNWxkR1Z6TVN3d0tnWUQNClZRUURFeU52Y0dWdWRXNXBjMjl1TFc5eVkyaGxjM1J5WVM1dmNHVnVkVzVwYzI5dUxuTjJZekNDQVNJd0RRWUoNCktvWklodmNOQVFFQkJRQURnZ0VQQURDQ0FRb0NnZ0VCQUt0WW5Pd05wejhyNXNBQWVDRVcwaUM1WmcyNGoxMDINCjlvcFRMSFE2S3RzK0xVZlI4ekV4VFJncG54V1hIbjRTbitKcHl5d01tdktRU3hmT3lCak1nNmlvcDlNZXQ2b1QNCkhWNTlLb0lzM2t6bitFeHFvd0JoeVRyOGFmL3NZb1pXdTlrcXkxWmErcEgzQXpiSWdIM2JjNllGQkQrak8yWDENCmNtSXdCOXFFbytRU2IvOURQVE1tU0tvbGZCdndha1ZVME1lYUwwbGlYUVNlZG5yNHZvZ20xRWR3Y1hRWkxheXANCkVadUVmTDBySVY1bTJiMzhncWYrcnNTSU9QT0RNVWN2VFpmSnQxczZWK2xTTXA2ZVp6TG9GMlF5MVViOHhBQUkNCldYb3hWOXVMdXFFb1FxZEpKTWI3b0ZGVUUvTmpodkVoV0E0c3ljQ3A5Yk1qVi9YMUp2QXlzQTBDQXdFQUFhTm4NCk1HVXdEd1lEVlIwVEFRSC9CQVV3QXdFQi96QU9CZ05WSFE4QkFmOEVCQU1DQWdRd0VnWURWUjBsQVFIL0JBZ3cNCkJnWUVWUjBsQURBdUJnTlZIUkVFSnpBbGdpTnZjR1Z1ZFc1cGMyOXVMVzl5WTJobGMzUnlZUzV2Y0dWdWRXNXANCmMyOXVMbk4yWXpBTkJna3Foa2lHOXcwQkFRc0ZBQU9DQVFFQWJ6dzlqV09tci9LMmdJdGdpVkd3UGFYRmNJZHMNCjNiamdvVkxYREJkVTFKK3lidm5EVnJEa2owNTlPRzVBOXhRTzEzWVEvbGIreVJNckEwZTdCWEVSR3oxZVlwZmsNCkZoczhCb3QrcENSRVlNMTU3cEtvOUpiN09FTVA4S0FET2xxMFpHYitqN2Zac0NSWlhSbzk5QUpmMms4R0s2WVoNCnFGZE43c1BXUnZXcDRBK2RRbVdPa1RKUVFhdVp5N1lvU1JROUFnSkxvdUxqbzd4QjdLM1hwTXBkNFl0UzBYd1kNClJxV0dreGh6eGtDaWJTZFhrSUFGSkZxc0pZVHFTdHQ4ZXdZaUQyL1Zob3ozcjJjZC8vcXZIWXljL1U5Zm9zM3UNCmR5R1NPWGhnS2pJSTRQblovOS9kemJDWFc0dlFFUVJEaENTTHpmSk5KS3NmMkt2aUNmM3BHa2tHNmc9PQ0KLS0tLS1FTkQgQ0VSVElGSUNBVEUtLS0tLQo="},"rules":[{"operations":["CREATE","UPDATE"],"apiGroups":["openunison.tremolo.io"],"apiVersions":["v1"],"resources":["authmechs"],"scope":"Namespaced"}],"failurePolicy":"Fail","matchPolicy":"Equivalent","namespaceSelector":{"matchExpressions":[{"key":"kubernetes.io/metadata.name","operator":"In","values":["openunison"]},{"key":"control-plane","operator":"DoesNotExist"}]},"objectSelector":{},"sideEffects":"None","timeoutSeconds":5,"admissionReviewVersions":["v1"]},{"name":"authchains-openunison.tremolo.io","clientConfig":{"service":{"namespace":"openunison","name":"openunison-orchestra","path":"/k8s/webhooks/v1/authchains","port":443},"caBundle":"LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUVGekNDQXYrZ0F3SUJBZ0lHQVhWWU5KTXhNQTBHQ1NxR1NJYjNEUUVCQ3dVQU1JR1hNUkl3RUFZRFZRUUcNCkV3bE5lVU52ZFc1MGNua3hHVEFYQmdOVkJBZ1RFRk4wWVhSbElHOW1JRU5zZFhOMFpYSXhFekFSQmdOVkJBY1QNCkNrMTVJRU5zZFhOMFpYSXhEakFNQmdOVkJBb1RCVTE1VDNKbk1STXdFUVlEVlFRTEV3cExkV0psY201bGRHVnoNCk1Td3dLZ1lEVlFRREV5TnZjR1Z1ZFc1cGMyOXVMVzl5WTJobGMzUnlZUzV2Y0dWdWRXNXBjMjl1TG5OMll6QWUNCkZ3MHlNREV3TWpRd01USTBNREZhRncweU1URXdNalF3TVRJME1ERmFNSUdYTVJJd0VBWURWUVFHRXdsTmVVTnYNCmRXNTBjbmt4R1RBWEJnTlZCQWdURUZOMFlYUmxJRzltSUVOc2RYTjBaWEl4RXpBUkJnTlZCQWNUQ2sxNUlFTnMNCmRYTjBaWEl4RGpBTUJnTlZCQW9UQlUxNVQzSm5NUk13RVFZRFZRUUxFd3BMZFdKbGNtNWxkR1Z6TVN3d0tnWUQNClZRUURFeU52Y0dWdWRXNXBjMjl1TFc5eVkyaGxjM1J5WVM1dmNHVnVkVzVwYzI5dUxuTjJZekNDQVNJd0RRWUoNCktvWklodmNOQVFFQkJRQURnZ0VQQURDQ0FRb0NnZ0VCQUt0WW5Pd05wejhyNXNBQWVDRVcwaUM1WmcyNGoxMDINCjlvcFRMSFE2S3RzK0xVZlI4ekV4VFJncG54V1hIbjRTbitKcHl5d01tdktRU3hmT3lCak1nNmlvcDlNZXQ2b1QNCkhWNTlLb0lzM2t6bitFeHFvd0JoeVRyOGFmL3NZb1pXdTlrcXkxWmErcEgzQXpiSWdIM2JjNllGQkQrak8yWDENCmNtSXdCOXFFbytRU2IvOURQVE1tU0tvbGZCdndha1ZVME1lYUwwbGlYUVNlZG5yNHZvZ20xRWR3Y1hRWkxheXANCkVadUVmTDBySVY1bTJiMzhncWYrcnNTSU9QT0RNVWN2VFpmSnQxczZWK2xTTXA2ZVp6TG9GMlF5MVViOHhBQUkNCldYb3hWOXVMdXFFb1FxZEpKTWI3b0ZGVUUvTmpodkVoV0E0c3ljQ3A5Yk1qVi9YMUp2QXlzQTBDQXdFQUFhTm4NCk1HVXdEd1lEVlIwVEFRSC9CQVV3QXdFQi96QU9CZ05WSFE4QkFmOEVCQU1DQWdRd0VnWURWUjBsQVFIL0JBZ3cNCkJnWUVWUjBsQURBdUJnTlZIUkVFSnpBbGdpTnZjR1Z1ZFc1cGMyOXVMVzl5WTJobGMzUnlZUzV2Y0dWdWRXNXANCmMyOXVMbk4yWXpBTkJna3Foa2lHOXcwQkFRc0ZBQU9DQVFFQWJ6dzlqV09tci9LMmdJdGdpVkd3UGFYRmNJZHMNCjNiamdvVkxYREJkVTFKK3lidm5EVnJEa2owNTlPRzVBOXhRTzEzWVEvbGIreVJNckEwZTdCWEVSR3oxZVlwZmsNCkZoczhCb3QrcENSRVlNMTU3cEtvOUpiN09FTVA4S0FET2xxMFpHYitqN2Zac0NSWlhSbzk5QUpmMms4R0s2WVoNCnFGZE43c1BXUnZXcDRBK2RRbVdPa1RKUVFhdVp5N1lvU1JROUFnSkxvdUxqbzd4QjdLM1hwTXBkNFl0UzBYd1kNClJxV0dreGh6eGtDaWJTZFhrSUFGSkZxc0pZVHFTdHQ4ZXdZaUQyL1Zob3ozcjJjZC8vcXZIWXljL1U5Zm9zM3UNCmR5R1NPWGhnS2pJSTRQblovOS9kemJDWFc0dlFFUVJEaENTTHpmSk5KS3NmMkt2aUNmM3BHa2tHNmc9PQ0KLS0tLS1FTkQgQ0VSVElGSUNBVEUtLS0tLQo="},"rules":[{"operations":["CREATE","UPDATE"],"apiGroups":["openunison.tremolo.io"],"apiVersions":["v1"],"resources":["authchains"],"scope":"Namespaced"}],"failurePolicy":"Fail","matchPolicy":"Equivalent","namespaceSelector":{"matchExpressions":[{"key":"kubernetes.io/metadata.name","operator":"In","values":["openunison"]},{"key":"control-plane","operator":"DoesNotExist"}]},"objectSelector":{},"sideEffects":"None","timeoutSeconds":5,"admissionReviewVersions":["v1"]},{"name":"applications-openunison.tremolo.io","clientConfig":{"service":{"namespace":"openunison","name":"openunison-orchestra","path":"/k8s/webhooks/v1/applications","port":443},"caBundle":"LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUVGekNDQXYrZ0F3SUJBZ0lHQVhWWU5KTXhNQTBHQ1NxR1NJYjNEUUVCQ3dVQU1JR1hNUkl3RUFZRFZRUUcNCkV3bE5lVU52ZFc1MGNua3hHVEFYQmdOVkJBZ1RFRk4wWVhSbElHOW1JRU5zZFhOMFpYSXhFekFSQmdOVkJBY1QNCkNrMTVJRU5zZFhOMFpYSXhEakFNQmdOVkJBb1RCVTE1VDNKbk1STXdFUVlEVlFRTEV3cExkV0psY201bGRHVnoNCk1Td3dLZ1lEVlFRREV5TnZjR1Z1ZFc1cGMyOXVMVzl5WTJobGMzUnlZUzV2Y0dWdWRXNXBjMjl1TG5OMll6QWUNCkZ3MHlNREV3TWpRd01USTBNREZhRncweU1URXdNalF3TVRJME1ERmFNSUdYTVJJd0VBWURWUVFHRXdsTmVVTnYNCmRXNTBjbmt4R1RBWEJnTlZCQWdURUZOMFlYUmxJRzltSUVOc2RYTjBaWEl4RXpBUkJnTlZCQWNUQ2sxNUlFTnMNCmRYTjBaWEl4RGpBTUJnTlZCQW9UQlUxNVQzSm5NUk13RVFZRFZRUUxFd3BMZFdKbGNtNWxkR1Z6TVN3d0tnWUQNClZRUURFeU52Y0dWdWRXNXBjMjl1TFc5eVkyaGxjM1J5WVM1dmNHVnVkVzVwYzI5dUxuTjJZekNDQVNJd0RRWUoNCktvWklodmNOQVFFQkJRQURnZ0VQQURDQ0FRb0NnZ0VCQUt0WW5Pd05wejhyNXNBQWVDRVcwaUM1WmcyNGoxMDINCjlvcFRMSFE2S3RzK0xVZlI4ekV4VFJncG54V1hIbjRTbitKcHl5d01tdktRU3hmT3lCak1nNmlvcDlNZXQ2b1QNCkhWNTlLb0lzM2t6bitFeHFvd0JoeVRyOGFmL3NZb1pXdTlrcXkxWmErcEgzQXpiSWdIM2JjNllGQkQrak8yWDENCmNtSXdCOXFFbytRU2IvOURQVE1tU0tvbGZCdndha1ZVME1lYUwwbGlYUVNlZG5yNHZvZ20xRWR3Y1hRWkxheXANCkVadUVmTDBySVY1bTJiMzhncWYrcnNTSU9QT0RNVWN2VFpmSnQxczZWK2xTTXA2ZVp6TG9GMlF5MVViOHhBQUkNCldYb3hWOXVMdXFFb1FxZEpKTWI3b0ZGVUUvTmpodkVoV0E0c3ljQ3A5Yk1qVi9YMUp2QXlzQTBDQXdFQUFhTm4NCk1HVXdEd1lEVlIwVEFRSC9CQVV3QXdFQi96QU9CZ05WSFE4QkFmOEVCQU1DQWdRd0VnWURWUjBsQVFIL0JBZ3cNCkJnWUVWUjBsQURBdUJnTlZIUkVFSnpBbGdpTnZjR1Z1ZFc1cGMyOXVMVzl5WTJobGMzUnlZUzV2Y0dWdWRXNXANCmMyOXVMbk4yWXpBTkJna3Foa2lHOXcwQkFRc0ZBQU9DQVFFQWJ6dzlqV09tci9LMmdJdGdpVkd3UGFYRmNJZHMNCjNiamdvVkxYREJkVTFKK3lidm5EVnJEa2owNTlPRzVBOXhRTzEzWVEvbGIreVJNckEwZTdCWEVSR3oxZVlwZmsNCkZoczhCb3QrcENSRVlNMTU3cEtvOUpiN09FTVA4S0FET2xxMFpHYitqN2Zac0NSWlhSbzk5QUpmMms4R0s2WVoNCnFGZE43c1BXUnZXcDRBK2RRbVdPa1RKUVFhdVp5N1lvU1JROUFnSkxvdUxqbzd4QjdLM1hwTXBkNFl0UzBYd1kNClJxV0dreGh6eGtDaWJTZFhrSUFGSkZxc0pZVHFTdHQ4ZXdZaUQyL1Zob3ozcjJjZC8vcXZIWXljL1U5Zm9zM3UNCmR5R1NPWGhnS2pJSTRQblovOS9kemJDWFc0dlFFUVJEaENTTHpmSk5KS3NmMkt2aUNmM3BHa2tHNmc9PQ0KLS0tLS1FTkQgQ0VSVElGSUNBVEUtLS0tLQo="},"rules":[{"operations":["CREATE","UPDATE"],"apiGroups":["openunison.tremolo.io"],"apiVersions":["v1"],"resources":["applications"],"scope":"Namespaced"}],"failurePolicy":"Fail","matchPolicy":"Equivalent","namespaceSelector":{"matchExpressions":[{"key":"kubernetes.io/metadata.name","operator":"In","values":["openunison"]},{"key":"control-plane","operator":"DoesNotExist"}]},"objectSelector":{},"sideEffects":"None","timeoutSeconds":5,"admissionReviewVersions":["v1"]}]}
}
need to update the webhook
Done invoking javascript
Checking if need to create a status for : 'MODIFIED'
Generating status
Creating status patch : {"idpCertificateFingerprints":{"http:\/\/www.okta.com\/":"QD8WSqMyA+sbTLlZtRKI+KyIqyeX01z9vMxaEN5Oz5Y="},"digest":"bxdq2qQThzkUor2usXi3ilXeqjkN33Q6RrInYTu4\/Sk=","conditions":{"lastTransitionTime":"2023-03-20 08:50:35UTC","type":"Completed","status":"True"}}
Patching to '/apis/openunison.tremolo.io/v6/namespaces/openunison/openunisons/orchestra/status'
Patch : '{"status":{"idpCertificateFingerprints":{"http:\/\/www.okta.com\/":"QD8WSqMyA+sbTLlZtRKI+KyIqyeX01z9vMxaEN5Oz5Y="},"digest":"bxdq2qQThzkUor2usXi3ilXeqjkN33Q6RrInYTu4\/Sk=","conditions":{"lastTransitionTime":"2023-03-20 08:50:35UTC","type":"Completed","status":"True"}}}'
{code=200, data={"apiVersion":"openunison.tremolo.io/v6","kind":"OpenUnison","metadata":{"annotations":{"argocd.argoproj.io/sync-wave":"20","helm-update":"Mar 20 22:50:03 2023 EET","meta.helm.sh/release-name":"orchestra","meta.helm.sh/release-namespace":"openunison"},"creationTimestamp":"2023-03-20T20:43:44Z","generation":1,"labels":{"app.kubernetes.io/component":"openunison","app.kubernetes.io/instance":"openunison-orchestra","app.kubernetes.io/managed-by":"Helm","app.kubernetes.io/name":"openunison","app.kubernetes.io/part-of":"openunison"},"managedFields":[{"apiVersion":"openunison.tremolo.io/v6","fieldsType":"FieldsV1","fieldsV1":{"f:metadata":{"f:annotations":{".":{},"f:argocd.argoproj.io/sync-wave":{},"f:helm-update":{},"f:meta.helm.sh/release-name":{},"f:meta.helm.sh/release-namespace":{}},"f:labels":{".":{},"f:app.kubernetes.io/component":{},"f:app.kubernetes.io/instance":{},"f:app.kubernetes.io/managed-by":{},"f:app.kubernetes.io/name":{},"f:app.kubernetes.io/part-of":{}}},"f:spec":{".":{},"f:activemq_image":{},"f:deployment_data":{".":{},"f:liveness_probe_command":{},"f:node_selectors":{},"f:pull_secret":{},"f:readiness_probe_command":{},"f:tokenrequest_api":{".":{},"f:audience":{},"f:enabled":{},"f:expirationSeconds":{}}},"f:dest_secret":{},"f:enable_activemq":{},"f:hosts":{},"f:image":{},"f:key_store":{".":{},"f:key_pairs":{".":{},"f:create_keypair_template":{},"f:keys":{}},"f:static_keys":{},"f:trusted_certificates":{},"f:update_controller":{".":{},"f:days_to_expire":{},"f:image":{},"f:schedule":{}}},"f:myvd_configmap":{},"f:non_secret_data":{},"f:openunison_network_configuration":{".":{},"f:activemq_dir":{},"f:allowed_client_names":{},"f:ciphers":{},"f:client_auth":{},"f:force_to_secure":{},"f:open_external_port":{},"f:open_port":{},"f:path_to_deployment":{},"f:path_to_env_file":{},"f:quartz_dir":{},"f:secure_external_port":{},"f:secure_key_alias":{},"f:secure_port":{}},"f:replicas":{},"f:saml_remote_idp":{},"f:secret_data":{},"f:source_secret":{}}},"manager":"ouctl","operation":"Update","time":"2023-03-20T20:50:28Z"},{"apiVersion":"openunison.tremolo.io/v6","fieldsType":"FieldsV1","fieldsV1":{"f:status":{".":{},"f:conditions":{".":{},"f:lastTransitionTime":{},"f:status":{},"f:type":{}},"f:digest":{},"f:idpCertificateFingerprints":{".":{},"f:http://www.okta.com/":{}}}},"manager":"Apache-HttpClient","operation":"Update","subresource":"status","time":"2023-03-20T20:50:35Z"}],"name":"orchestra","namespace":"openunison","resourceVersion":"61036319","uid":"fc89707c-f553-48ff-8cf1-9835ee610cbb"},"spec":{"activemq_image":"docker.io/tremolosecurity/activemq-docker:latest","deployment_data":{"liveness_probe_command":["/usr/local/openunison/bin/check_alive.sh","https://127.0.0.1:8443/auth/idp/k8sIdp/.well-known/openid-configuration","issuer","https://127.0.0.1:8443/check_alive","alive"],"node_selectors":[],"pull_secret":"","readiness_probe_command":["/usr/local/openunison/bin/check_alive.sh","https://127.0.0.1:8443/auth/idp/k8sIdp/.well-known/openid-configuration","issuer","https://127.0.0.1:8443/check_alive","alive"],"tokenrequest_api":{"audience":"api","enabled":false,"expirationSeconds":600}},"dest_secret":"orchestra","enable_activemq":false,"hosts":[{"annotations":[],"ingress_name":"openunison","ingress_type":"nginx","names":[{"env_var":"OU_HOST","name":"k8sou.com"},{"env_var":"K8S_DASHBOARD_HOST","name":"k8sdb.com"}],"secret_name":"ou-tls-certificate"}],"image":"docker.io/tremolosecurity/openunison-k8s","key_store":{"key_pairs":{"create_keypair_template":[{"name":"ou","value":"Kubernetes"},{"name":"o","value":""},{"name":"l","value":"Production Cluster"},{"name":"st","value":"East US 2"},{"name":"c","value":"USA"}],"keys":[{"create_data":{"ca_cert":true,"key_size":2048,"server_name":"openunison-orchestra.openunison.svc","sign_by_k8s_ca":false,"subject_alternative_names":[]},"import_into_ks":"keypair","name":"unison-tls"},{"create_data":{"ca_cert":true,"delete_pods_labels":["k8s-app=kubernetes-dashboard"],"key_size":2048,"secret_info":{"cert_name":"dashboard.crt","key_name":"dashboard.key","type_of_secret":"Opaque"},"server_name":"kubernetes-dashboard.kubernetes-dashboard.svc","sign_by_k8s_ca":false,"subject_alternative_names":[],"target_namespace":"kubernetes-dashboard"},"import_into_ks":"certificate","name":"kubernetes-dashboard","replace_if_exists":true,"tls_secret_name":"kubernetes-dashboard-certs"},{"create_data":{"ca_cert":true,"key_size":2048,"server_name":"unison-saml2-rp-sig","sign_by_k8s_ca":false,"subject_alternative_names":[]},"import_into_ks":"keypair","name":"unison-saml2-rp-sig"},{"create_data":{"ca_cert":false,"key_size":2048,"server_name":"remote-k8s-idp-sig","sign_by_k8s_ca":false,"subject_alternative_names":[]},"import_into_ks":"keypair","name":"remote-k8s-idp-sig"}]},"static_keys":[{"name":"session-unison","version":1},{"name":"lastmile-oidc","version":1}],"trusted_certificates":[{"name":"okta{"days_to_expire":10,"image":"docker.io/tremolosecurity/kubernetes-artifact-deployment:1.1.0","schedule":"0 2 * * *"}},"myvd_configmap":"myvd","non_secret_data":[{"name":"K8S_URL","value":"https://vf11-vos-aks-prod-dns-a6211d47.hcp.eastus2.azmk8s.io:443"},{"name":"SESSION_INACTIVITY_TIMEOUT_SECONDS","value":"900"},{"name":"K8S_DASHBOARD_NAMESPACE","value":"kubernetes-dashboard"},{"name":"K8S_DASHBOARD_SERVICE","value":"kubernetes-dashboard"},{"name":"K8S_CLUSTER_NAME","value":"openunison-cp"},{"name":"OPENUNISON_PROVISIONING_ENABLED","value":"false"},{"name":"K8S_IMPERSONATION","value":"false"},{"name":"PROMETHEUS_SERVICE_ACCOUNT","value":"system:serviceaccount:monitoring:prometheus-k8s"},{"name":"OU_SVC_NAME","value":"openunison-orchestra.openunison.svc"},{"name":"K8S_TOKEN_TYPE","value":"legacy"},{"name":"K8S_DB_SSO","value":"saml2"},{"name":"PROMETHEUS_SERVICE_ACCOUNT","value":"system:serviceaccount:monitoring:prometheus-k8s"},{"name":"SHOW_PORTAL_ORGS","value":"false"}],"openunison_network_configuration":{"activemq_dir":"/tmp/amq","allowed_client_names":[],"ciphers":["TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384","TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384","TLS_RSA_WITH_AES_256_GCM_SHA384","TLS_ECDH_ECDSA_WITH_AES_256_GCM_SHA384","TLS_ECDH_RSA_WITH_AES_256_GCM_SHA384","TLS_DHE_RSA_WITH_AES_256_GCM_SHA384"],"client_auth":"none","force_to_secure":true,"open_external_port":80,"open_port":8080,"path_to_deployment":"/usr/local/openunison/work","path_to_env_file":"/etc/openunison/ou.env","quartz_dir":"/tmp/quartz","secure_external_port":443,"secure_key_alias":"unison-tls","secure_port":8443},"replicas":1,"saml_remote_idp":[{"mapping":{"encryption_cert_alias":"idp-saml2-enc","entity_id":"IDP_ENTITY_ID","logout_url":"IDP_LOGOUT","post_url":"IDP_POST","redirect_url":"IDP_REDIR","signing_cert_alias":"idp-saml2-sig"},"source":{"url":"https://.okta.com/app//sso/saml/metadata"}}],"secret_data":["AD_BIND_PASSWORD","K8S_DB_SECRET","unisonKeystorePassword"],"source_secret":"orchestra-secrets-source"},"status":{"conditions":{"lastTransitionTime":"2023-03-20 08:50:35UTC","status":"True","type":"Completed"},"digest":"bxdq2qQThzkUor2usXi3ilXeqjkN33Q6RrInYTu4/Sk=","idpCertificateFingerprints":{"http://www.okta.com/":"QD8WSqMyA+sbTLlZtRKI+KyIqyeX01z9vMxaEN5Oz5Y="}}}
}
https://10.0.0.1:443/apis/openunison.tremolo.io/v6/namespaces/openunison/openunisons?watch=true&timeoutSeconds=30&allowWatchBookmarks=true&resourceVersion=61036268
No change, skipping
https://10.0.0.1:443/apis/openunison.tremolo.io/v6/namespaces/openunison/openunisons?watch=true&timeoutSeconds=30&allowWatchBookmarks=true&resourceVersion=61036319
https://10.0.0.1:443/apis/openunison.tremolo.io/v6/namespaces/openunison/openunisons?watch=true&timeoutSeconds=30&allowWatchBookmarks=true&resourceVersion=61036319
https://10.0.0.1:443/apis/openunison.tremolo.io/v6/namespaces/openunison/openunisons?watch=true&timeoutSeconds=30&allowWatchBookmarks=true&resourceVersion=61036319
https://10.0.0.1:443/apis/openunison.tremolo.io/v6/namespaces/openunison/openunisons?watch=true&timeoutSeconds=30&allowWatchBookmarks=true&resourceVersion=61036319
https://10.0.0.1:443/apis/openunison.tremolo.io/v6/namespaces/openunison/openunisons?watch=true&timeoutSeconds=30&allowWatchBookmarks=true&resourceVersion=61036319

Log for orchestra

[2023-03-20 20:53:04,327][XNIO-1 task-1] INFO  AccessLog - [AzSuccess] - CheckAlive - https://127.0.0.1:8443/check_alive - uid=Anonymous,o=Tremolo -  [127.0.0.1] - [f63783c0b3e24f1a39f7b54c92588f38e513909fb]
[2023-03-20 20:53:04,336][XNIO-1 task-11] INFO  AccessLog - [Error] - UNKNOWN - https://127.0.0.1:8443/auth/idp/k8sIdp/.well-known/openid-configuration - cn=none - NONE [127.0.0.1] - [f0a969ed8dd634fc8445744d6b26a4f78989a19a7]
[2023-03-20 20:53:04,339][XNIO-1 task-11] ERROR ConfigSys - Could not process request
javax.servlet.ServletException: Unknown URI : /auth/idp/k8sIdp/.well-known/openid-configuration
    at com.tremolosecurity.proxy.auth.AuthMgrSys.doAuthMgr(AuthMgrSys.java:116) ~[unison-server-core-1.0.33.jar:?]
    at com.tremolosecurity.embedd.NextEmbSys.nextSys(NextEmbSys.java:126) ~[unison-server-core-1.0.33.jar:?]
    at com.tremolosecurity.proxy.auth.AzSys.doAz(AzSys.java:89) ~[unison-sdk-1.0.33.jar:?]
    at com.tremolosecurity.embedd.NextEmbSys.nextSys(NextEmbSys.java:111) ~[unison-server-core-1.0.33.jar:?]
    at com.tremolosecurity.proxy.auth.AuthSys.doAuth(AuthSys.java:88) ~[unison-server-core-1.0.33.jar:?]
    at com.tremolosecurity.embedd.NextEmbSys.nextSys(NextEmbSys.java:105) ~[unison-server-core-1.0.33.jar:?]
    at com.tremolosecurity.proxy.ConfigSys.doConfig(ConfigSys.java:269) [unison-server-core-1.0.33.jar:?]
    at com.tremolosecurity.embedd.NextEmbSys.nextSys(NextEmbSys.java:93) [unison-server-core-1.0.33.jar:?]
    at com.tremolosecurity.filter.UnisonServletFilter.doFilter(UnisonServletFilter.java:299) [unison-server-core-1.0.33.jar:?]
    at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:67) [undertow-servlet-2.2.23.Final.jar:2.2.23.Final]
    at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131) [undertow-servlet-2.2.23.Final.jar:2.2.23.Final]
    at io.undertow.servlet.handlers.FilterHandler.handleRequest(FilterHandler.java:84) [undertow-servlet-2.2.23.Final.jar:2.2.23.Final]
    at io.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:62) [undertow-servlet-2.2.23.Final.jar:2.2.23.Final]
    at io.undertow.servlet.handlers.ServletChain$1.handleRequest(ServletChain.java:68) [undertow-servlet-2.2.23.Final.jar:2.2.23.Final]
    at io.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36) [undertow-servlet-2.2.23.Final.jar:2.2.23.Final]
    at io.undertow.servlet.handlers.RedirectDirHandler.handleRequest(RedirectDirHandler.java:68) [undertow-servlet-2.2.23.Final.jar:2.2.23.Final]
    at io.undertow.servlet.handlers.security.SSLInformationAssociationHandler.handleRequest(SSLInformationAssociationHandler.java:117) [undertow-servlet-2.2.23.Final.jar:2.2.23.Final]
    at io.undertow.servlet.handlers.security.ServletAuthenticationCallHandler.handleRequest(ServletAuthenticationCallHandler.java:57) [undertow-servlet-2.2.23.Final.jar:2.2.23.Final]
    at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) [undertow-core-2.2.23.Final.jar:2.2.23.Final]
    at io.undertow.security.handlers.AbstractConfidentialityHandler.handleRequest(AbstractConfidentialityHandler.java:46) [undertow-core-2.2.23.Final.jar:2.2.23.Final]
    at io.undertow.servlet.handlers.security.ServletConfidentialityConstraintHandler.handleRequest(ServletConfidentialityConstraintHandler.java:64) [undertow-servlet-2.2.23.Final.jar:2.2.23.Final]
    at io.undertow.security.handlers.AuthenticationMechanismsHandler.handleRequest(AuthenticationMechanismsHandler.java:60) [undertow-core-2.2.23.Final.jar:2.2.23.Final]
    at io.undertow.servlet.handlers.security.CachedAuthenticatedSessionHandler.handleRequest(CachedAuthenticatedSessionHandler.java:77) [undertow-servlet-2.2.23.Final.jar:2.2.23.Final]
    at io.undertow.security.handlers.AbstractSecurityContextAssociationHandler.handleRequest(AbstractSecurityContextAssociationHandler.java:43) [undertow-core-2.2.23.Final.jar:2.2.23.Final]
    at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) [undertow-core-2.2.23.Final.jar:2.2.23.Final]
    at io.undertow.servlet.handlers.SendErrorPageHandler.handleRequest(SendErrorPageHandler.java:52) [undertow-servlet-2.2.23.Final.jar:2.2.23.Final]
    at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) [undertow-core-2.2.23.Final.jar:2.2.23.Final]
    at io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:275) [undertow-servlet-2.2.23.Final.jar:2.2.23.Final]
    at io.undertow.servlet.handlers.ServletInitialHandler.access$100(ServletInitialHandler.java:79) [undertow-servlet-2.2.23.Final.jar:2.2.23.Final]
    at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:134) [undertow-servlet-2.2.23.Final.jar:2.2.23.Final]
    at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:131) [undertow-servlet-2.2.23.Final.jar:2.2.23.Final]
    at io.undertow.servlet.core.ServletRequestContextThreadSetupAction$1.call(ServletRequestContextThreadSetupAction.java:48) [undertow-servlet-2.2.23.Final.jar:2.2.23.Final]
    at io.undertow.servlet.core.ContextClassLoaderSetupAction$1.call(ContextClassLoaderSetupAction.java:43) [undertow-servlet-2.2.23.Final.jar:2.2.23.Final]
    at io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:255) [undertow-servlet-2.2.23.Final.jar:2.2.23.Final]
    at io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:79) [undertow-servlet-2.2.23.Final.jar:2.2.23.Final]
    at io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:100) [undertow-servlet-2.2.23.Final.jar:2.2.23.Final]
    at io.undertow.server.Connectors.executeRootHandler(Connectors.java:393) [undertow-core-2.2.23.Final.jar:2.2.23.Final]
    at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:852) [undertow-core-2.2.23.Final.jar:2.2.23.Final]
    at org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35) [jboss-threads-2.3.6.Final.jar:2.3.6.Final]
    at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1982) [jboss-threads-2.3.6.Final.jar:2.3.6.Final]
    at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1486) [jboss-threads-2.3.6.Final.jar:2.3.6.Final]
    at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1377) [jboss-threads-2.3.6.Final.jar:2.3.6.Final]
    at org.xnio.XnioWorker$WorkerThreadFactory$1$1.run(XnioWorker.java:1282) [xnio-api-3.8.8.Final.jar:3.8.8.Final]
    at java.lang.Thread.run(Thread.java:829) [?:?]
[2023-03-20 20:53:04,369][XNIO-1 task-11] INFO  AccessLog - [AzSuccess] - CheckAlive - https://127.0.0.1:8443/check_alive - uid=Anonymous,o=Tremolo -  [127.0.0.1] - [f8972cd6580cf38514c5b7435ec3033ab0b84ec9d]
eyupdzhanY1 commented 1 year ago

@mlbiam I have edited logs a bit to remove some identifiers, hope it does not affect debugging.

mlbiam commented 1 year ago

It looks like from the first log everything went through smoothly. The error you're seeing in orchestra:

javax.servlet.ServletException: Unknown URI : /auth/idp/k8sIdp/.well-known/openid-configuration

is to be expected because you haven't deployed the orchesta-login-portal chart yet. What happens when you deploy the last chart?

eyupdzhanY1 commented 1 year ago

I am applying through ouctl

2023/03/21 15:46:43 purge requested for orchestra-login-portal
Waiting a few seconds...
Try #%!i(int=1)
2023/03/21 15:47:02 creating 52 resource(s)
Error installing chart orchestra-login-portal - Internal error occurred: failed calling webhook "applications-openunison.tremolo.io": failed to call webhook: Post "https://openunison-orchestra.openunison.svc:443/k8s/webhooks/v1/applications?timeout=5s": x509: certificate signed by unknown authority, deleting and retrying
2023/03/21 15:47:07 uninstall: Deleting orchestra-login-portal
2023/03/21 15:47:11 Starting delete for "ouhtml-orchestra-login-portal" Service

After 5 tried it fails.

mlbiam commented 1 year ago

you mentioned having ArgoCD, is it still trying to sync? Also, what's the kubernetes distrobution?

eyupdzhanY1 commented 1 year ago

ArgoCD is not trying to sync , 1.25.5

mlbiam commented 1 year ago

what distro? (AKS, kubeadm, EKS, etc)

eyupdzhanY1 commented 1 year ago

Ah sorry, AKS

mlbiam commented 1 year ago

odd. got several production deployments on AKS. something is either keeping the new certs from being applied to the webhooks or is overwriting them. try cleaning everything out:

helm delete orchestra-login-portal -n openunison;helm delete orchestra -n openunison;k delete ns openunison;helm delete kubernetes-dashboard -n kubernetes-dashboard;k delete ns kubernetes-dashboard

and redeploying using ouctl

eyupdzhanY1 commented 1 year ago

Did exactly that , getting same output, i have few deployment running in the cluster, jenkins argocd etc. all point to same ingress and use same tls if it is related to that.

mlbiam commented 1 year ago

this is really strange. try kubectl delete ValidatingWebhookConfiguration openunison-workflow-validation-orchestra and then wait a min and check to see if it gets recreated. If it doesn't, run ouctl again and post the yaml that gets created?

eyupdzhanY1 commented 1 year ago
kubectl delete ValidatingWebhookConfiguration openunison-workflow-validation 
kubectl delete ValidatingWebhookConfiguration openunison-workflow-validation-orchestra

then ouctl fixed the issue, there was old webhook as you suspected , thanks again @mlbiam !