Closed eyupdzhanY1 closed 1 year ago
Probably still an issue in the operator. The operator is responsible for generating certs for openunison and then patching the webhook yaml. My guess is the operator is failing before it hits that point. What you can do is delete the the openunson-operator pod, rerun ouctl, then capture the entire log from the operator (I know there's a ton of output that can be difficult to decipher). Post that output and it should give us the answer.
Is there an option for ouctl to state ingress not to use tls secret of its own ?
ingress:
- ip: 11.11.222.2111
spec:
tls:
- hosts:
- k8sou.com
- k8sdb.com
rules:
- host:
So ingress nginx handles the tls connection
Is there an option for ouctl to state ingress not to use tls secret of its own ?
Yes, in your values.yaml set network.createIngressCertificate
to false
- Here are the TLS customization docs - https://openunison.github.io/knowledgebase/certificates/
Log for operator
Using version 'openunison.tremolo.io/v6'
https://10.0.0.1:443/apis/openunison.tremolo.io/v6/namespaces/openunison/openunisons?watch=true&timeoutSeconds=30&allowWatchBookmarks=true&resourceVersion=61034055
Warning: Nashorn engine is planned to be removed from a future JDK release
Loading Script : '/usr/local/openunison/js/globals.js'
Loading Script : '/usr/local/openunison/js/deploy-upstream-k8s.js'
Loading Script : '/usr/local/openunison/js/deploy-objs.js'
Loading Script : '/usr/local/openunison/js/operator.js'
Loading Script : '/usr/local/openunison/js/deploy-openshift.js'
Loading Script : '/usr/local/openunison/js/helpers.js'
Invoking javascript
in js : {"type":"MODIFIED","object":{"metadata":{"generation":1,"uid":"fc89707c-f553-48ff-8cf1-9835ee610cbb","managedFields":[{"apiVersion":"openunison.tremolo.io\/v6","fieldsV1":{"f:status":{"f:conditions":{"f:lastTransitionTime":{},"f:type":{},"f:status":{},".":{}},"f:digest":{},"f:idpCertificateFingerprints":{"f:http:\/\/www.okta.com\/":{},".":{}},".":{}}},"manager":"Apache-HttpClient","subresource":"status","time":"2023-03-20T20:43:52Z","operation":"Update","fieldsType":"FieldsV1"},{"apiVersion":"openunison.tremolo.io\/v6","fieldsV1":{"f:metadata":{"f:annotations":{"f:meta.helm.sh\/release-namespace":{},"f:meta.helm.sh\/release-name":{},"f:argocd.argoproj.io\/sync-wave":{},"f:helm-update":{},".":{}},"f:labels":{"f:app.kubernetes.io\/instance":{},"f:app.kubernetes.io\/name":{},"f:app.kubernetes.io\/component":{},"f:app.kubernetes.io\/part-of":{},".":{},"f:app.kubernetes.io\/managed-by":{}}},"f:spec":{"f:key_store":{"f:static_keys":{},"f:update_controller":{"f:days_to_expire":{},"f:schedule":{},"f:image":{},".":{}},"f:trusted_certificates":{},".":{},"f:key_pairs":{"f:create_keypair_template":{},"f:keys":{},".":{}}},"f:source_secret":{},"f:deployment_data":{"f:node_selectors":{},"f:tokenrequest_api":{"f:audience":{},"f:enabled":{},"f:expirationSeconds":{},".":{}},"f:readiness_probe_command":{},"f:pull_secret":{},"f:liveness_probe_command":{},".":{}},"f:image":{},"f:dest_secret":{},".":{},"f:replicas":{},"f:hosts":{},"f:enable_activemq":{},"f:non_secret_data":{},"f:activemq_image":{},"f:openunison_network_configuration":{"f:activemq_dir":{},"f:quartz_dir":{},"f:secure_key_alias":{},"f:path_to_env_file":{},"f:secure_port":{},"f:ciphers":{},".":{},"f:allowed_client_names":{},"f:open_port":{},"f:path_to_deployment":{},"f:client_auth":{},"f:open_external_port":{},"f:force_to_secure":{},"f:secure_external_port":{}},"f:secret_data":{},"f:myvd_configmap":{},"f:saml_remote_idp":{}}},"manager":"ouctl","time":"2023-03-20T20:50:28Z","operation":"Update","fieldsType":"FieldsV1"}],"resourceVersion":"61036268","creationTimestamp":"2023-03-20T20:43:44Z","name":"orchestra","namespace":"openunison","annotations":{"helm-update":"Mar 20 22:50:03 2023 EET","meta.helm.sh\/release-name":"orchestra","meta.helm.sh\/release-namespace":"openunison","argocd.argoproj.io\/sync-wave":"20"},"labels":{"app.kubernetes.io\/managed-by":"Helm","app.kubernetes.io\/name":"openunison","app.kubernetes.io\/part-of":"openunison","app.kubernetes.io\/instance":"openunison-orchestra","app.kubernetes.io\/component":"openunison"}},"apiVersion":"openunison.tremolo.io\/v6","kind":"OpenUnison","spec":{"image":"docker.io\/tremolosecurity\/openunison-k8s","source_secret":"orchestra-secrets-source","openunison_network_configuration":{"client_auth":"none","secure_external_port":443,"activemq_dir":"\/tmp\/amq","force_to_secure":true,"quartz_dir":"\/tmp\/quartz","allowed_client_names":[],"secure_port":8443,"secure_key_alias":"unison-tls","open_external_port":80,"path_to_deployment":"\/usr\/local\/openunison\/work","ciphers":["TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384","TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384","TLS_RSA_WITH_AES_256_GCM_SHA384","TLS_ECDH_ECDSA_WITH_AES_256_GCM_SHA384","TLS_ECDH_RSA_WITH_AES_256_GCM_SHA384","TLS_DHE_RSA_WITH_AES_256_GCM_SHA384"],"path_to_env_file":"\/etc\/openunison\/ou.env","open_port":8080},"saml_remote_idp":[{"mapping":{"logout_url":"IDP_LOGOUT","signing_cert_alias":"idp-saml2-sig","post_url":"IDP_POST","encryption_cert_alias":"idp-saml2-enc","entity_id":"IDP_ENTITY_ID","redirect_url":"IDP_REDIR"},"source":{"url":"https:\/\/.okta.com\/app\/\/sso\/saml\/metadata"}}],"hosts":[{"secret_name":"ou-tls-certificate","ingress_name":"openunison","names":[{"env_var":"OU_HOST","name":"k8sou.com"},{"env_var":"K8S_DASHBOARD_HOST","name":"k8sdb.com"}],"annotations":[],"ingress_type":"nginx"}],"replicas":1,"non_secret_data":[{"name":"K8S_URL","value":"https:\/\/vf11-vos-aks-prod-dns-a6211d47.hcp.eastus2.azmk8s.io:443"},{"name":"SESSION_INACTIVITY_TIMEOUT_SECONDS","value":"900"},{"name":"K8S_DASHBOARD_NAMESPACE","value":"kubernetes-dashboard"},{"name":"K8S_DASHBOARD_SERVICE","value":"kubernetes-dashboard"},{"name":"K8S_CLUSTER_NAME","value":"openunison-cp"},{"name":"OPENUNISON_PROVISIONING_ENABLED","value":"false"},{"name":"K8S_IMPERSONATION","value":"false"},{"name":"PROMETHEUS_SERVICE_ACCOUNT","value":"system:serviceaccount:monitoring:prometheus-k8s"},{"name":"OU_SVC_NAME","value":"openunison-orchestra.openunison.svc"},{"name":"K8S_TOKEN_TYPE","value":"legacy"},{"name":"K8S_DB_SSO","value":"saml2"},{"name":"PROMETHEUS_SERVICE_ACCOUNT","value":"system:serviceaccount:monitoring:prometheus-k8s"},{"name":"SHOW_PORTAL_ORGS","value":"false"}],"key_store":{"trusted_certificates"cert"}],"static_keys":[{"name":"session-unison","version":1},{"name":"lastmile-oidc","version":1}],"update_controller":{"days_to_expire":10,"image":"docker.io\/tremolosecurity\/kubernetes-artifact-deployment:1.1.0","schedule":"0 2 * * *"},"key_pairs":{"create_keypair_template":[{"name":"ou","value":"Kubernetes"},{"name":"o","value":""},{"name":"l","value":"Production Cluster"},{"name":"st","value":"East US 2"},{"name":"c","value":"USA"}],"keys":[{"create_data":{"server_name":"openunison-orchestra.openunison.svc","subject_alternative_names":[],"ca_cert":true,"sign_by_k8s_ca":false,"key_size":2048},"name":"unison-tls","import_into_ks":"keypair"},{"create_data":{"server_name":"kubernetes-dashboard.kubernetes-dashboard.svc","subject_alternative_names":[],"secret_info":{"key_name":"dashboard.key","cert_name":"dashboard.crt","type_of_secret":"Opaque"},"ca_cert":true,"delete_pods_labels":["k8s-app=kubernetes-dashboard"],"sign_by_k8s_ca":false,"key_size":2048,"target_namespace":"kubernetes-dashboard"},"replace_if_exists":true,"name":"kubernetes-dashboard","tls_secret_name":"kubernetes-dashboard-certs","import_into_ks":"certificate"},{"create_data":{"server_name":"unison-saml2-rp-sig","subject_alternative_names":[],"ca_cert":true,"sign_by_k8s_ca":false,"key_size":2048},"name":"unison-saml2-rp-sig","import_into_ks":"keypair"},{"create_data":{"server_name":"remote-k8s-idp-sig","subject_alternative_names":[],"ca_cert":false,"sign_by_k8s_ca":false,"key_size":2048},"name":"remote-k8s-idp-sig","import_into_ks":"keypair"}]}},"enable_activemq":false,"dest_secret":"orchestra","activemq_image":"docker.io\/tremolosecurity\/activemq-docker:latest","myvd_configmap":"myvd","deployment_data":{"node_selectors":[],"tokenrequest_api":{"audience":"api","expirationSeconds":600,"enabled":false},"liveness_probe_command":["\/usr\/local\/openunison\/bin\/check_alive.sh","https:\/\/127.0.0.1:8443\/auth\/idp\/k8sIdp\/.well-known\/openid-configuration","issuer","https:\/\/127.0.0.1:8443\/check_alive","alive"],"readiness_probe_command":["\/usr\/local\/openunison\/bin\/check_alive.sh","https:\/\/127.0.0.1:8443\/auth\/idp\/k8sIdp\/.well-known\/openid-configuration","issuer","https:\/\/127.0.0.1:8443\/check_alive","alive"],"pull_secret":""},"secret_data":["AD_BIND_PASSWORD","K8S_DB_SECRET","unisonKeystorePassword"]},"status":{"idpCertificateFingerprints":{"http:\/\/www.okta.com\/":"QD8WSqMyA+sbTLlZtRKI+KyIqyeX01z9vMxaEN5Oz5Y="},"digest":"nMEwJy2o7KZqS72Q3bg1P9yAXmEJ3Bk1eID+1IwpAwk=","conditions":{"lastTransitionTime":"2023-03-20 08:43:51UTC","type":"Completed","status":"True"}}}}
Getting host variable names
Host #0
Name #0
OU_HOST
k8sou.com
Name #1
K8S_DASHBOARD_HOST
k8sdb.com
Done adding host variables
Creating openunison keystore
Storing k8s certificate
Storing trusted certificates
Processing keypairs
Number of keys : '4'
0
Processing key 'unison-tls'
Checking if kubernetes secret exists
Secret exists
Adding existing secret to keystore
Storing to keystore
0
1
Processing key 'kubernetes-dashboard'
Checking if kubernetes secret exists
Secret exists
Adding existing secret to keystore
Storing just the certificate3
1
2
Processing key 'unison-saml2-rp-sig'
Checking if kubernetes secret exists
Secret exists
Adding existing secret to keystore
Storing to keystore
2
3
Processing key 'remote-k8s-idp-sig'
Checking if kubernetes secret exists
Secret exists
Adding existing secret to keystore
Storing to keystore
3
loading static secrets from /api/v1/namespaces/openunison/secrets/orchestra-static-keys
Secret exists, deleting
importing 'lastmile-oidc' from secret
importing 'session-unison' from secret
Checking static key :'session-unison'
import key from secret
Checking static key :'lastmile-oidc'
import key from secret
Posting secret
Remote Identity Providers : [object Object]
Downloading metadata from : https://.okta.com/app//sso/saml/metadata'
XML Metadata :
--------------
<?xml version="1.0" encoding="UTF-8"?><md:EntityDescriptor entityID="http://www.okta.com/" xmlns:md="urn:oasis:names:tc:SAML:2.0:metadata"><md:IDPSSODescriptor WantAuthnRequestsSigned="false" protocolSupportEnumeration="urn:oasis:names:tc:SAML:2.0:protocol"><md:KeyDescriptor use="signing"><ds:KeyInfo xmlns:ds="http://www.w3.org/2000/09/xmldsig#"><ds:X509Data><ds:X509Certificate>MIIDoDCCAoigAwIBAgIGAYb+OcMcMA0GCSqGSIb3DQEBCwUAMIGQMQswCQYDVQQGEwJVUzETMBEG
A1UECAwKQ2FsaWZvcm5pYTEWMBQGA1UEBwwNU2FuIEZyYW5jaXNjbzENMAsGA1UECgwET2t0YTEU
MBIGA1UECwwLU1NPUHJvdmlkZXIxETAPBgNVBAMMCHZlcmlmb25lMRwwGgYJKoZIhvcNAQkBFg1p
bmZvQG9rdGEuY29tMB4XDTIzMDMyMDA4NTMwM1oXDTMzMDMyMDA4NTQwM1owgZAxCzAJBgNVBAYT
AlVTMRMwEQYDVQQIDApDYWxpZm9ybmlhMRYwFAYDVQQHDA1TYW4gRnJhbmNpc2NvMQ0wCwYDVQQK
DARPa3RhMRQwEgYDVQQLDAtTU09Qcm92aWRlcjERMA8GA1UEAwwIdmVyaWZvbmUxHDAaBgkqhkiG
9w0BCQEWDWluZm9Ab2t0IwXRdLSfX3eTyEOoIORttAsjwqS8M3vN
zQLJBGRvTxc1INqetYwhdnUIa4ZtUo28aMTiRSdiJiX9iFho/ICLsM/VOGAjGKQ+0RW8LXvf/Tj3
lD1nP5Qs86e9H1OTsNIeCVsRgpU=</ds:X509Certificate></ds:X509Data></ds:KeyInfo></md:KeyDescriptor><md:NameIDFormat>urn:oasis:names:tc:SAML:1.1:nameid-format:unspecified</md:NameIDFormat><md:NameIDFormat>urn:oasis:names:tc:SAML:1.1:nameid-format:emailAddress</md:NameIDFormat><md:SingleSignOnService Binding="urn:oasis:names:tc:SAML:2.0:bindings:HTTP-POST" Location="https://.okta.com/app/_vosaks_1//sso/saml"/><md:SingleSignOnService Binding="urn:oasis:names:tc:SAML:2.0:bindings:HTTP-Redirect" Location="https://.okta.com/app/_vosaks_1//sso/saml"/></md:IDPSSODescriptor></md:EntityDescriptor>
--------------
Downloaded
Saving fingerprints
{idpCertificateFingerprints=[object Object]}
Importing CACerts
New cacerts generated : java.security.KeyStore@2d834691
DIGEST : EKpEduBSNKBAj7H5k7VC0VAaGcZST+cnJ2+a94wAOC4=
No secret data has changed, not updating the secret
Done
Problem calling '/api/v1/namespaces/openunison/secrets/amq-secrets-orchestra' - 404
{"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"secrets \"amq-secrets-orchestra\" not found","reason":"NotFound","details":{"name":"amq-secrets-orchestra","kind":"secrets"},"code":404}
Obj '/api/v1/namespaces/openunison/secrets/amq-secrets-orchestra' doesn't exist, skipping
Problem calling '/api/v1/namespaces/openunison/secrets/amq-env-secrets-orchestra' - 404
{"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"secrets \"amq-env-secrets-orchestra\" not found","reason":"NotFound","details":{"name":"amq-env-secrets-orchestra","kind":"secrets"},"code":404}
Obj '/api/v1/namespaces/openunison/secrets/amq-env-secrets-orchestra' doesn't exist, skipping
Problem calling '/api/v1/namespaces/openunison/services/amq' - 404
{"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"services \"amq\" not found","reason":"NotFound","details":{"name":"amq","kind":"services"},"code":404}
Obj '/api/v1/namespaces/openunison/services/amq' doesn't exist, skipping
Problem calling '/apis/apps/v1/namespaces/openunison/deployments/amq-orchestra' - 404
{"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"deployments.apps \"amq-orchestra\" not found","reason":"NotFound","details":{"name":"amq-orchestra","group":"apps","kind":"deployments"},"code":404}
Obj '/apis/apps/v1/namespaces/openunison/deployments/amq-orchestra' doesn't exist, skipping
Ingress already exists, not creating
Problem calling '/apis/batch/v1beta1/namespaces/openunison/cronjobs' - 404
{"apiVersion":"batch/v1beta1","kind":"CronJob","metadata":{"labels":{"app":"openunison-orchestra","operated-by":"openunison-operator"},"name":"check-certs-orchestra","namespace":"openunison"},"spec":{"schedule":"0 2 * * *","jobTemplate":{"spec":{"template":{"spec":{"containers":[{"name":"check-certs-orchestra","image":"docker.io/tremolosecurity/kubernetes-artifact-deployment:1.1.0","env":[{"name":"CERT_DAYS_EXPIRE","value":"10"}],"command":["java","-jar","/usr/local/artifactdeploy/artifact-deploy.jar","-extraCertsPath","/etc/extracerts","-installScriptURL","file:///etc/input-maps/cert-check.js","-kubernetesURL","https://kubernetes.default.svc.cluster.local","-rootCaPath","/var/run/secrets/kubernetes.io/serviceaccount/ca.crt","-secretsPath","/etc/input-maps/input.props","-tokenPath","/var/run/secrets/kubernetes.io/serviceaccount/token","-deploymentTemplate","file:///etc/input-maps/deployment.yaml"],"volumeMounts":[{"name":"extra-certs-dir","mountPath":"/etc/extracerts","readOnly":true},{"name":"input-maps","mountPath":"/etc/input-maps","readOnly":true}]}],"restartPolicy":"Never","serviceAccount":"openunison-operator","serviceAccountName":"openunison-operator","volumes":[{"name":"extra-certs-dir","configMap":{"name":"cert-controller-js-orchestra"}},{"name":"input-maps","configMap":{"name":"cert-controller-js-orchestra"}}]}},"backoffLimit":1}}}}
{"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"the server could not find the requested resource","reason":"NotFound","details":{},"code":404}
Problem calling '/apis/batch/v1beta1/namespaces/openunison/cronjobs' - 404
{"apiVersion":"batch/v1beta1","kind":"CronJob","metadata":{"labels":{"app":"openunison-orchestra","operated-by":"openunison-operator"},"name":"check-certs-orchestra","namespace":"openunison"},"spec":{"schedule":"0 2 * * *","jobTemplate":{"spec":{"template":{"spec":{"containers":[{"name":"check-certs-orchestra","image":"docker.io/tremolosecurity/kubernetes-artifact-deployment:1.1.0","env":[{"name":"CERT_DAYS_EXPIRE","value":"10"}],"command":["java","-jar","/usr/local/artifactdeploy/artifact-deploy.jar","-extraCertsPath","/etc/extracerts","-installScriptURL","file:///etc/input-maps/cert-check.js","-kubernetesURL","https://kubernetes.default.svc.cluster.local","-rootCaPath","/var/run/secrets/kubernetes.io/serviceaccount/ca.crt","-secretsPath","/etc/input-maps/input.props","-tokenPath","/var/run/secrets/kubernetes.io/serviceaccount/token","-deploymentTemplate","file:///etc/input-maps/deployment.yaml"],"volumeMounts":[{"name":"extra-certs-dir","mountPath":"/etc/extracerts","readOnly":true},{"name":"input-maps","mountPath":"/etc/input-maps","readOnly":true}]}],"restartPolicy":"Never","serviceAccount":"openunison-operator","serviceAccountName":"openunison-operator","volumes":[{"name":"extra-certs-dir","configMap":{"name":"cert-controller-js-orchestra"}},{"name":"input-maps","configMap":{"name":"cert-controller-js-orchestra"}}]}},"backoffLimit":1}}}}
{"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"the server could not find the requested resource","reason":"NotFound","details":{},"code":404}
looking up '/apis/admissionregistration.k8s.io/v1/validatingwebhookconfigurations/openunison-workflow-validation-orchestra'
{code=200, data={"kind":"ValidatingWebhookConfiguration","apiVersion":"admissionregistration.k8s.io/v1","metadata":{"name":"openunison-workflow-validation-orchestra","uid":"974bf337-71db-48cf-abaf-670c8c4c568b","resourceVersion":"61036274","generation":30,"creationTimestamp":"2023-03-20T14:31:56Z","labels":{"app.kubernetes.io/component":"webhooks","app.kubernetes.io/instance":"openunison-orchestra","app.kubernetes.io/managed-by":"Helm","app.kubernetes.io/name":"openunison","app.kubernetes.io/part-of":"openunison"},"annotations":{"argocd.argoproj.io/sync-wave":"15","meta.helm.sh/release-name":"orchestra","meta.helm.sh/release-namespace":"openunison"},"managedFields":[{"manager":"admissionsenforcer","operation":"Update","apiVersion":"admissionregistration.k8s.io/v1","time":"2023-03-20T20:50:29Z","fieldsType":"FieldsV1","fieldsV1":{"f:webhooks":{"k:{\"name\":\"applications-openunison.tremolo.io\"}":{"f:namespaceSelector":{}},"k:{\"name\":\"authchains-openunison.tremolo.io\"}":{"f:namespaceSelector":{}},"k:{\"name\":\"authmechs-openunison.tremolo.io\"}":{"f:namespaceSelector":{}},"k:{\"name\":\"customazs-openunison.tremolo.io\"}":{"f:namespaceSelector":{}},"k:{\"name\":\"workflows-openunison.tremolo.io\"}":{"f:namespaceSelector":{}}}}},{"manager":"ouctl","operation":"Update","apiVersion":"admissionregistration.k8s.io/v1","time":"2023-03-20T20:50:29Z","fieldsType":"FieldsV1","fieldsV1":{"f:metadata":{"f:annotations":{".":{},"f:argocd.argoproj.io/sync-wave":{},"f:meta.helm.sh/release-name":{},"f:meta.helm.sh/release-namespace":{}},"f:labels":{".":{},"f:app.kubernetes.io/component":{},"f:app.kubernetes.io/instance":{},"f:app.kubernetes.io/managed-by":{},"f:app.kubernetes.io/name":{},"f:app.kubernetes.io/part-of":{}}},"f:webhooks":{".":{},"k:{\"name\":\"applications-openunison.tremolo.io\"}":{".":{},"f:admissionReviewVersions":{},"f:clientConfig":{".":{},"f:caBundle":{},"f:service":{".":{},"f:name":{},"f:namespace":{},"f:path":{},"f:port":{}}},"f:failurePolicy":{},"f:matchPolicy":{},"f:name":{},"f:objectSelector":{},"f:rules":{},"f:sideEffects":{},"f:timeoutSeconds":{}},"k:{\"name\":\"authchains-openunison.tremolo.io\"}":{".":{},"f:admissionReviewVersions":{},"f:clientConfig":{".":{},"f:caBundle":{},"f:service":{".":{},"f:name":{},"f:namespace":{},"f:path":{},"f:port":{}}},"f:failurePolicy":{},"f:matchPolicy":{},"f:name":{},"f:objectSelector":{},"f:rules":{},"f:sideEffects":{},"f:timeoutSeconds":{}},"k:{\"name\":\"authmechs-openunison.tremolo.io\"}":{".":{},"f:admissionReviewVersions":{},"f:clientConfig":{".":{},"f:caBundle":{},"f:service":{".":{},"f:name":{},"f:namespace":{},"f:path":{},"f:port":{}}},"f:failurePolicy":{},"f:matchPolicy":{},"f:name":{},"f:objectSelector":{},"f:rules":{},"f:sideEffects":{},"f:timeoutSeconds":{}},"k:{\"name\":\"customazs-openunison.tremolo.io\"}":{".":{},"f:admissionReviewVersions":{},"f:clientConfig":{".":{},"f:caBundle":{},"f:service":{".":{},"f:name":{},"f:namespace":{},"f:path":{},"f:port":{}}},"f:failurePolicy":{},"f:matchPolicy":{},"f:name":{},"f:objectSelector":{},"f:rules":{},"f:sideEffects":{},"f:timeoutSeconds":{}},"k:{\"name\":\"workflows-openunison.tremolo.io\"}":{".":{},"f:admissionReviewVersions":{},"f:clientConfig":{".":{},"f:caBundle":{},"f:service":{".":{},"f:name":{},"f:namespace":{},"f:path":{},"f:port":{}}},"f:failurePolicy":{},"f:matchPolicy":{},"f:name":{},"f:objectSelector":{},"f:rules":{},"f:sideEffects":{},"f:timeoutSeconds":{}}}}}]},"webhooks":[{"name":"workflows-openunison.tremolo.io","clientConfig":{"service":{"namespace":"openunison","name":"openunison-orchestra","path":"/k8s/webhooks/v1/workflows","port":443},"caBundle":"LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUVGekNDQXYrZ0F3SUJBZ0lHQVhWWU5KTXhNQTBHQ1NxR1NJYjNEUUVCQ3dVQU1JR1hNUkl3RUFZRFZRUUcNCkV3bE5lVU52ZFc1MGNua3hHVEFYQmdOVkJBZ1RFRk4wWVhSbElHOW1JRU5zZFhOMFpYSXhFekFSQmdOVkJBY1QNCkNrMTVJRU5zZFhOMFpYSXhEakFNQmdOVkJBb1RCVTE1VDNKbk1STXdFUVlEVlFRTEV3cExkV0psY201bGRHVnoNCk1Td3dLZ1lEVlFRREV5TnZjR1Z1ZFc1cGMyOXVMVzl5WTJobGMzUnlZUzV2Y0dWdWRXNXBjMjl1TG5OMll6QWUNCkZ3MHlNREV3TWpRd01USTBNREZhRncweU1URXdNalF3TVRJME1ERmFNSUdYTVJJd0VBWURWUVFHRXdsTmVVTnYNCmRXNTBjbmt4R1RBWEJnTlZCQWdURUZOMFlYUmxJRzltSUVOc2RYTjBaWEl4RXpBUkJnTlZCQWNUQ2sxNUlFTnMNCmRYTjBaWEl4RGpBTUJnTlZCQW9UQlUxNVQzSm5NUk13RVFZRFZRUUxFd3BMZFdKbGNtNWxkR1Z6TVN3d0tnWUQNClZRUURFeU52Y0dWdWRXNXBjMjl1TFc5eVkyaGxjM1J5WVM1dmNHVnVkVzVwYzI5dUxuTjJZekNDQVNJd0RRWUoNCktvWklodmNOQVFFQkJRQURnZ0VQQURDQ0FRb0NnZ0VCQUt0WW5Pd05wejhyNXNBQWVDRVcwaUM1WmcyNGoxMDINCjlvcFRMSFE2S3RzK0xVZlI4ekV4VFJncG54V1hIbjRTbitKcHl5d01tdktRU3hmT3lCak1nNmlvcDlNZXQ2b1QNCkhWNTlLb0lzM2t6bitFeHFvd0JoeVRyOGFmL3NZb1pXdTlrcXkxWmErcEgzQXpiSWdIM2JjNllGQkQrak8yWDENCmNtSXdCOXFFbytRU2IvOURQVE1tU0tvbGZCdndha1ZVME1lYUwwbGlYUVNlZG5yNHZvZ20xRWR3Y1hRWkxheXANCkVadUVmTDBySVY1bTJiMzhncWYrcnNTSU9QT0RNVWN2VFpmSnQxczZWK2xTTXA2ZVp6TG9GMlF5MVViOHhBQUkNCldYb3hWOXVMdXFFb1FxZEpKTWI3b0ZGVUUvTmpodkVoV0E0c3ljQ3A5Yk1qVi9YMUp2QXlzQTBDQXdFQUFhTm4NCk1HVXdEd1lEVlIwVEFRSC9CQVV3QXdFQi96QU9CZ05WSFE4QkFmOEVCQU1DQWdRd0VnWURWUjBsQVFIL0JBZ3cNCkJnWUVWUjBsQURBdUJnTlZIUkVFSnpBbGdpTnZjR1Z1ZFc1cGMyOXVMVzl5WTJobGMzUnlZUzV2Y0dWdWRXNXANCmMyOXVMbk4yWXpBTkJna3Foa2lHOXcwQkFRc0ZBQU9DQVFFQWJ6dzlqV09tci9LMmdJdGdpVkd3UGFYRmNJZHMNCjNiamdvVkxYREJkVTFKK3lidm5EVnJEa2owNTlPRzVBOXhRTzEzWVEvbGIreVJNckEwZTdCWEVSR3oxZVlwZmsNCkZoczhCb3QrcENSRVlNMTU3cEtvOUpiN09FTVA4S0FET2xxMFpHYitqN2Zac0NSWlhSbzk5QUpmMms4R0s2WVoNCnFGZE43c1BXUnZXcDRBK2RRbVdPa1RKUVFhdVp5N1lvU1JROUFnSkxvdUxqbzd4QjdLM1hwTXBkNFl0UzBYd1kNClJxV0dreGh6eGtDaWJTZFhrSUFGSkZxc0pZVHFTdHQ4ZXdZaUQyL1Zob3ozcjJjZC8vcXZIWXljL1U5Zm9zM3UNCmR5R1NPWGhnS2pJSTRQblovOS9kemJDWFc0dlFFUVJEaENTTHpmSk5KS3NmMkt2aUNmM3BHa2tHNmc9PQ0KLS0tLS1FTkQgQ0VSVElGSUNBVEUtLS0tLQo="},"rules":[{"operations":["CREATE","UPDATE"],"apiGroups":["openunison.tremolo.io"],"apiVersions":["v1"],"resources":["workflows"],"scope":"Namespaced"}],"failurePolicy":"Fail","matchPolicy":"Equivalent","namespaceSelector":{"matchExpressions":[{"key":"kubernetes.io/metadata.name","operator":"In","values":["openunison"]},{"key":"control-plane","operator":"DoesNotExist"}]},"objectSelector":{},"sideEffects":"None","timeoutSeconds":5,"admissionReviewVersions":["v1"]},{"name":"customazs-openunison.tremolo.io","clientConfig":{"service":{"namespace":"openunison","name":"openunison-orchestra","path":"/k8s/webhooks/v1/customazs","port":443},"caBundle":"LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUVGekNDQXYrZ0F3SUJBZ0lHQVhWWU5KTXhNQTBHQ1NxR1NJYjNEUUVCQ3dVQU1JR1hNUkl3RUFZRFZRUUcNCkV3bE5lVU52ZFc1MGNua3hHVEFYQmdOVkJBZ1RFRk4wWVhSbElHOW1JRU5zZFhOMFpYSXhFekFSQmdOVkJBY1QNCkNrMTVJRU5zZFhOMFpYSXhEakFNQmdOVkJBb1RCVTE1VDNKbk1STXdFUVlEVlFRTEV3cExkV0psY201bGRHVnoNCk1Td3dLZ1lEVlFRREV5TnZjR1Z1ZFc1cGMyOXVMVzl5WTJobGMzUnlZUzV2Y0dWdWRXNXBjMjl1TG5OMll6QWUNCkZ3MHlNREV3TWpRd01USTBNREZhRncweU1URXdNalF3TVRJME1ERmFNSUdYTVJJd0VBWURWUVFHRXdsTmVVTnYNCmRXNTBjbmt4R1RBWEJnTlZCQWdURUZOMFlYUmxJRzltSUVOc2RYTjBaWEl4RXpBUkJnTlZCQWNUQ2sxNUlFTnMNCmRYTjBaWEl4RGpBTUJnTlZCQW9UQlUxNVQzSm5NUk13RVFZRFZRUUxFd3BMZFdKbGNtNWxkR1Z6TVN3d0tnWUQNClZRUURFeU52Y0dWdWRXNXBjMjl1TFc5eVkyaGxjM1J5WVM1dmNHVnVkVzVwYzI5dUxuTjJZekNDQVNJd0RRWUoNCktvWklodmNOQVFFQkJRQURnZ0VQQURDQ0FRb0NnZ0VCQUt0WW5Pd05wejhyNXNBQWVDRVcwaUM1WmcyNGoxMDINCjlvcFRMSFE2S3RzK0xVZlI4ekV4VFJncG54V1hIbjRTbitKcHl5d01tdktRU3hmT3lCak1nNmlvcDlNZXQ2b1QNCkhWNTlLb0lzM2t6bitFeHFvd0JoeVRyOGFmL3NZb1pXdTlrcXkxWmErcEgzQXpiSWdIM2JjNllGQkQrak8yWDENCmNtSXdCOXFFbytRU2IvOURQVE1tU0tvbGZCdndha1ZVME1lYUwwbGlYUVNlZG5yNHZvZ20xRWR3Y1hRWkxheXANCkVadUVmTDBySVY1bTJiMzhncWYrcnNTSU9QT0RNVWN2VFpmSnQxczZWK2xTTXA2ZVp6TG9GMlF5MVViOHhBQUkNCldYb3hWOXVMdXFFb1FxZEpKTWI3b0ZGVUUvTmpodkVoV0E0c3ljQ3A5Yk1qVi9YMUp2QXlzQTBDQXdFQUFhTm4NCk1HVXdEd1lEVlIwVEFRSC9CQVV3QXdFQi96QU9CZ05WSFE4QkFmOEVCQU1DQWdRd0VnWURWUjBsQVFIL0JBZ3cNCkJnWUVWUjBsQURBdUJnTlZIUkVFSnpBbGdpTnZjR1Z1ZFc1cGMyOXVMVzl5WTJobGMzUnlZUzV2Y0dWdWRXNXANCmMyOXVMbk4yWXpBTkJna3Foa2lHOXcwQkFRc0ZBQU9DQVFFQWJ6dzlqV09tci9LMmdJdGdpVkd3UGFYRmNJZHMNCjNiamdvVkxYREJkVTFKK3lidm5EVnJEa2owNTlPRzVBOXhRTzEzWVEvbGIreVJNckEwZTdCWEVSR3oxZVlwZmsNCkZoczhCb3QrcENSRVlNMTU3cEtvOUpiN09FTVA4S0FET2xxMFpHYitqN2Zac0NSWlhSbzk5QUpmMms4R0s2WVoNCnFGZE43c1BXUnZXcDRBK2RRbVdPa1RKUVFhdVp5N1lvU1JROUFnSkxvdUxqbzd4QjdLM1hwTXBkNFl0UzBYd1kNClJxV0dreGh6eGtDaWJTZFhrSUFGSkZxc0pZVHFTdHQ4ZXdZaUQyL1Zob3ozcjJjZC8vcXZIWXljL1U5Zm9zM3UNCmR5R1NPWGhnS2pJSTRQblovOS9kemJDWFc0dlFFUVJEaENTTHpmSk5KS3NmMkt2aUNmM3BHa2tHNmc9PQ0KLS0tLS1FTkQgQ0VSVElGSUNBVEUtLS0tLQo="},"rules":[{"operations":["CREATE","UPDATE"],"apiGroups":["openunison.tremolo.io"],"apiVersions":["v1"],"resources":["customazs"],"scope":"Namespaced"}],"failurePolicy":"Fail","matchPolicy":"Equivalent","namespaceSelector":{"matchExpressions":[{"key":"kubernetes.io/metadata.name","operator":"In","values":["openunison"]},{"key":"control-plane","operator":"DoesNotExist"}]},"objectSelector":{},"sideEffects":"None","timeoutSeconds":5,"admissionReviewVersions":["v1"]},{"name":"authmechs-openunison.tremolo.io","clientConfig":{"service":{"namespace":"openunison","name":"openunison-orchestra","path":"/k8s/webhooks/v1/authmechs","port":443},"caBundle":"LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUVGekNDQXYrZ0F3SUJBZ0lHQVhWWU5KTXhNQTBHQ1NxR1NJYjNEUUVCQ3dVQU1JR1hNUkl3RUFZRFZRUUcNCkV3bE5lVU52ZFc1MGNua3hHVEFYQmdOVkJBZ1RFRk4wWVhSbElHOW1JRU5zZFhOMFpYSXhFekFSQmdOVkJBY1QNCkNrMTVJRU5zZFhOMFpYSXhEakFNQmdOVkJBb1RCVTE1VDNKbk1STXdFUVlEVlFRTEV3cExkV0psY201bGRHVnoNCk1Td3dLZ1lEVlFRREV5TnZjR1Z1ZFc1cGMyOXVMVzl5WTJobGMzUnlZUzV2Y0dWdWRXNXBjMjl1TG5OMll6QWUNCkZ3MHlNREV3TWpRd01USTBNREZhRncweU1URXdNalF3TVRJME1ERmFNSUdYTVJJd0VBWURWUVFHRXdsTmVVTnYNCmRXNTBjbmt4R1RBWEJnTlZCQWdURUZOMFlYUmxJRzltSUVOc2RYTjBaWEl4RXpBUkJnTlZCQWNUQ2sxNUlFTnMNCmRYTjBaWEl4RGpBTUJnTlZCQW9UQlUxNVQzSm5NUk13RVFZRFZRUUxFd3BMZFdKbGNtNWxkR1Z6TVN3d0tnWUQNClZRUURFeU52Y0dWdWRXNXBjMjl1TFc5eVkyaGxjM1J5WVM1dmNHVnVkVzVwYzI5dUxuTjJZekNDQVNJd0RRWUoNCktvWklodmNOQVFFQkJRQURnZ0VQQURDQ0FRb0NnZ0VCQUt0WW5Pd05wejhyNXNBQWVDRVcwaUM1WmcyNGoxMDINCjlvcFRMSFE2S3RzK0xVZlI4ekV4VFJncG54V1hIbjRTbitKcHl5d01tdktRU3hmT3lCak1nNmlvcDlNZXQ2b1QNCkhWNTlLb0lzM2t6bitFeHFvd0JoeVRyOGFmL3NZb1pXdTlrcXkxWmErcEgzQXpiSWdIM2JjNllGQkQrak8yWDENCmNtSXdCOXFFbytRU2IvOURQVE1tU0tvbGZCdndha1ZVME1lYUwwbGlYUVNlZG5yNHZvZ20xRWR3Y1hRWkxheXANCkVadUVmTDBySVY1bTJiMzhncWYrcnNTSU9QT0RNVWN2VFpmSnQxczZWK2xTTXA2ZVp6TG9GMlF5MVViOHhBQUkNCldYb3hWOXVMdXFFb1FxZEpKTWI3b0ZGVUUvTmpodkVoV0E0c3ljQ3A5Yk1qVi9YMUp2QXlzQTBDQXdFQUFhTm4NCk1HVXdEd1lEVlIwVEFRSC9CQVV3QXdFQi96QU9CZ05WSFE4QkFmOEVCQU1DQWdRd0VnWURWUjBsQVFIL0JBZ3cNCkJnWUVWUjBsQURBdUJnTlZIUkVFSnpBbGdpTnZjR1Z1ZFc1cGMyOXVMVzl5WTJobGMzUnlZUzV2Y0dWdWRXNXANCmMyOXVMbk4yWXpBTkJna3Foa2lHOXcwQkFRc0ZBQU9DQVFFQWJ6dzlqV09tci9LMmdJdGdpVkd3UGFYRmNJZHMNCjNiamdvVkxYREJkVTFKK3lidm5EVnJEa2owNTlPRzVBOXhRTzEzWVEvbGIreVJNckEwZTdCWEVSR3oxZVlwZmsNCkZoczhCb3QrcENSRVlNMTU3cEtvOUpiN09FTVA4S0FET2xxMFpHYitqN2Zac0NSWlhSbzk5QUpmMms4R0s2WVoNCnFGZE43c1BXUnZXcDRBK2RRbVdPa1RKUVFhdVp5N1lvU1JROUFnSkxvdUxqbzd4QjdLM1hwTXBkNFl0UzBYd1kNClJxV0dreGh6eGtDaWJTZFhrSUFGSkZxc0pZVHFTdHQ4ZXdZaUQyL1Zob3ozcjJjZC8vcXZIWXljL1U5Zm9zM3UNCmR5R1NPWGhnS2pJSTRQblovOS9kemJDWFc0dlFFUVJEaENTTHpmSk5KS3NmMkt2aUNmM3BHa2tHNmc9PQ0KLS0tLS1FTkQgQ0VSVElGSUNBVEUtLS0tLQo="},"rules":[{"operations":["CREATE","UPDATE"],"apiGroups":["openunison.tremolo.io"],"apiVersions":["v1"],"resources":["authmechs"],"scope":"Namespaced"}],"failurePolicy":"Fail","matchPolicy":"Equivalent","namespaceSelector":{"matchExpressions":[{"key":"kubernetes.io/metadata.name","operator":"In","values":["openunison"]},{"key":"control-plane","operator":"DoesNotExist"}]},"objectSelector":{},"sideEffects":"None","timeoutSeconds":5,"admissionReviewVersions":["v1"]},{"name":"authchains-openunison.tremolo.io","clientConfig":{"service":{"namespace":"openunison","name":"openunison-orchestra","path":"/k8s/webhooks/v1/authchains","port":443},"caBundle":"LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUVGekNDQXYrZ0F3SUJBZ0lHQVhWWU5KTXhNQTBHQ1NxR1NJYjNEUUVCQ3dVQU1JR1hNUkl3RUFZRFZRUUcNCkV3bE5lVU52ZFc1MGNua3hHVEFYQmdOVkJBZ1RFRk4wWVhSbElHOW1JRU5zZFhOMFpYSXhFekFSQmdOVkJBY1QNCkNrMTVJRU5zZFhOMFpYSXhEakFNQmdOVkJBb1RCVTE1VDNKbk1STXdFUVlEVlFRTEV3cExkV0psY201bGRHVnoNCk1Td3dLZ1lEVlFRREV5TnZjR1Z1ZFc1cGMyOXVMVzl5WTJobGMzUnlZUzV2Y0dWdWRXNXBjMjl1TG5OMll6QWUNCkZ3MHlNREV3TWpRd01USTBNREZhRncweU1URXdNalF3TVRJME1ERmFNSUdYTVJJd0VBWURWUVFHRXdsTmVVTnYNCmRXNTBjbmt4R1RBWEJnTlZCQWdURUZOMFlYUmxJRzltSUVOc2RYTjBaWEl4RXpBUkJnTlZCQWNUQ2sxNUlFTnMNCmRYTjBaWEl4RGpBTUJnTlZCQW9UQlUxNVQzSm5NUk13RVFZRFZRUUxFd3BMZFdKbGNtNWxkR1Z6TVN3d0tnWUQNClZRUURFeU52Y0dWdWRXNXBjMjl1TFc5eVkyaGxjM1J5WVM1dmNHVnVkVzVwYzI5dUxuTjJZekNDQVNJd0RRWUoNCktvWklodmNOQVFFQkJRQURnZ0VQQURDQ0FRb0NnZ0VCQUt0WW5Pd05wejhyNXNBQWVDRVcwaUM1WmcyNGoxMDINCjlvcFRMSFE2S3RzK0xVZlI4ekV4VFJncG54V1hIbjRTbitKcHl5d01tdktRU3hmT3lCak1nNmlvcDlNZXQ2b1QNCkhWNTlLb0lzM2t6bitFeHFvd0JoeVRyOGFmL3NZb1pXdTlrcXkxWmErcEgzQXpiSWdIM2JjNllGQkQrak8yWDENCmNtSXdCOXFFbytRU2IvOURQVE1tU0tvbGZCdndha1ZVME1lYUwwbGlYUVNlZG5yNHZvZ20xRWR3Y1hRWkxheXANCkVadUVmTDBySVY1bTJiMzhncWYrcnNTSU9QT0RNVWN2VFpmSnQxczZWK2xTTXA2ZVp6TG9GMlF5MVViOHhBQUkNCldYb3hWOXVMdXFFb1FxZEpKTWI3b0ZGVUUvTmpodkVoV0E0c3ljQ3A5Yk1qVi9YMUp2QXlzQTBDQXdFQUFhTm4NCk1HVXdEd1lEVlIwVEFRSC9CQVV3QXdFQi96QU9CZ05WSFE4QkFmOEVCQU1DQWdRd0VnWURWUjBsQVFIL0JBZ3cNCkJnWUVWUjBsQURBdUJnTlZIUkVFSnpBbGdpTnZjR1Z1ZFc1cGMyOXVMVzl5WTJobGMzUnlZUzV2Y0dWdWRXNXANCmMyOXVMbk4yWXpBTkJna3Foa2lHOXcwQkFRc0ZBQU9DQVFFQWJ6dzlqV09tci9LMmdJdGdpVkd3UGFYRmNJZHMNCjNiamdvVkxYREJkVTFKK3lidm5EVnJEa2owNTlPRzVBOXhRTzEzWVEvbGIreVJNckEwZTdCWEVSR3oxZVlwZmsNCkZoczhCb3QrcENSRVlNMTU3cEtvOUpiN09FTVA4S0FET2xxMFpHYitqN2Zac0NSWlhSbzk5QUpmMms4R0s2WVoNCnFGZE43c1BXUnZXcDRBK2RRbVdPa1RKUVFhdVp5N1lvU1JROUFnSkxvdUxqbzd4QjdLM1hwTXBkNFl0UzBYd1kNClJxV0dreGh6eGtDaWJTZFhrSUFGSkZxc0pZVHFTdHQ4ZXdZaUQyL1Zob3ozcjJjZC8vcXZIWXljL1U5Zm9zM3UNCmR5R1NPWGhnS2pJSTRQblovOS9kemJDWFc0dlFFUVJEaENTTHpmSk5KS3NmMkt2aUNmM3BHa2tHNmc9PQ0KLS0tLS1FTkQgQ0VSVElGSUNBVEUtLS0tLQo="},"rules":[{"operations":["CREATE","UPDATE"],"apiGroups":["openunison.tremolo.io"],"apiVersions":["v1"],"resources":["authchains"],"scope":"Namespaced"}],"failurePolicy":"Fail","matchPolicy":"Equivalent","namespaceSelector":{"matchExpressions":[{"key":"kubernetes.io/metadata.name","operator":"In","values":["openunison"]},{"key":"control-plane","operator":"DoesNotExist"}]},"objectSelector":{},"sideEffects":"None","timeoutSeconds":5,"admissionReviewVersions":["v1"]},{"name":"applications-openunison.tremolo.io","clientConfig":{"service":{"namespace":"openunison","name":"openunison-orchestra","path":"/k8s/webhooks/v1/applications","port":443},"caBundle":"LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUVGekNDQXYrZ0F3SUJBZ0lHQVhWWU5KTXhNQTBHQ1NxR1NJYjNEUUVCQ3dVQU1JR1hNUkl3RUFZRFZRUUcNCkV3bE5lVU52ZFc1MGNua3hHVEFYQmdOVkJBZ1RFRk4wWVhSbElHOW1JRU5zZFhOMFpYSXhFekFSQmdOVkJBY1QNCkNrMTVJRU5zZFhOMFpYSXhEakFNQmdOVkJBb1RCVTE1VDNKbk1STXdFUVlEVlFRTEV3cExkV0psY201bGRHVnoNCk1Td3dLZ1lEVlFRREV5TnZjR1Z1ZFc1cGMyOXVMVzl5WTJobGMzUnlZUzV2Y0dWdWRXNXBjMjl1TG5OMll6QWUNCkZ3MHlNREV3TWpRd01USTBNREZhRncweU1URXdNalF3TVRJME1ERmFNSUdYTVJJd0VBWURWUVFHRXdsTmVVTnYNCmRXNTBjbmt4R1RBWEJnTlZCQWdURUZOMFlYUmxJRzltSUVOc2RYTjBaWEl4RXpBUkJnTlZCQWNUQ2sxNUlFTnMNCmRYTjBaWEl4RGpBTUJnTlZCQW9UQlUxNVQzSm5NUk13RVFZRFZRUUxFd3BMZFdKbGNtNWxkR1Z6TVN3d0tnWUQNClZRUURFeU52Y0dWdWRXNXBjMjl1TFc5eVkyaGxjM1J5WVM1dmNHVnVkVzVwYzI5dUxuTjJZekNDQVNJd0RRWUoNCktvWklodmNOQVFFQkJRQURnZ0VQQURDQ0FRb0NnZ0VCQUt0WW5Pd05wejhyNXNBQWVDRVcwaUM1WmcyNGoxMDINCjlvcFRMSFE2S3RzK0xVZlI4ekV4VFJncG54V1hIbjRTbitKcHl5d01tdktRU3hmT3lCak1nNmlvcDlNZXQ2b1QNCkhWNTlLb0lzM2t6bitFeHFvd0JoeVRyOGFmL3NZb1pXdTlrcXkxWmErcEgzQXpiSWdIM2JjNllGQkQrak8yWDENCmNtSXdCOXFFbytRU2IvOURQVE1tU0tvbGZCdndha1ZVME1lYUwwbGlYUVNlZG5yNHZvZ20xRWR3Y1hRWkxheXANCkVadUVmTDBySVY1bTJiMzhncWYrcnNTSU9QT0RNVWN2VFpmSnQxczZWK2xTTXA2ZVp6TG9GMlF5MVViOHhBQUkNCldYb3hWOXVMdXFFb1FxZEpKTWI3b0ZGVUUvTmpodkVoV0E0c3ljQ3A5Yk1qVi9YMUp2QXlzQTBDQXdFQUFhTm4NCk1HVXdEd1lEVlIwVEFRSC9CQVV3QXdFQi96QU9CZ05WSFE4QkFmOEVCQU1DQWdRd0VnWURWUjBsQVFIL0JBZ3cNCkJnWUVWUjBsQURBdUJnTlZIUkVFSnpBbGdpTnZjR1Z1ZFc1cGMyOXVMVzl5WTJobGMzUnlZUzV2Y0dWdWRXNXANCmMyOXVMbk4yWXpBTkJna3Foa2lHOXcwQkFRc0ZBQU9DQVFFQWJ6dzlqV09tci9LMmdJdGdpVkd3UGFYRmNJZHMNCjNiamdvVkxYREJkVTFKK3lidm5EVnJEa2owNTlPRzVBOXhRTzEzWVEvbGIreVJNckEwZTdCWEVSR3oxZVlwZmsNCkZoczhCb3QrcENSRVlNMTU3cEtvOUpiN09FTVA4S0FET2xxMFpHYitqN2Zac0NSWlhSbzk5QUpmMms4R0s2WVoNCnFGZE43c1BXUnZXcDRBK2RRbVdPa1RKUVFhdVp5N1lvU1JROUFnSkxvdUxqbzd4QjdLM1hwTXBkNFl0UzBYd1kNClJxV0dreGh6eGtDaWJTZFhrSUFGSkZxc0pZVHFTdHQ4ZXdZaUQyL1Zob3ozcjJjZC8vcXZIWXljL1U5Zm9zM3UNCmR5R1NPWGhnS2pJSTRQblovOS9kemJDWFc0dlFFUVJEaENTTHpmSk5KS3NmMkt2aUNmM3BHa2tHNmc9PQ0KLS0tLS1FTkQgQ0VSVElGSUNBVEUtLS0tLQo="},"rules":[{"operations":["CREATE","UPDATE"],"apiGroups":["openunison.tremolo.io"],"apiVersions":["v1"],"resources":["applications"],"scope":"Namespaced"}],"failurePolicy":"Fail","matchPolicy":"Equivalent","namespaceSelector":{"matchExpressions":[{"key":"kubernetes.io/metadata.name","operator":"In","values":["openunison"]},{"key":"control-plane","operator":"DoesNotExist"}]},"objectSelector":{},"sideEffects":"None","timeoutSeconds":5,"admissionReviewVersions":["v1"]}]}
}
need to update the webhook
Done invoking javascript
Checking if need to create a status for : 'MODIFIED'
Generating status
Creating status patch : {"idpCertificateFingerprints":{"http:\/\/www.okta.com\/":"QD8WSqMyA+sbTLlZtRKI+KyIqyeX01z9vMxaEN5Oz5Y="},"digest":"bxdq2qQThzkUor2usXi3ilXeqjkN33Q6RrInYTu4\/Sk=","conditions":{"lastTransitionTime":"2023-03-20 08:50:35UTC","type":"Completed","status":"True"}}
Patching to '/apis/openunison.tremolo.io/v6/namespaces/openunison/openunisons/orchestra/status'
Patch : '{"status":{"idpCertificateFingerprints":{"http:\/\/www.okta.com\/":"QD8WSqMyA+sbTLlZtRKI+KyIqyeX01z9vMxaEN5Oz5Y="},"digest":"bxdq2qQThzkUor2usXi3ilXeqjkN33Q6RrInYTu4\/Sk=","conditions":{"lastTransitionTime":"2023-03-20 08:50:35UTC","type":"Completed","status":"True"}}}'
{code=200, data={"apiVersion":"openunison.tremolo.io/v6","kind":"OpenUnison","metadata":{"annotations":{"argocd.argoproj.io/sync-wave":"20","helm-update":"Mar 20 22:50:03 2023 EET","meta.helm.sh/release-name":"orchestra","meta.helm.sh/release-namespace":"openunison"},"creationTimestamp":"2023-03-20T20:43:44Z","generation":1,"labels":{"app.kubernetes.io/component":"openunison","app.kubernetes.io/instance":"openunison-orchestra","app.kubernetes.io/managed-by":"Helm","app.kubernetes.io/name":"openunison","app.kubernetes.io/part-of":"openunison"},"managedFields":[{"apiVersion":"openunison.tremolo.io/v6","fieldsType":"FieldsV1","fieldsV1":{"f:metadata":{"f:annotations":{".":{},"f:argocd.argoproj.io/sync-wave":{},"f:helm-update":{},"f:meta.helm.sh/release-name":{},"f:meta.helm.sh/release-namespace":{}},"f:labels":{".":{},"f:app.kubernetes.io/component":{},"f:app.kubernetes.io/instance":{},"f:app.kubernetes.io/managed-by":{},"f:app.kubernetes.io/name":{},"f:app.kubernetes.io/part-of":{}}},"f:spec":{".":{},"f:activemq_image":{},"f:deployment_data":{".":{},"f:liveness_probe_command":{},"f:node_selectors":{},"f:pull_secret":{},"f:readiness_probe_command":{},"f:tokenrequest_api":{".":{},"f:audience":{},"f:enabled":{},"f:expirationSeconds":{}}},"f:dest_secret":{},"f:enable_activemq":{},"f:hosts":{},"f:image":{},"f:key_store":{".":{},"f:key_pairs":{".":{},"f:create_keypair_template":{},"f:keys":{}},"f:static_keys":{},"f:trusted_certificates":{},"f:update_controller":{".":{},"f:days_to_expire":{},"f:image":{},"f:schedule":{}}},"f:myvd_configmap":{},"f:non_secret_data":{},"f:openunison_network_configuration":{".":{},"f:activemq_dir":{},"f:allowed_client_names":{},"f:ciphers":{},"f:client_auth":{},"f:force_to_secure":{},"f:open_external_port":{},"f:open_port":{},"f:path_to_deployment":{},"f:path_to_env_file":{},"f:quartz_dir":{},"f:secure_external_port":{},"f:secure_key_alias":{},"f:secure_port":{}},"f:replicas":{},"f:saml_remote_idp":{},"f:secret_data":{},"f:source_secret":{}}},"manager":"ouctl","operation":"Update","time":"2023-03-20T20:50:28Z"},{"apiVersion":"openunison.tremolo.io/v6","fieldsType":"FieldsV1","fieldsV1":{"f:status":{".":{},"f:conditions":{".":{},"f:lastTransitionTime":{},"f:status":{},"f:type":{}},"f:digest":{},"f:idpCertificateFingerprints":{".":{},"f:http://www.okta.com/":{}}}},"manager":"Apache-HttpClient","operation":"Update","subresource":"status","time":"2023-03-20T20:50:35Z"}],"name":"orchestra","namespace":"openunison","resourceVersion":"61036319","uid":"fc89707c-f553-48ff-8cf1-9835ee610cbb"},"spec":{"activemq_image":"docker.io/tremolosecurity/activemq-docker:latest","deployment_data":{"liveness_probe_command":["/usr/local/openunison/bin/check_alive.sh","https://127.0.0.1:8443/auth/idp/k8sIdp/.well-known/openid-configuration","issuer","https://127.0.0.1:8443/check_alive","alive"],"node_selectors":[],"pull_secret":"","readiness_probe_command":["/usr/local/openunison/bin/check_alive.sh","https://127.0.0.1:8443/auth/idp/k8sIdp/.well-known/openid-configuration","issuer","https://127.0.0.1:8443/check_alive","alive"],"tokenrequest_api":{"audience":"api","enabled":false,"expirationSeconds":600}},"dest_secret":"orchestra","enable_activemq":false,"hosts":[{"annotations":[],"ingress_name":"openunison","ingress_type":"nginx","names":[{"env_var":"OU_HOST","name":"k8sou.com"},{"env_var":"K8S_DASHBOARD_HOST","name":"k8sdb.com"}],"secret_name":"ou-tls-certificate"}],"image":"docker.io/tremolosecurity/openunison-k8s","key_store":{"key_pairs":{"create_keypair_template":[{"name":"ou","value":"Kubernetes"},{"name":"o","value":""},{"name":"l","value":"Production Cluster"},{"name":"st","value":"East US 2"},{"name":"c","value":"USA"}],"keys":[{"create_data":{"ca_cert":true,"key_size":2048,"server_name":"openunison-orchestra.openunison.svc","sign_by_k8s_ca":false,"subject_alternative_names":[]},"import_into_ks":"keypair","name":"unison-tls"},{"create_data":{"ca_cert":true,"delete_pods_labels":["k8s-app=kubernetes-dashboard"],"key_size":2048,"secret_info":{"cert_name":"dashboard.crt","key_name":"dashboard.key","type_of_secret":"Opaque"},"server_name":"kubernetes-dashboard.kubernetes-dashboard.svc","sign_by_k8s_ca":false,"subject_alternative_names":[],"target_namespace":"kubernetes-dashboard"},"import_into_ks":"certificate","name":"kubernetes-dashboard","replace_if_exists":true,"tls_secret_name":"kubernetes-dashboard-certs"},{"create_data":{"ca_cert":true,"key_size":2048,"server_name":"unison-saml2-rp-sig","sign_by_k8s_ca":false,"subject_alternative_names":[]},"import_into_ks":"keypair","name":"unison-saml2-rp-sig"},{"create_data":{"ca_cert":false,"key_size":2048,"server_name":"remote-k8s-idp-sig","sign_by_k8s_ca":false,"subject_alternative_names":[]},"import_into_ks":"keypair","name":"remote-k8s-idp-sig"}]},"static_keys":[{"name":"session-unison","version":1},{"name":"lastmile-oidc","version":1}],"trusted_certificates":[{"name":"okta{"days_to_expire":10,"image":"docker.io/tremolosecurity/kubernetes-artifact-deployment:1.1.0","schedule":"0 2 * * *"}},"myvd_configmap":"myvd","non_secret_data":[{"name":"K8S_URL","value":"https://vf11-vos-aks-prod-dns-a6211d47.hcp.eastus2.azmk8s.io:443"},{"name":"SESSION_INACTIVITY_TIMEOUT_SECONDS","value":"900"},{"name":"K8S_DASHBOARD_NAMESPACE","value":"kubernetes-dashboard"},{"name":"K8S_DASHBOARD_SERVICE","value":"kubernetes-dashboard"},{"name":"K8S_CLUSTER_NAME","value":"openunison-cp"},{"name":"OPENUNISON_PROVISIONING_ENABLED","value":"false"},{"name":"K8S_IMPERSONATION","value":"false"},{"name":"PROMETHEUS_SERVICE_ACCOUNT","value":"system:serviceaccount:monitoring:prometheus-k8s"},{"name":"OU_SVC_NAME","value":"openunison-orchestra.openunison.svc"},{"name":"K8S_TOKEN_TYPE","value":"legacy"},{"name":"K8S_DB_SSO","value":"saml2"},{"name":"PROMETHEUS_SERVICE_ACCOUNT","value":"system:serviceaccount:monitoring:prometheus-k8s"},{"name":"SHOW_PORTAL_ORGS","value":"false"}],"openunison_network_configuration":{"activemq_dir":"/tmp/amq","allowed_client_names":[],"ciphers":["TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384","TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384","TLS_RSA_WITH_AES_256_GCM_SHA384","TLS_ECDH_ECDSA_WITH_AES_256_GCM_SHA384","TLS_ECDH_RSA_WITH_AES_256_GCM_SHA384","TLS_DHE_RSA_WITH_AES_256_GCM_SHA384"],"client_auth":"none","force_to_secure":true,"open_external_port":80,"open_port":8080,"path_to_deployment":"/usr/local/openunison/work","path_to_env_file":"/etc/openunison/ou.env","quartz_dir":"/tmp/quartz","secure_external_port":443,"secure_key_alias":"unison-tls","secure_port":8443},"replicas":1,"saml_remote_idp":[{"mapping":{"encryption_cert_alias":"idp-saml2-enc","entity_id":"IDP_ENTITY_ID","logout_url":"IDP_LOGOUT","post_url":"IDP_POST","redirect_url":"IDP_REDIR","signing_cert_alias":"idp-saml2-sig"},"source":{"url":"https://.okta.com/app//sso/saml/metadata"}}],"secret_data":["AD_BIND_PASSWORD","K8S_DB_SECRET","unisonKeystorePassword"],"source_secret":"orchestra-secrets-source"},"status":{"conditions":{"lastTransitionTime":"2023-03-20 08:50:35UTC","status":"True","type":"Completed"},"digest":"bxdq2qQThzkUor2usXi3ilXeqjkN33Q6RrInYTu4/Sk=","idpCertificateFingerprints":{"http://www.okta.com/":"QD8WSqMyA+sbTLlZtRKI+KyIqyeX01z9vMxaEN5Oz5Y="}}}
}
https://10.0.0.1:443/apis/openunison.tremolo.io/v6/namespaces/openunison/openunisons?watch=true&timeoutSeconds=30&allowWatchBookmarks=true&resourceVersion=61036268
No change, skipping
https://10.0.0.1:443/apis/openunison.tremolo.io/v6/namespaces/openunison/openunisons?watch=true&timeoutSeconds=30&allowWatchBookmarks=true&resourceVersion=61036319
https://10.0.0.1:443/apis/openunison.tremolo.io/v6/namespaces/openunison/openunisons?watch=true&timeoutSeconds=30&allowWatchBookmarks=true&resourceVersion=61036319
https://10.0.0.1:443/apis/openunison.tremolo.io/v6/namespaces/openunison/openunisons?watch=true&timeoutSeconds=30&allowWatchBookmarks=true&resourceVersion=61036319
https://10.0.0.1:443/apis/openunison.tremolo.io/v6/namespaces/openunison/openunisons?watch=true&timeoutSeconds=30&allowWatchBookmarks=true&resourceVersion=61036319
https://10.0.0.1:443/apis/openunison.tremolo.io/v6/namespaces/openunison/openunisons?watch=true&timeoutSeconds=30&allowWatchBookmarks=true&resourceVersion=61036319
Log for orchestra
[2023-03-20 20:53:04,327][XNIO-1 task-1] INFO AccessLog - [AzSuccess] - CheckAlive - https://127.0.0.1:8443/check_alive - uid=Anonymous,o=Tremolo - [127.0.0.1] - [f63783c0b3e24f1a39f7b54c92588f38e513909fb]
[2023-03-20 20:53:04,336][XNIO-1 task-11] INFO AccessLog - [Error] - UNKNOWN - https://127.0.0.1:8443/auth/idp/k8sIdp/.well-known/openid-configuration - cn=none - NONE [127.0.0.1] - [f0a969ed8dd634fc8445744d6b26a4f78989a19a7]
[2023-03-20 20:53:04,339][XNIO-1 task-11] ERROR ConfigSys - Could not process request
javax.servlet.ServletException: Unknown URI : /auth/idp/k8sIdp/.well-known/openid-configuration
at com.tremolosecurity.proxy.auth.AuthMgrSys.doAuthMgr(AuthMgrSys.java:116) ~[unison-server-core-1.0.33.jar:?]
at com.tremolosecurity.embedd.NextEmbSys.nextSys(NextEmbSys.java:126) ~[unison-server-core-1.0.33.jar:?]
at com.tremolosecurity.proxy.auth.AzSys.doAz(AzSys.java:89) ~[unison-sdk-1.0.33.jar:?]
at com.tremolosecurity.embedd.NextEmbSys.nextSys(NextEmbSys.java:111) ~[unison-server-core-1.0.33.jar:?]
at com.tremolosecurity.proxy.auth.AuthSys.doAuth(AuthSys.java:88) ~[unison-server-core-1.0.33.jar:?]
at com.tremolosecurity.embedd.NextEmbSys.nextSys(NextEmbSys.java:105) ~[unison-server-core-1.0.33.jar:?]
at com.tremolosecurity.proxy.ConfigSys.doConfig(ConfigSys.java:269) [unison-server-core-1.0.33.jar:?]
at com.tremolosecurity.embedd.NextEmbSys.nextSys(NextEmbSys.java:93) [unison-server-core-1.0.33.jar:?]
at com.tremolosecurity.filter.UnisonServletFilter.doFilter(UnisonServletFilter.java:299) [unison-server-core-1.0.33.jar:?]
at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:67) [undertow-servlet-2.2.23.Final.jar:2.2.23.Final]
at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131) [undertow-servlet-2.2.23.Final.jar:2.2.23.Final]
at io.undertow.servlet.handlers.FilterHandler.handleRequest(FilterHandler.java:84) [undertow-servlet-2.2.23.Final.jar:2.2.23.Final]
at io.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:62) [undertow-servlet-2.2.23.Final.jar:2.2.23.Final]
at io.undertow.servlet.handlers.ServletChain$1.handleRequest(ServletChain.java:68) [undertow-servlet-2.2.23.Final.jar:2.2.23.Final]
at io.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36) [undertow-servlet-2.2.23.Final.jar:2.2.23.Final]
at io.undertow.servlet.handlers.RedirectDirHandler.handleRequest(RedirectDirHandler.java:68) [undertow-servlet-2.2.23.Final.jar:2.2.23.Final]
at io.undertow.servlet.handlers.security.SSLInformationAssociationHandler.handleRequest(SSLInformationAssociationHandler.java:117) [undertow-servlet-2.2.23.Final.jar:2.2.23.Final]
at io.undertow.servlet.handlers.security.ServletAuthenticationCallHandler.handleRequest(ServletAuthenticationCallHandler.java:57) [undertow-servlet-2.2.23.Final.jar:2.2.23.Final]
at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) [undertow-core-2.2.23.Final.jar:2.2.23.Final]
at io.undertow.security.handlers.AbstractConfidentialityHandler.handleRequest(AbstractConfidentialityHandler.java:46) [undertow-core-2.2.23.Final.jar:2.2.23.Final]
at io.undertow.servlet.handlers.security.ServletConfidentialityConstraintHandler.handleRequest(ServletConfidentialityConstraintHandler.java:64) [undertow-servlet-2.2.23.Final.jar:2.2.23.Final]
at io.undertow.security.handlers.AuthenticationMechanismsHandler.handleRequest(AuthenticationMechanismsHandler.java:60) [undertow-core-2.2.23.Final.jar:2.2.23.Final]
at io.undertow.servlet.handlers.security.CachedAuthenticatedSessionHandler.handleRequest(CachedAuthenticatedSessionHandler.java:77) [undertow-servlet-2.2.23.Final.jar:2.2.23.Final]
at io.undertow.security.handlers.AbstractSecurityContextAssociationHandler.handleRequest(AbstractSecurityContextAssociationHandler.java:43) [undertow-core-2.2.23.Final.jar:2.2.23.Final]
at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) [undertow-core-2.2.23.Final.jar:2.2.23.Final]
at io.undertow.servlet.handlers.SendErrorPageHandler.handleRequest(SendErrorPageHandler.java:52) [undertow-servlet-2.2.23.Final.jar:2.2.23.Final]
at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) [undertow-core-2.2.23.Final.jar:2.2.23.Final]
at io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:275) [undertow-servlet-2.2.23.Final.jar:2.2.23.Final]
at io.undertow.servlet.handlers.ServletInitialHandler.access$100(ServletInitialHandler.java:79) [undertow-servlet-2.2.23.Final.jar:2.2.23.Final]
at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:134) [undertow-servlet-2.2.23.Final.jar:2.2.23.Final]
at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:131) [undertow-servlet-2.2.23.Final.jar:2.2.23.Final]
at io.undertow.servlet.core.ServletRequestContextThreadSetupAction$1.call(ServletRequestContextThreadSetupAction.java:48) [undertow-servlet-2.2.23.Final.jar:2.2.23.Final]
at io.undertow.servlet.core.ContextClassLoaderSetupAction$1.call(ContextClassLoaderSetupAction.java:43) [undertow-servlet-2.2.23.Final.jar:2.2.23.Final]
at io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:255) [undertow-servlet-2.2.23.Final.jar:2.2.23.Final]
at io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:79) [undertow-servlet-2.2.23.Final.jar:2.2.23.Final]
at io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:100) [undertow-servlet-2.2.23.Final.jar:2.2.23.Final]
at io.undertow.server.Connectors.executeRootHandler(Connectors.java:393) [undertow-core-2.2.23.Final.jar:2.2.23.Final]
at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:852) [undertow-core-2.2.23.Final.jar:2.2.23.Final]
at org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35) [jboss-threads-2.3.6.Final.jar:2.3.6.Final]
at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1982) [jboss-threads-2.3.6.Final.jar:2.3.6.Final]
at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1486) [jboss-threads-2.3.6.Final.jar:2.3.6.Final]
at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1377) [jboss-threads-2.3.6.Final.jar:2.3.6.Final]
at org.xnio.XnioWorker$WorkerThreadFactory$1$1.run(XnioWorker.java:1282) [xnio-api-3.8.8.Final.jar:3.8.8.Final]
at java.lang.Thread.run(Thread.java:829) [?:?]
[2023-03-20 20:53:04,369][XNIO-1 task-11] INFO AccessLog - [AzSuccess] - CheckAlive - https://127.0.0.1:8443/check_alive - uid=Anonymous,o=Tremolo - [127.0.0.1] - [f8972cd6580cf38514c5b7435ec3033ab0b84ec9d]
@mlbiam I have edited logs a bit to remove some identifiers, hope it does not affect debugging.
It looks like from the first log everything went through smoothly. The error you're seeing in orchestra:
javax.servlet.ServletException: Unknown URI : /auth/idp/k8sIdp/.well-known/openid-configuration
is to be expected because you haven't deployed the orchesta-login-portal
chart yet. What happens when you deploy the last chart?
I am applying through ouctl
2023/03/21 15:46:43 purge requested for orchestra-login-portal
Waiting a few seconds...
Try #%!i(int=1)
2023/03/21 15:47:02 creating 52 resource(s)
Error installing chart orchestra-login-portal - Internal error occurred: failed calling webhook "applications-openunison.tremolo.io": failed to call webhook: Post "https://openunison-orchestra.openunison.svc:443/k8s/webhooks/v1/applications?timeout=5s": x509: certificate signed by unknown authority, deleting and retrying
2023/03/21 15:47:07 uninstall: Deleting orchestra-login-portal
2023/03/21 15:47:11 Starting delete for "ouhtml-orchestra-login-portal" Service
After 5 tried it fails.
you mentioned having ArgoCD, is it still trying to sync? Also, what's the kubernetes distrobution?
ArgoCD is not trying to sync , 1.25.5
what distro? (AKS, kubeadm, EKS, etc)
Ah sorry, AKS
odd. got several production deployments on AKS. something is either keeping the new certs from being applied to the webhooks or is overwriting them. try cleaning everything out:
helm delete orchestra-login-portal -n openunison;helm delete orchestra -n openunison;k delete ns openunison;helm delete kubernetes-dashboard -n kubernetes-dashboard;k delete ns kubernetes-dashboard
and redeploying using ouctl
Did exactly that , getting same output, i have few deployment running in the cluster, jenkins argocd etc. all point to same ingress and use same tls if it is related to that.
this is really strange. try kubectl delete ValidatingWebhookConfiguration openunison-workflow-validation-orchestra
and then wait a min and check to see if it gets recreated. If it doesn't, run ouctl again and post the yaml that gets created?
kubectl delete ValidatingWebhookConfiguration openunison-workflow-validation
kubectl delete ValidatingWebhookConfiguration openunison-workflow-validation-orchestra
then ouctl fixed the issue, there was old webhook as you suspected , thanks again @mlbiam !
I tested with ouctl and helm