search

OpenUnison / openunison-k8s

Access portal for Kubernetes
Apache License 2.0
93 stars 5 forks source link

Exception in thread "main" java.io.IOException: Invalid keystore format #92

Open Mihai-CMM opened 7 months ago

Mihai-CMM commented 7 months ago

Hello, I am trying to install openunison on top of k0s . looks like whatever parameters i chose i always hit this issue Exception in thread "main" java.io.IOException: Invalid keystore format

Can you please advice what should i do to have openunison working with LDAP connector (i dont have any oidc flags on on the k0s ) I am using nginx ingress controller whee i terminate TLS with a valid and trusted certificate

 cat /home/openunison/secret.yaml
apiVersion: v1
type: Opaque
metadata:
  name: orchestra-secrets-source
  namespace: openunison
data:
  K8S_DB_SECRET: WinSuc....
  unisonKeystorePassword: WinSu...
  AD_BIND_PASSWORD: WinSuc.......
kind: Secret

network:
  openunison_host: "openunison-in.k8s.test"
  dashboard_host: "dashboard-in.k8s.test"
  api_server_host: "k8smasters-in.k8s.test"
  k8s_url: https://k8smasters-in.k8s.test:6443
  session_inactivity_timeout_seconds: 900
  createIngressCertificate: false
  force_redirect_to_tls: false
  ingress_type: nginx
  ingress_annotations: {}

cert_template:
  ou: "Kubernetes"
  o: "MyOrg"
  l: "My Cluster"
  st: "State of Cluster"
  c: "MyCountry"

myvd_config_path: "WEB-INF/myvd.conf"
k8s_cluster_name: datalake-in-sit
enable_impersonation: true
impersonation:
  use_jetstack: true
  explicit_certificate_trust: true

dashboard:
  namespace: "kubernetes-dashboard"
  cert_name: "kubernetes-dashboard-certs"
  label: "kube-app=kubernetes-dashboard"
  service_name: kubernetes-dashboard
  require_session: true
certs:
  use_k8s_cm: false

trusted_certs: []
#trusted_certs:
#  - name: ldaps
#    pem_b64: blabla

monitoring:
  prometheus_service_account: system:serviceaccount:monitoring:prometheus-k8s

# Uncomment one of the below options for authentication

active_directory:
  base: "DC=ad,DC=redacted,DC=redacted"
  host: "10.192.yy.xx"
 # port: "636"  for TLS
  port: "389"
 # bind_dn: "cn=Administrator,cn=users,dc=ent2k12,dc=domain,dc=com"
  bind_dn: "CN=k8s_serviceaccount,OU=Domain-Service-Accounts,DC=ad,DC=redacted,DC=redacted"
  con_type: ldap
  srv_dns: "false"

#oidc:
#  client_id: xxxxxx
#  issuer: https://xxxxxx.okta.com/
#  user_in_idtoken: false
#  domain: ""
#  scopes: openid email profile groups
#  claims:
#    sub: sub
#    email: email
#    given_name: given_name
#    family_name: family_name
#    display_name: name
#    groups: groups

#github:
#  client_id: d85d77c55a08c9bcbb15
#  teams: TremoloSecurity/

#saml:
#  idp_url: "https://portal.apps.tremolo.io/idp-test/metadata/dfbe4040-cd32-470e-a9b6-809c8f857c40"

network_policies:
  enabled: false
  ingress:
    enabled: true
    labels:
      app.kubernetes.io/name: ingress-nginx
  monitoring:
    enabled: false
    labels:
      app.kubernetes.io/name: monitoring
  apiserver:
    enabled: false
    labels:
      app.kubernetes.io/name: kube-system

services:
  enable_tokenrequest: false
  token_request_audience: api
  token_request_expiration_seconds: 600
  node_selectors: []

openunison:
  replicas: 1
  non_secret_data:
    K8S_DB_SSO: activedirectory
    PROMETHEUS_SERVICE_ACCOUNT: system:serviceaccount:monitoring:prometheus-k8s
    SHOW_PORTAL_ORGS: "false"
  secrets: []
  enable_provisioning: false
  use_standard_jit_workflow: true
  #az_groups:[]
  #- CN=k8s-users,CN=Users,DC=ent2k12,DC=domain,DC=com

#myvd_configmap: myvdconfig

# For Namespace as a Service

#database:
#  hibernate_dialect: org.hibernate.dialect.MySQL5InnoDBDialect
#  quartz_dialect: org.quartz.impl.jdbcjobstore.StdJDBCDelegate
#  driver: com.mysql.jdbc.Driver
#  url: jdbc:mysql://mariadb.mariadb.svc.cluster.local:3306/unison
#  user: unison
#  validation: SELECT 1

#smtp:
#  host: blackhole.blackhole.svc.cluster.local
#  port: 1025
#  user: "none"
#  from: donotreply@domain.com
#  tls: false

Then this is the pod crashing log

NAME                                       READY   STATUS              RESTARTS      AGE
kube-oidc-proxy-orchestra-9649777b-mg9gq   0/1     ContainerCreating   0             7m22s
openunison-operator-6ccd5f44d7-ftjzj       1/1     Running             0             7m28s
openunison-orchestra-57546b4bcb-vp7zw      0/1     CrashLoopBackOff    6 (90s ago)   7m22s
openunison-orchestra-68d96b8695-sgds5      0/1     CrashLoopBackOff    6 (95s ago)   7m18s
`

`[2023-12-12 08:50:20,598][main] INFO  OpenUnisonOnUndertow - Loading keystore for Undertow
[2023-12-12 08:50:20,599][main] INFO  OpenUnisonOnUndertow - OpenUnison XML File : '/usr/local/openunison/work/webapp/WEB-INF/unison.xml'
[2023-12-12 08:50:20,607][main] INFO  OpenUnisonConfigLoader - No config from include files, using original
[2023-12-12 08:50:20,911][main] INFO  OpenUnisonOnUndertow - Loading keystore : '/etc/openunison/unisonKeyStore.p12'
[2023-12-12 08:50:20,911][main] INFO  OpenUnisonOnUndertow - Building Undertow
[2023-12-12 08:50:20,926][main] INFO  OpenUnisonOnUndertow - Check if enabling HTTP2 - false
[2023-12-12 08:50:20,926][main] INFO  OpenUnisonOnUndertow - Enabling HTTP2
[2023-12-12 08:50:20,927][main] INFO  OpenUnisonOnUndertow - Adding open port : '8080'
Exception in thread "main" java.io.IOException: Invalid keystore format
        at java.base/com.sun.crypto.provider.JceKeyStore.engineLoad(JceKeyStore.java:725)
        at java.base/java.security.KeyStore.load(KeyStore.java:1479)
        at com.tremolosecurity.openunison.undertow.OpenUnisonOnUndertow.setupTlsListener(OpenUnisonOnUndertow.java:533)
        at com.tremolosecurity.openunison.undertow.OpenUnisonOnUndertow.main(OpenUnisonOnUndertow.java:280)

`

mlbiam commented 7 months ago

What are the logs from the openunison-operator-6ccd5f44d7-ftjzj say?

Mihai-CMM commented 7 months ago 
command: operator
url: https://kubernetes.default.svc
namespace: openunison
path to  token: /var/run/secrets/kubernetes.io/serviceaccount/token
path to certificate: /var/run/secrets/kubernetes.io/serviceaccount/ca.crt
versions: 2,3,4,5,6
webhooks to update: /
Testing version 6
URL: https://kubernetes.default.svc/apis/openunison.tremolo.io/v6/namespaces/openunison/openunisons
Watch URL: https://kubernetes.default.svc/apis/openunison.tremolo.io/v6/namespaces/openunison/openunisons
Processing {"apiVersion":"openunison.tremolo.io/v6","items":[],"kind":"OpenUnisonList","metadata":{"continue":"","resourceVersion":"4513071"}}

Watching https://kubernetes.default.svc/apis/openunison.tremolo.io/v6/namespaces/openunison/openunisons?watch=true&timeoutSeconds=10&allowWatchBookmarks=true
Watching https://kubernetes.default.svc/apis/openunison.tremolo.io/v6/namespaces/openunison/openunisons?watch=true&timeoutSeconds=10&allowWatchBookmarks=true
Watching https://kubernetes.default.svc/apis/openunison.tremolo.io/v6/namespaces/openunison/openunisons?watch=true&timeoutSeconds=10&allowWatchBookmarks=true
Watching https://kubernetes.default.svc/apis/openunison.tremolo.io/v6/namespaces/openunison/openunisons?watch=true&timeoutSeconds=10&allowWatchBookmarks=true
Watching https://kubernetes.default.svc/apis/openunison.tremolo.io/v6/namespaces/openunison/openunisons?watch=true&timeoutSeconds=10&allowWatchBookmarks=true
mlbiam commented 7 months ago

odd, there should be some output. what method did you use for install? ouctl?

Mihai-CMM commented 7 months ago

ouctl yes

Mihai-CMM commented 7 months ago

Ok redeployed and i see this: (secret yaml content above)

hecking static key lastmile-oidc the static key doesn't exist in the secret, create it Creating a new Secret Problem patching secret - 201 / ..............................................

QMEAgEFAAQgV+7prN0tVPLHI3p1pdrsSurqVikq6c0Si4iHGFjQWoYEFE6DS9oJfds3SFEzTtC9y\/NZbrpcAgInEA=="},"kind":"Secret","type":"Opqaue"} Starting webhook check, looking up /apis/admissionregistration.k8s.io/v1/validatingwebhookconfigurations/openunison-workflow-validation-orchestra Webhook needs to be udpated Webhook successfully patched Patched /apis/apps/v1/namespaces/openunison/deployments/openunison-orchestra {"status":{"digest":"mVI9SZ1oHMf329smTv\/vu\/J6FJS3b58xHTQ3+bKVpEM=","conditions":{"lastTransitionTime":"2023-12-12T13:47:17.469Z","type":"Completed","status":"True"}}} Resource patched

mlbiam commented 7 months ago

odd, what version of Kubernetes and what distrobution (ie kubeadmin, EKS, etc)? Also, list out the Secrets in the openunison namespace?

Mihai-CMM commented 7 months ago

k0s is the distro, https://docs.k0sproject.io/

Client Version: v1.28.0 Kustomize Version: v5.0.4-0.20230601165947-6ce0bf390ce3 Server Version: v1.28.3+k0s

k0s version v1.28.3+k0s.0

kubectl get secret -n openunison NAME TYPE DATA AGE orchestra Opqaue 4 8m55s orchestra-secrets-source Opaque 3 9m7s orchestra-static-keys Opqaue 2 8m55s remote-k8s-idp-sig kubernetes.io/tls 2 8m56s sh.helm.release.v1.openunison.v1 helm.sh/release.v1 1 9m6s sh.helm.release.v1.orchestra.v1 helm.sh/release.v1 1 9m1s unison-saml2-rp-sig kubernetes.io/tls 2 8m56s unison-tls kubernetes.io/tls 2 8m57s

Thanks again Marc

mlbiam commented 7 months ago

everything looks OK there. I don't think this would do it but openunison.non_secret_data.K8S_DB_SSO must be saml2 or oidc. this is internal to openunison, so the fact you're using LDAP doesn't matter. Try setting it to oidc and redeploying?

If that doesn't do it, can you run kubectl get openunison orchestra -n openunison -o yaml and set the contents here?

Mihai-CMM commented 7 months ago

Failed the same with saml2

apiVersion: openunison.tremolo.io/v6
kind: OpenUnison
metadata:
  annotations:
    argocd.argoproj.io/sync-wave: "20"
    helm-update: Dec 12 15:38:45 2023 CET
    meta.helm.sh/release-name: orchestra
    meta.helm.sh/release-namespace: openunison
  creationTimestamp: "2023-12-12T14:38:48Z"
  generation: 1
  labels:
    app.kubernetes.io/component: openunison
    app.kubernetes.io/instance: openunison-orchestra
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/name: openunison
    app.kubernetes.io/part-of: openunison
  name: orchestra
  namespace: openunison
  resourceVersion: "4534933"
  uid: db300a68-c1de-45e8-9641-fb1b561c8383
spec:
  activemq_image: ghcr.io/tremolosecurity/activemq-docker:5.16.6
  deployment_data:
    liveness_probe_command:
    - /usr/local/openunison/bin/check_alive.sh
    - https://127.0.0.1:8443/auth/idp/k8sIdp/.well-known/openid-configuration
    - issuer
    - https://127.0.0.1:8443/check_alive
    - alive
    node_selectors: []
    pull_secret: ""
    readiness_probe_command:
    - /usr/local/openunison/bin/check_alive.sh
    - https://127.0.0.1:8443/auth/idp/k8sIdp/.well-known/openid-configuration
    - issuer
    - https://127.0.0.1:8443/check_alive
    - alive
    tokenrequest_api:
      audience: api
      enabled: false
      expirationSeconds: 600
  dest_secret: orchestra
  enable_activemq: false
  hosts:
  - annotations: []
    ingress_name: openunison
    ingress_type: nginx
    names:
    - env_var: OU_HOST
      name: openunison-in.k8s.test
    - env_var: K8S_DASHBOARD_HOST
      name: dashboard-in.k8s.test
    - env_var: K8S_API_HOST
      name: k8smasters-in.k8s.test
      service_name: kube-oidc-proxy-orchestra
    secret_name: ou-tls-certificate
  image: ghcr.io/openunison/openunison-k8s:1.0.38
  key_store:
    key_pairs:
      create_keypair_template:
      - name: ou
        value: Kubernetes
      - name: o
        value: MyOrg
      - name: l
        value: My Cluster
      - name: st
        value: State of Cluster
      - name: c
        value: MyCountry
      keys:
      - create_data:
          ca_cert: true
          key_size: 2048
          server_name: openunison-orchestra.openunison.svc
          sign_by_k8s_ca: false
          subject_alternative_names:
          - k8smasters-in.k8s.test
        import_into_ks: keypair
        name: unison-tls
      - create_data:
          ca_cert: true
          delete_pods_labels:
          - kube-app=kubernetes-dashboard
          key_size: 2048
          secret_info:
            cert_name: dashboard.crt
            key_name: dashboard.key
            type_of_secret: Opaque
          server_name: kubernetes-dashboard.kubernetes-dashboard.svc
          sign_by_k8s_ca: false
          subject_alternative_names: []
          target_namespace: kubernetes-dashboard
        import_into_ks: certificate
        name: kubernetes-dashboard
        replace_if_exists: true
        tls_secret_name: kubernetes-dashboard-certs
      - create_data:
          ca_cert: true
          key_size: 2048
          server_name: unison-saml2-rp-sig
          sign_by_k8s_ca: false
          subject_alternative_names: []
        import_into_ks: keypair
        name: unison-saml2-rp-sig
      - create_data:
          ca_cert: false
          key_size: 2048
          server_name: remote-k8s-idp-sig
          sign_by_k8s_ca: false
          subject_alternative_names: []
        import_into_ks: keypair
        name: remote-k8s-idp-sig
    static_keys:
    - name: session-unison
      version: 1
    - name: lastmile-oidc
      version: 1
    trusted_certificates: []
    update_controller:
      days_to_expire: 10
      image: ghcr.io/openunison/openunison-kubernetes-operator:1.0.4
      schedule: 0 2 * * *
  myvd_configmap: myvd
  non_secret_data:
  - name: K8S_URL
    value: https://k8smasters-in.k8s.test
  - name: SESSION_INACTIVITY_TIMEOUT_SECONDS
    value: "900"
  - name: K8S_DASHBOARD_NAMESPACE
    value: kubernetes-dashboard
  - name: K8S_DASHBOARD_SERVICE
    value: kubernetes-dashboard
  - name: K8S_CLUSTER_NAME
    value: datalake-in-sit
  - name: OPENUNISON_PROVISIONING_ENABLED
    value: "false"
  - name: K8S_IMPERSONATION
    value: "true"
  - name: PROMETHEUS_SERVICE_ACCOUNT
    value: system:serviceaccount:monitoring:prometheus-k8s
  - name: OU_SVC_NAME
    value: openunison-orchestra.openunison.svc
  - name: K8S_TOKEN_TYPE
    value: legacy
  - name: K8S_DB_SSO
    value: saml2
  - name: PROMETHEUS_SERVICE_ACCOUNT
    value: system:serviceaccount:monitoring:prometheus-k8s
  - name: SHOW_PORTAL_ORGS
    value: "false"
  - name: K8S_OPENUNISON_NS
    value: openunison
  openunison_network_configuration:
    activemq_dir: /tmp/amq
    allowed_client_names: []
    ciphers:
    - TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384
    - TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
    - TLS_RSA_WITH_AES_256_GCM_SHA384
    - TLS_ECDH_ECDSA_WITH_AES_256_GCM_SHA384
    - TLS_ECDH_RSA_WITH_AES_256_GCM_SHA384
    - TLS_DHE_RSA_WITH_AES_256_GCM_SHA384
    client_auth: none
    force_to_secure: false
    open_external_port: 80
    open_port: 8080
    path_to_deployment: /usr/local/openunison/work
    path_to_env_file: /etc/openunison/ou.env
    quartz_dir: /tmp/quartz
    secure_external_port: 443
    secure_key_alias: unison-tls
    secure_port: 8443
  replicas: 1
  secret_data:
  - AD_BIND_PASSWORD
  - K8S_DB_SECRET
  - unisonKeystorePassword
  source_secret: orchestra-secrets-source
status:
  conditions:
    lastTransitionTime: "2023-12-12T14:38:53.080Z"
    status: "True"
    type: Completed
  digest: IT1cWBkQA/9WASEW9pYkGXXmqby9K7GHNkKMBkVgJqQ=
Mihai-CMM commented 7 months ago

I wonder what you expect here:

path to  token: /var/run/secrets/kubernetes.io/serviceaccount/token
path to certificate: /var/run/secrets/kubernetes.io/serviceaccount/ca.crt

k0s sets the kubelet under this path 

# ll /var/lib/k0s/
total 20
drwxr-xr-x  2 root root 4096 Dec  1 12:08 bin
drwx--x--x 12 root root 4096 Dec  1 12:08 containerd
drwxr-xr-x  2 root root    6 Dec  1 12:08 images
drwxr-xr-x  7 root root  142 Dec  1 12:08 kubelet
-rw-r--r--  1 root root 1656 Dec  1 12:08 kubelet-config.yaml
-rw-------  1 root root 1970 Dec  1 12:08 kubelet.conf
drwxr-x--x  2 root root   20 Dec  1 12:08 pki
-rw-r--r--  1 root root 1984 Dec  1 12:08 worker-profile.yaml
mlbiam commented 7 months ago

Odd. This usually happens when the API server gets "confused" about OpenUnison object versions. But that's usually becauae the myvd config isn't loaded. What happens when you run:

kubectl get secret orchestra -n openunison -o json | jq -r '.data["ou.env"]' | base64 -d | grep myvd

Does it come back as MYVD_CONFIG_PATH=/etc/myvd/myvd.conf?

Mihai-CMM commented 7 months ago

kubectl get secret orchestra -n openunison -o json | jq -r '.data["ou.env"]' | base64 -d | grep myvd MYVD_CONFIG_PATH=/etc/myvd/myvd.conf

mlbiam commented 7 months ago

i'll see if i can reproduce. there's nothing really strange going on here but i've never tried on k0s before.

Mihai-CMM commented 7 months ago

thx

mlbiam commented 7 months ago

so i took your values.yaml, deployed it onto k0s with just different host names and ldap connection info and it worked perfectly. Can you please provide the part of the logs from the beginning og the container? tere shouldn't be anything sensitive.

Mihai-CMM commented 7 months ago

Ok : I waited on night and the cronjob was executed i think ok , but the pods are still failing

Error getting SSL certificate "openunison/ou-tls-certificate": local SSL certificate openunison/ou-tls-certificate was not found. Using default certificate

kubectl -n openunison get secret                                               NAME                               TYPE                 DATA   AGE
orchestra                          Opqaue               4      17h
orchestra-secrets-source           Opaque               3      17h
orchestra-static-keys              Opqaue               2      17h
remote-k8s-idp-sig                 kubernetes.io/tls    2      17h
sh.helm.release.v1.openunison.v1   helm.sh/release.v1   1      17h
sh.helm.release.v1.orchestra.v1    helm.sh/release.v1   1      17h
unison-saml2-rp-sig                kubernetes.io/tls    2      17h
unison-tls                         kubernetes.io/tls    2      17h
command: check-certs
url: https://kubernetes.default.svc
namespace: openunison
path to  token: /var/run/secrets/kubernetes.io/serviceaccount/token
path to certificate: /var/run/secrets/kubernetes.io/serviceaccount/ca.crt
versions: 2,3,4,5,6
webhooks to update:
Testing version 6
URL: https://kubernetes.default.svc/apis/openunison.tremolo.io/v6/namespaces/openunison/openunisons
Watch URL: https://kubernetes.default.svc/apis/openunison.tremolo.io/v6/namespaces/openunison/openunisons
Checking certificats in namespace
Checking openunison / orchestra
unison-tls
Secret stored in openunison / unison-tls
Checking key tls.crt
Not expiring
kubernetes-dashboard
Secret stored in kubernetes-dashboard / kubernetes-dashboard-certs
Checking key dashboard.crt
Not expiring
unison-saml2-rp-sig
Secret stored in openunison / unison-saml2-rp-sig
Checking key tls.crt
Not expiring
remote-k8s-idp-sig
Secret stored in openunison / remote-k8s-idp-sig
Checking key tls.crt
Not expiring
NAME                                       READY   STATUS              RESTARTS          AGE
check-certs-orchestra-28373820-49jlp       0/1     Completed           0                 6h44m
kube-oidc-proxy-orchestra-9649777b-tqlsv   0/1     ContainerCreating   0                 17h
openunison-operator-6ccd5f44d7-7xd6m       1/1     Running             0                 17h
openunison-orchestra-57546b4bcb-npdg9      0/1     CrashLoopBackOff    203 (3m28s ago)   17h
openunison-orchestra-6fb4c4cfc4-lq525      0/1     CrashLoopBackOff    203 (2m49s ago)   17h

here is the full log

kubectl -n openunison logs openunison-orchestra-6fb4c4cfc4-lq525
/usr/local/openunison/work/webapp/WEB-INF/lib/*:/usr/local/openunison/work/webapp/WEB-INF/classes:/tmp/quartz
[2023-12-13 07:41:14,417][main] INFO  OpenUnisonOnUndertow - Starting OpenUnison on Undertow 1.0.38-2023120501
[2023-12-13 07:41:14,423][main] INFO  OpenUnisonOnUndertow - Parsing YAML : '/etc/openunison/openunison.yaml'
[2023-12-13 07:41:14,515][main] INFO  OpenUnisonOnUndertow - Config Open Port : '8080'
[2023-12-13 07:41:14,516][main] INFO  OpenUnisonOnUndertow - Disable HTTP2 : 'false'
[2023-12-13 07:41:14,516][main] INFO  OpenUnisonOnUndertow - Allow unescaped characters : 'false'
[2023-12-13 07:41:14,516][main] INFO  OpenUnisonOnUndertow - Config Open External Port : '80'
[2023-12-13 07:41:14,516][main] INFO  OpenUnisonOnUndertow - Config Secure Port : '8443'
[2023-12-13 07:41:14,517][main] INFO  OpenUnisonOnUndertow - Config Secure External Port : '443'
[2023-12-13 07:41:14,517][main] INFO  OpenUnisonOnUndertow - Config Context Root :  '/'
[2023-12-13 07:41:14,517][main] INFO  OpenUnisonOnUndertow - Force to Secure : 'false'
[2023-12-13 07:41:14,517][main] INFO  OpenUnisonOnUndertow - ActiveMQ Directory : '/tmp/amq'
[2023-12-13 07:41:14,518][main] INFO  OpenUnisonOnUndertow - Quartz Directory : '/tmp/quartz'
[2023-12-13 07:41:14,518][main] INFO  OpenUnisonOnUndertow - Config TLS Client Auth Mode : 'none'
[2023-12-13 07:41:14,518][main] INFO  OpenUnisonOnUndertow - Config TLS Allowed Client Subjects : '[]'
[2023-12-13 07:41:14,519][main] INFO  OpenUnisonOnUndertow - Config TLS Protocols : 'null'
[2023-12-13 07:41:14,519][main] INFO  OpenUnisonOnUndertow - Config TLS Ciphers : '[TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384, TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384, TLS_RSA_WITH_AES_256_GCM_SHA384, TLS_ECDH_ECDSA_WITH_AES_256_GCM_SHA384, TLS_ECDH_RSA_WITH_AES_256_GCM_SHA384, TLS_DHE_RSA_WITH_AES_256_GCM_SHA384]'
[2023-12-13 07:41:14,519][main] INFO  OpenUnisonOnUndertow - Config Path to Deployment : '/usr/local/openunison/work'
[2023-12-13 07:41:14,519][main] INFO  OpenUnisonOnUndertow - Config Path to Environment File : '/etc/openunison/ou.env'
[2023-12-13 07:41:14,520][main] INFO  OpenUnisonOnUndertow - Redirect to contex root : 'false'
[2023-12-13 07:41:14,520][main] INFO  OpenUnisonOnUndertow - Support socket shutdown : false
[2023-12-13 07:41:14,523][main] INFO  OpenUnisonOnUndertow - true
[2023-12-13 07:41:14,523][main] INFO  OpenUnisonOnUndertow - Creating unisonServiceProps
[2023-12-13 07:41:14,541][main] INFO  OpenUnisonOnUndertow - Temporary unisonServiceProps : '/tmp/unisonService9418919455708874686props'
[2023-12-13 07:41:14,549][main] INFO  OpenUnisonOnUndertow - Loading environment file : '/etc/openunison/ou.env'
[2023-12-13 07:41:14,550][main] INFO  OpenUnisonOnUndertow - Adding property : 'metadata'
[2023-12-13 07:41:14,550][main] INFO  OpenUnisonOnUndertow - Adding property : 'data'
[2023-12-13 07:41:14,550][main] INFO  OpenUnisonOnUndertow - Adding property : 'K8S_SELF_LINK'
[2023-12-13 07:41:14,550][main] INFO  OpenUnisonOnUndertow - Adding property : 'OU_SVC_NAME'
[2023-12-13 07:41:14,550][main] INFO  OpenUnisonOnUndertow - Adding property : 'type'
[2023-12-13 07:41:14,550][main] INFO  OpenUnisonOnUndertow - Adding property : 'K8S_TOKEN_TYPE'
[2023-12-13 07:41:14,550][main] INFO  OpenUnisonOnUndertow - Adding property : 'K8S_API_HOST'
[2023-12-13 07:41:14,550][main] INFO  OpenUnisonOnUndertow - Adding property : 'OU_HOST'
[2023-12-13 07:41:14,550][main] INFO  OpenUnisonOnUndertow - Adding property : 'AD_BIND_PASSWORD'
[2023-12-13 07:41:14,550][main] INFO  OpenUnisonOnUndertow - Adding property : 'K8S_IMPERSONATION'
[2023-12-13 07:41:14,550][main] INFO  OpenUnisonOnUndertow - Adding property : 'K8S_DASHBOARD_HOST'
[2023-12-13 07:41:14,550][main] INFO  OpenUnisonOnUndertow - Adding property : 'SHOW_PORTAL_ORGS'
[2023-12-13 07:41:14,550][main] INFO  OpenUnisonOnUndertow - Adding property : 'MYVD_CONFIG_PATH'
[2023-12-13 07:41:14,551][main] INFO  OpenUnisonOnUndertow - Adding property : 'K8S_DB_SSO'
[2023-12-13 07:41:14,551][main] INFO  OpenUnisonOnUndertow - Adding property : 'unisonKeystorePassword'
[2023-12-13 07:41:14,551][main] INFO  OpenUnisonOnUndertow - Adding property : 'kind'
[2023-12-13 07:41:14,551][main] INFO  OpenUnisonOnUndertow - Adding property : 'K8S_DB_SECRET'
[2023-12-13 07:41:14,551][main] INFO  OpenUnisonOnUndertow - Adding property : 'K8S_DASHBOARD_NAMESPACE'
[2023-12-13 07:41:14,551][main] INFO  OpenUnisonOnUndertow - Adding property : 'OU_QUARTZ_MASK'
[2023-12-13 07:41:14,551][main] INFO  OpenUnisonOnUndertow - Adding property : 'K8S_OPENUNISON_NS'
[2023-12-13 07:41:14,551][main] INFO  OpenUnisonOnUndertow - Adding property : 'K8S_DASHBOARD_SERVICE'
[2023-12-13 07:41:14,551][main] INFO  OpenUnisonOnUndertow - Adding property : 'PROMETHEUS_SERVICE_ACCOUNT'
[2023-12-13 07:41:14,551][main] INFO  OpenUnisonOnUndertow - Adding property : 'K8S_URL'
[2023-12-13 07:41:14,551][main] INFO  OpenUnisonOnUndertow - Adding property : 'namespace'
[2023-12-13 07:41:14,551][main] INFO  OpenUnisonOnUndertow - Adding property : 'name'
[2023-12-13 07:41:14,551][main] INFO  OpenUnisonOnUndertow - Adding property : 'SESSION_INACTIVITY_TIMEOUT_SECONDS'
[2023-12-13 07:41:14,551][main] INFO  OpenUnisonOnUndertow - Adding property : 'OPENUNISON_PROVISIONING_ENABLED'
[2023-12-13 07:41:14,551][main] INFO  OpenUnisonOnUndertow - Adding property : 'K8S_CLUSTER_NAME'
[2023-12-13 07:41:14,552][main] INFO  OpenUnisonOnUndertow - Loading keystore for Undertow
[2023-12-13 07:41:14,552][main] INFO  OpenUnisonOnUndertow - OpenUnison XML File : '/usr/local/openunison/work/webapp/WEB-INF/unison.xml'
[2023-12-13 07:41:14,560][main] INFO  OpenUnisonConfigLoader - No config from include files, using original
[2023-12-13 07:41:14,863][main] INFO  OpenUnisonOnUndertow - Loading keystore : '/etc/openunison/unisonKeyStore.p12'
[2023-12-13 07:41:14,863][main] INFO  OpenUnisonOnUndertow - Building Undertow
[2023-12-13 07:41:14,876][main] INFO  OpenUnisonOnUndertow - Check if enabling HTTP2 - false
[2023-12-13 07:41:14,876][main] INFO  OpenUnisonOnUndertow - Enabling HTTP2
[2023-12-13 07:41:14,877][main] INFO  OpenUnisonOnUndertow - Adding open port : '8080'
Exception in thread "main" java.io.IOException: Invalid keystore format
        at java.base/com.sun.crypto.provider.JceKeyStore.engineLoad(JceKeyStore.java:725)
        at java.base/java.security.KeyStore.load(KeyStore.java:1479)
        at com.tremolosecurity.openunison.undertow.OpenUnisonOnUndertow.setupTlsListener(OpenUnisonOnUndertow.java:533)
        at com.tremolosecurity.openunison.undertow.OpenUnisonOnUndertow.main(OpenUnisonOnUndertow.java:280)
mlbiam commented 7 months ago

everything looks normal. try setting image in your values.yaml to docker.io/tremolosecurity/betas:1.0.39 and redeploy. you should get an additional log line that starts with Could not create PKCS12 from.... I need that, the stack trace, and the entire preceeding log.

Mihai-CMM commented 7 months ago 
# kubectl -n openunison describe pod  openunison-orchestra-657bc46df5-v5sgw | grep -i image -C2
  openunison-orchestra:
    Container ID:   containerd://8b99c92f2b0db48a4f74f5e1f5341e1f5540c5d6c54114e0f8d02a53c0daf4d7
    Image:          docker.io/tremolosecurity/betas:1.0.39
[2023-12-14 09:11:09,047][main] WARN  OpenUnisonOnUndertow - Could not create PKCS12 from /etc/openunison/unisonKeyStore.p12, falling back to JCEKS
java.io.IOException: keystore password was incorrect
        at java.base/sun.security.pkcs12.PKCS12KeyStore.engineLoad(PKCS12KeyStore.java:2092) ~[?:?]
        at java.base/sun.security.util.KeyStoreDelegator.engineLoad(KeyStoreDelegator.java:222) ~[?:?]
        at java.base/java.security.KeyStore.load(KeyStore.java:1479) ~[?:?]
        at com.tremolosecurity.openunison.undertow.OpenUnisonOnUndertow.setupTlsListener(OpenUnisonOnUndertow.java:530) [openunison-on-undertow-1.0.39.jar:?]
        at com.tremolosecurity.openunison.undertow.OpenUnisonOnUndertow.main(OpenUnisonOnUndertow.java:280) [openunison-on-undertow-1.0.39.jar:?]
Caused by: java.security.UnrecoverableKeyException: failed to decrypt safe contents entry: javax.crypto.BadPaddingException: Given final block not properly padded. Such issues can arise if a bad key is used during decryption.
        ... 5 more
Exception in thread "main" java.io.IOException: Invalid keystore format
        at java.base/com.sun.crypto.provider.JceKeyStore.engineLoad(JceKeyStore.java:725)
        at java.base/java.security.KeyStore.load(KeyStore.java:1479)
        at com.tremolosecurity.openunison.undertow.OpenUnisonOnUndertow.setupTlsListener(OpenUnisonOnUndertow.java:536)
        at com.tremolosecurity.openunison.undertow.OpenUnisonOnUndertow.main(OpenUnisonOnUndertow.java:280)

java.io.IOException: keystore password was incorrect - Which password?

Mihai-CMM commented 7 months ago

Ok, i found it - thx a lot the secret file that is not a k8s should no longer have K8S_DB_SECRET and unisonKeystorePassword.

Can you please update the documentation if you feel necessary on how that secret should look like 

 cat /home/openunison/secret.yaml
apiVersion: v1
type: Opaque
metadata:
  name: orchestra-secrets-source
  namespace: openunison
data:
  K8S_DB_SECRET: WinSuc....
  unisonKeystorePassword: WinSu...
  AD_BIND_PASSWORD: WinSuc.......
kind: Secret

should only be 

 cat /home/openunison/secret.yaml
apiVersion: v1
type: Opaque
metadata:
  name: orchestra-secrets-source
  namespace: openunison
data:
  AD_BIND_PASSWORD: WinSuc.......
kind: Secret

Again thanks a lot

Mihai-CMM commented 7 months ago

Now - myabe I open a new ticket? I dont have the oicd flags enabled on k0s so i chose 

enable_impersonation: true
impersonation:
  use_jetstack: true
  explicit_certificate_trust: true
NAME                                             READY   STATUS              RESTARTS   AGE
kube-oidc-proxy-orchestra-9649777b-hmv8c         0/1     ContainerCreating   0          10m
openunison-operator-6ccd5f44d7-xd6ks             1/1     Running             0          10m
openunison-orchestra-54db86f588-r58kp            1/1     Running             0          10m
ouhtml-orchestra-login-portal-749df6c7d9-c66cr   1/1     Running             0          9m19

Is it normal to have that pod stuck in creating

Later edit: I am authorized by AD but still the pod stuck in creating

mlbiam commented 6 months ago

that's odd. the orchesta-secrets-source Secret is generated by ouctl. did you change the key values after generation? Can you can delete that Secret and ouctl recreate it?

Mihai-CMM commented 6 months ago

When I do the install I use ouctl install-auth-portal -s /home/openunison/secret.yaml /home/openunison/openunison-default.yaml. Now I did the install from scratch and I redeployed openunison. If i have other key: values in the secret except AD_BIND_PASSWORD it fails with error from ticket title. If I install without any secret provided it fails differently. Maybe my install command is wrong?

mysiki commented 6 months ago

Hello, I got the problem, in my case it was password character fault. Using password with special character give me the same error. Change for password with letter and '_' only work. Don't know with character are in fault. I don't take time to read all the thread, I hope this can help :D

Mihai-CMM commented 6 months ago

Ok So i was able to do 3 times the deployments and it worked. I notice though another strange behavior. If dashboard was deployed before open unison there is an issue with untrusted certificate DN (Empty issuer DN not allowed in X509Certificates) Anyway form my point of view this is ok and can be close - Whoever reads this only the secret needs to have a content like this 

apiVersion: v1
type: Opaque
metadata:
  name: orchestra-secrets-source
  namespace: openunison
data:
  AD_BIND_PASSWORD: WinSuc.......
kind: Secret