Closed danielSundsvallSCIT closed 2 years ago
mlbiam
commented
2 years ago First, can you provide your values.yaml?
Is SADC016.domain.se the name of a specific domain controller or is it the name of your domain and you want to get domain controllers based on the domain instead of pointing to a specific controller or load balancer?
danielSundsvallSCIT
commented
2 years ago network:
openunison_host: "k8sou.sundsvall.se"
dashboard_host: "k8sdashboard.sundsvall.se"
api_server_host: "k8s-poc.sundsvall.se"
session_inactivity_timeout_seconds: 900
k8s_url: https://10.197.20.250:6443
force_redirect_to_tls: false
createIngressCertificate: true
ingress_type: nginx
ingress_annotations:
kubernetes.io/ingress.class: nginx
cert_template:
ou: "Kubernetes-dan"
o: POC
l: king
st: "Medelpad"
c: "SE"
image: docker.io/tremolosecurity/openunison-k8s
myvd_config_path: "WEB-INF/myvd.conf"
k8s_cluster_name: kubernetes-poc
enable_impersonation: false
impersonation:
use_jetstack: true
jetstack_oidc_proxy_image: docker.io/tremolosecurity/kube-oidc-proxy:latest
explicit_certificate_trust: true
dashboard:
namespace: "kubernetes-dashboard"
cert_name: "kubernetes-dashboard-certs"
label: "k8s-app=kubernetes-dashboard"
service_name: kubernetes-dashboard
certs:
use_k8s_cm: false
trusted_certs:
- name: intermediate-ca
pem_b64: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUZyakNDQTVhZ0F3SUJBZ0lLWWJ6UHNBQUJBQUFBRHpBTkJna3Foa2lHOXcwQkFRc0ZBREJHTVFzd0NRWUQKVlFRR0V3SlRSVEVhTUJnR0ExVUVDaE1SVTNWdVpITjJZV3hzY3lCcmIyMXRkVzR4R3pBWkJnTlZCQU1URWxOMQpibVJ6ZG1Gc2JDQlNiMjkwUTBFd01UQWVGdzB4TlRFeE1ESXhNRE14TXpkYUZ3MHlOVEV4TURJeE1EUXhNemRhCk1FSXhDekFKQmdOVkJBWVRBbE5GTVJvd0dBWURWUVFLRXhGVGRXNWtjM1poYkd4eklHdHZiVzExYmpFWE1CVUcKQTFVRUF4TU9VR1Z5YzI5dVlXeElkME5oTURFd2dnRWlNQTBHQ1NxR1NJYjNEUUVCQVFVQUE0SUJEd0F3Z2dFSwpBb0lCQVFDcWNpZTdEREt1NkxTaklOR2hxOUhxeHlVZ0tZaUpJN25GOHBUdEwvZDFrd0wyNWN4VUVDNFRqcFdZCnB2NkxBRUNrenEyR1hTcENQSmZUbVA1RUIzVklpY25KVUN4Uy9BVVBEYjJtTElNOWU2ZHZRVVRJSEpqL0hqaEwKZElPYUIyaHRidFNGM05YOERRYVQ5dGEyaDUwNmZ0b21hdm5meGpzbXVhYVlTcWlOMzMzTEpjRFpVZHVqYzBiMgpTUnU3NFRQUFlKZk5DcXhmdHJRakZQbGRMTlFtd3cyaEI2eVUzd3N4SFJ6NCt4QkEzRythMTNSOW1XUDhQSEtRCk1oUExUbCtHa2tTeXBuM1FVZDhTd2dBdU1xWXFBWWRFS1E3K3VoaStsV0tJYVQ0RG1lYU1ibEE3YnE2UjlYQ2EKNTVjSDZmeDFJVURIR3RCUkt6SGNFUDRiTGVrekFnTUJBQUdqZ2dHZ01JSUJuREFRQmdrckJnRUVBWUkzRlFFRQpBd0lCQVRBakJna3JCZ0VFQVlJM0ZRSUVGZ1FVRlZXSTlUZDlHL1lXSXMzTjB2dXM2c01LWE8wd0hRWURWUjBPCkJCWUVGTExUWGZlMUJjdWVuWkFRT0tSajR5TGh3SEdDTUVzR0ExVWRJQVJFTUVJd1FBWUhLb1Z3Z1Q4QkFUQTEKTURNR0NDc0dBUVVGQndJQkZpZG9kSFJ3T2k4dmNHdHBMbk4xYm1SemRtRnNiQzV6WlM5RFVGTjBZWFJsYldWdQpkQzV3WkdZd0dRWUpLd1lCQkFHQ054UUNCQXdlQ2dCVEFIVUFZZ0JEQUVFd0N3WURWUjBQQkFRREFnR0dNQThHCkExVWRFd0VCL3dRRk1BTUJBZjh3SHdZRFZSMGpCQmd3Rm9BVWRhc1dXQTZtL2Y5Y1k3ZGFQM3NjUUxranZGUXcKUkFZRFZSMGZCRDB3T3pBNW9EZWdOWVl6YUhSMGNEb3ZMM0JyYVM1emRXNWtjM1poYkd3dWMyVXZVM1Z1WkhOMgpZV3hzSlRJd1VtOXZkRU5CTURFb01Ta3VZM0pzTUZjR0NDc0dBUVVGQndFQkJFc3dTVEJIQmdnckJnRUZCUWN3CkFvWTdhSFIwY0RvdkwzQnJhUzV6ZFc1a2MzWmhiR3d1YzJVdlUwRkJVekE1T0Y5VGRXNWtjM1poYkd3bE1qQlMKYjI5MFEwRXdNU2d4S1M1amNuUXdEUVlKS29aSWh2Y05BUUVMQlFBRGdnSUJBSEpuZkcxWEtJNTRSQm5aOWdqVgpnem9INmhrRDAzanBNRnA0V1h6K2g1dndLOEhSK29lRHBxbXZ5OFcrL2xZejhvOGROYTBYSmRrV0hwNDFBUDhOCkJESmIyMHNwd2Rsa3FndGVkb1Ixd3pBVXBadXkreWtGTFExc3Q0cUNzN1Vvd3ltTDdKWXpvdXZHbTRaN2hMcEIKYnJhLzl1ZCsvNGR3UXM1a1p2R2ZXeFd6bjdjdTJXZU56enhrMEVqTkVUTldRWFovZHE2MnpNN2hsL01hNUxZMgpTL1MrOVVIbTFTeE52THI1VmlIUWU0ZERvSWg3MG5jSUdsK1dOREpLOUZYS0R2TE1JeFNvYjMxU1FnTXd4Y2RLCm9LY1dyQkhab21rcENaSjJtVDNXbnBxbUlSczBnZjZQS0pZTEdWNHVpSWxOKzBJZkNnT0hnN3orRVZlSXhVS3UKUXpjakhOckM3OGNiWGVramk5emdrbFRoaW5Ga1FiY1k2ZjB5dE9mbHd3QVZ4UHBENjcrb045QUIxNzh5a1h5WApJbzJoTjA1SXhGQ3JLKzdjVWpqTTVUOThzMnRLbVk0MUg3amJVdi9TdW5mekx3cmRscXF1SlYwcmdhSGtzQ3BLCjhxbi8xOEpwNUlhQXNVeGkvck5tWVpodW1oVkM4RWVwL1JnWFhYOUl3UlVVaGpTT3JULzYyajJmRU82VVNWNXgKY29NRjNERDJlWkFueFM0Um5UK1U1UW9UR3p6VzE1dmE0RVRUdUgrSDhoclkxZkJxWTY5QzRMWDFDT3hKeC9MUwpuWCs4VnlNWEw0Q0F4NkJMcTh0QS9veFpSbzdWTUJBdGJVaWR4dmJqSncxV08rUXNXeGt4R09zV2wveDduaktZCjQ0QmtCNU1vRlVxanBBdjVoQnZCOUM4aQotLS0tLUVORCBDRVJUSUZJQ0FURS0tLS0t
- name: sundsvallroot
pem_b64: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUYzVENDQThXZ0F3SUJBZ0lRYkRKb3hXbXFEN0pGQk0wMEJCSTl1akFOQmdrcWhraUc5dzBCQVFzRkFEQkcKTVFzd0NRWURWUVFHRXdKVFJURWFNQmdHQTFVRUNoTVJVM1Z1WkhOMllXeHNjeUJyYjIxdGRXNHhHekFaQmdOVgpCQU1URWxOMWJtUnpkbUZzYkNCU2IyOTBRMEV3TVRBZUZ3MHhOVEV4TURJd056STVNRGRhRncwek5URXhNREl3Ck56TTVNRGRhTUVZeEN6QUpCZ05WQkFZVEFsTkZNUm93R0FZRFZRUUtFeEZUZFc1a2MzWmhiR3h6SUd0dmJXMTEKYmpFYk1Ca0dBMVVFQXhNU1UzVnVaSE4yWVd4c0lGSnZiM1JEUVRBeE1JSUNJakFOQmdrcWhraUc5dzBCQVFFRgpBQU9DQWc4QU1JSUNDZ0tDQWdFQXlPaVAxaVdJRHc5azVVRG51Q3VReC9ObVU4MTZMZVNXZ2NIM2xrYjNUNGFBCkxpbGlVVTJMWWNHOXQ4YUdNa0N0Q2ltTTRyZmU3eVllOFBUbEN1dmZCWm9GdFlwb2hlZlJkTmJiS0JQcGxIN0IKNkNZNDJKZVVNSUVLWnRCOWFkekQ3SXc1dTJIYVFjQUx5TytoUVcxNHVNaXR3SmtMVC9FRXIveXAvVU5XRVVQego2QTRSb25GQVg1SjNISFVEV3BJcXBFdGYyaGJ1NFJ6bmJoS2cvSVFjS3R1MDZBd3VTOE4xNW43T0hQMzV6QVRBCnlEL3cwWFk4d1hzK3dhMzBIcGNlUDdVVXN3VXdWYmxTTE8ydWtXTFAyVGFOcmlDRWlmOGxlNVZ5MHlOeWJVWDcKSWZDNmZBUEt2WXRoS3ozaEJaUGNJNDBtekhUSWJGZ296eitFYkFiVjh5MXY4TTRZOW5EWnB4UTF6TkZVU2kvTwo5SHIydGl5ZHNLeGlLb2gzcjJXODBjWFdDOXgyYW8xUGFBL2xaNkhPR0FGU2xwRUJWL1B5eDRjUFlsK1lSK3k5ClpsV1NxR2xZZXMzL2JzenRNNldYYjZRWCtGdWZ3YzZMRFNpanM4SHJ3ZU9OdGY0UGFrdjlEcXZQWVNCYmFzYXgKbHRHMFh2OFpOWXBDb28rS0xvUk95eDZrZkxnanNJOTVlbUtiZC9qRlFPTnY5TVVYNk45ZVlaR2c4UzhUenF0UwpKTVpxb3hsbUtPOXhiYXlCRTJiQndxWE9yQkdma21taVVFbGUzYWE3WTVzd2RpeER0bDFNcE42NWdkd2tlTHpBCkhvRnlqTFZDdnN3Ykhoc0RQc3RKVkJkVytnbzV5R09TODd5c3AxR2RQM1N4V1Baem5hdm5LcTBGRDZYOUNMTUMKQXdFQUFhT0J4akNCd3pBTEJnTlZIUThFQkFNQ0FZWXdEd1lEVlIwVEFRSC9CQVV3QXdFQi96QWRCZ05WSFE0RQpGZ1FVZGFzV1dBNm0vZjljWTdkYVAzc2NRTGtqdkZRd0VnWUpLd1lCQkFHQ054VUJCQVVDQXdFQUFUQkxCZ05WCkhTQUVSREJDTUVBR0J5cUZjSUUvQVFFd05UQXpCZ2dyQmdFRkJRY0NBUlluYUhSMGNEb3ZMM0JyYVM1emRXNWsKYzNaaGJHd3VjMlV2UTFCVGRHRjBaVzFsYm5RdWNHUm1NQ01HQ1NzR0FRUUJnamNWQWdRV0JCUXFUUHY2UGVUZgpPQzhDK1hNcXhiK1kySHl2d3pBTkJna3Foa2lHOXcwQkFRc0ZBQU9DQWdFQVBvUkx2WXZVWXZmb1FIMmZ4djVaCmpZWnNsSjFYVzZ6T2pMT3AzSjdobEdsV2hwdVlwdGwzek9ZRzN2dEF0RUVOZTFJeG5TeXcwTUV4Vy9UT1NTSHMKcTA1TUgxMlJUZkFSRmhnZVdlV0RGdXZQNFNDUnN4b3F1QXNnTUJmckRJZmxQalZkMUMyUUJscFNCa29lLzBQLwpiV2dEUzJ0OGx2Qmg2Q0Z6MU5lT1BOQzZVdFdNTjF2cEd5ZDROSHB5VFdBWmRoVkFYMnJQR0JzSE5ZSG9MNVRJCmdIQWk4Rk4vQjdJQURySzFGOUV5N244bzVDMW1KVGRZczFxcnVrbVpPY1pWQ0txcEY4UHVPOTdLcGh6bEJMTmgKcjlLdVJlY2FTQjBkRUhjUFM3SFFLQ3FvYmp4RVViaHREQWtXV05pRXZKSlNaRWFvQ2dNWE15TkhHK3lmRHVwTApSRGdqbEdkQWRGUUhwcCszK0s1cVhsUDd5TlUxTmVBK1VqNTZYU1ZEcjhKQ0p2YjBUM093Sk96MFJERElvTTRJCnVxbEo2TWFyZ3Y4azRJd0dNck1YcXF6S1VKc2xBTVQ2T1Z0ZWJWTk4wR0M5Z1pGUE5lTnpMb2Z6U3dkRmJ4R0IKZGRDN0dpNlQzeVhPb0EzMlBSV0RrNmpkMkxqMEhPZkhhdUMzdXJaeHFQMHpYdHUrZGpMdUtWWjVMQUxtb1NaUwp3anNhOXFWUkxYWDhMdDhETJIb1lxdGVnOEpTSHFtUmtpYVB4ZE5hVmNMaWtLalRER1o5ckE2eFR1L2hFUEM2Cmc2TGh2MTBoL3o4bk9jWlRYOWw5Z3d0K2xjdjZvN1M0UEdhYUhrTng5N25SREY4aWVQUlFGNTlER2pTUmxCQWwKSmowNC9aT0loYzA4NDVBcXBDaVMyb0E9Ci0tLS0tRU5EIENFUlRJRklDQVRFLS0tLS0=
monitoring:
prometheus_service_account: system:serviceaccount:monitoring:prometheus-k8s
# Uncomment one of the below options for authentication
active_directory:
base: OU=MIIS,OU=SK,DC=personal,DC=domain,DC=se
host: SADC016.personal.domain.se
port: 636
bind_dn: "CN=Kubernetes_dan,OU=Service accounts,OU=Users,OU=SK,DC=personal,DC=domain,DC=se"
con_type: ldaps
srv_dns: false
#oidc:
# client_id: xxxxxx
# issuer: https://xxxxxx.okta.com/
# user_in_idtoken: false
# domain: ""
# scopes: openid email profile groups
# claims:
# sub: sub
# email: email
# given_name: given_name
# family_name: family_name
# display_name: name
# groups: groups
#github:
# client_id: d85d77c55a08c9bcbb15
# teams: TremoloSecurity/
#saml:
# idp_url: "https://portal.apps.tremolo.io/idp-test/metadata/dfbe4040-cd32-470e-a9b6-809c8f857c40"
network_policies:
enabled: false
ingress:
enabled: true
labels:
app.kubernetes.io/name: ingress-nginx
monitoring:
enabled: true
labels:
app.kubernetes.io/name: monitoring
apiserver:
enabled: false
labels:
app.kubernetes.io/name: kube-system
services:
enable_tokenrequest: false
token_request_audience: api
token_request_expiration_seconds: 600
node_selectors: []
openunison:
replicas: 1
non_secret_data:
K8S_DB_SSO: oidc
PROMETHEUS_SERVICE_ACCOUNT: system:serviceaccount:monitoring:prometheus-k8s
SHOW_PORTAL_ORGS: "false"
secrets: []
html:
image: docker.io/tremolosecurity/openunison-k8s-html
enable_provisioning: false
#az_groups:
#- CN=k8s-users,CN=Users,DC=ent2k12,DC=domain,DC=com
#myvd_configmap: myvdconfig
# For Namespace as a Service
#database:
# hibernate_dialect: org.hibernate.dialect.MySQL5InnoDBDialect
# quartz_dialect: org.quartz.impl.jdbcjobstore.StdJDBCDelegate
# driver: com.mysql.jdbc.Driver
# url: jdbc:mysql://mariadb.mariadb.svc.cluster.local:3306/unison
# user: unison
# validation: SELECT 1
#smtp:
# host: blackhole.blackhole.svc.cluster.local
# port: 1025
# user: "none"
# from: donotreply@domain.com
# tls: falseIs
SADC016.domain.sethe name of a specific domain controller or is it the name of your domain and you want to get domain controllers based on the domain instead of pointing to a specific controller or load balancer?
That is the specifik name of a domain controller.
mlbiam
commented
2 years ago That is the specifik name of a domain controller.
Then it looks like there's a DNS issue. The Caused by: java.net.UnknownHostException: SADC016.domain.se error means that OpenUnison can't get an IP for SADC016.domain.se.
danielSundsvallSCIT
commented
2 years ago Then it looks like there's a DNS issue. The
Caused by: java.net.UnknownHostException: SADC016.domain.seerror means that OpenUnison can't get an IP forSADC016.domain.se.
If I would change from hostname to IP, would I have to upgrade orchestrator-login-portal only or the Orchestrator chart also?
mlbiam
commented
2 years ago both orchestra and orchestra-login-portal would need to be updated
danielSundsvallSCIT
commented
2 years ago both
orchestraandorchestra-login-portalwould need to be updated
Seems that the LDAP error is gone, now I get something else.
[2022-04-27 13:05:26,808][XNIO-1 task-3] INFO AccessLog - SRCH op=3 con=2 base='o=Data' filter='(uid=dan11ost)' scope='2' attribs=''
[2022-04-27 13:05:26,904][XNIO-1 task-3] INFO AccessLog - RESULT op=3 con=2 result=0 time=96
[2022-04-27 13:05:26,909][XNIO-1 task-3] INFO AccessLog - BIND op=4 con=3 dn='CN=dan11ost,OU=KSK,ou=activedirectory,o=Data'
[2022-04-27 13:05:26,914][XNIO-1 task-3] INFO AccessLog - RESULT op=4 con=3 result=0 time=5
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by com.cedarsoftware.util.io.MetaUtils (file:/usr/local/openunison/work/webapp/WEB-INF/lib/json-io-4.13.0.jar) to field java.lang.Enum.name
WARNING: Please consider reporting this to the maintainers of com.cedarsoftware.util.io.MetaUtils
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
[2022-04-27 13:05:27,043][XNIO-1 task-3] INFO AccessLog - SRCH op=5 con=4 base='o=Tremolo' filter='(uid=dan11ost)' scope='2' attribs=''
[2022-04-27 13:05:27,080][XNIO-1 task-3] INFO AccessLog - RESULT op=5 con=4 result=0 time=38
[2022-04-27 13:05:27,091][XNIO-1 task-3] INFO AccessLog - SRCH op=6 con=5 base='uid=danx-49-xx-49-xost,ou=shadow,o=Tremolo' filter='(objectClass=*)' scope='0' attribs=''
[2022-04-27 13:05:27,111][XNIO-1 task-3] INFO AccessLog - RESULT op=6 con=5 result=0 time=20
[2022-04-27 13:05:27,281][XNIO-1 task-3] INFO AccessLog - SRCH op=7 con=6 base='uid=danx-49-xx-49-xost,ou=shadow,o=Tremolo' filter='(objectClass=*)' scope='0' attribs=''
[2022-04-27 13:05:27,303][XNIO-1 task-3] INFO AccessLog - RESULT op=7 con=6 result=0 time=22
[2022-04-27 13:05:27,309][XNIO-1 task-3] INFO AccessLog - SRCH op=8 con=7 base='uid=danx-49-xx-49-xost,ou=shadow,o=Tremolo' filter='(objectClass=*)' scope='0' attribs=''
[2022-04-27 13:05:27,345][XNIO-1 task-3] INFO AccessLog - RESULT op=8 con=7 result=0 time=36
[2022-04-27 13:05:27,350][XNIO-1 task-3] INFO AccessLog - SRCH op=9 con=8 base='uid=danx-49-xx-49-xost,ou=shadow,o=Tremolo' filter='(objectClass=*)' scope='0' attribs=''
[2022-04-27 13:05:27,371][XNIO-1 task-3] INFO AccessLog - RESULT op=9 con=8 result=0 time=21
[2022-04-27 13:05:27,499][XNIO-1 task-3] INFO AccessLog - [AuSuccess] - completelogin - https://k8sou.sundsvall.se/auth/formlogin - uid=danx-49-xx-49-xost,ou=shadow,o=Tremolo - 20 / enterprise-idp [10.197.32.98] - [fa6a0961adae3f8ebb16876411e83f36ec4dbe2cf]
[2022-04-27 13:05:27,545][XNIO-1 task-3] INFO AccessLog - [AzSuccess] - completelogin - https://k8sou.sundsvall.se/login/auth - uid=danx-49-xx-49-xost,ou=shadow,o=Tremolo - NONE [10.197.32.98] - [fa6a0961adae3f8ebb16876411e83f36ec4dbe2cf]
[2022-04-27 13:05:27,575][XNIO-1 task-3] INFO AccessLog - [AzSuccess] - scale - https://k8sou.sundsvall.se/scale/ - uid=danx-49-xx-49-xost,ou=shadow,o=Tremolo - NONE [10.197.32.98] - [fa6a0961adae3f8ebb16876411e83f36ec4dbe2cf]
[2022-04-27 13:05:30,903][XNIO-1 task-2] INFO AccessLog - [AzSuccess] - k8sidp - https://127.0.0.1:8443/auth/idp/k8sIdp/.well-known/openid-configuration - uid=Anonymous,o=Tremolo - NONE [127.0.0.1] - [f781e707eb9164bad67c7f139931057e8414e3944]
[2022-04-27 13:05:30,903][XNIO-1 task-1] INFO AccessLog - [AzSuccess] - k8sidp - https://127.0.0.1:8443/auth/idp/k8sIdp/.well-known/openid-configuration - uid=Anonymous,o=Tremolo - NONE [127.0.0.1] - [f3d353a13e9898cf99f6e70fea800d3df65f3dc87]
[2022-04-27 13:05:37,617][XNIO-1 task-3] ERROR ProxySys - Error Executing Request :
[2022-04-27 13:05:37,617][XNIO-1 task-3] INFO AccessLog - [Error] - scale - https://k8sou.sundsvall.se/scale/ - uid=danx-49-xx-49-xost,ou=shadow,o=Tremolo - NONE [10.197.32.98] - [fa6a0961adae3f8ebb16876411e83f36ec4dbe2cf]
[2022-04-27 13:05:37,617][XNIO-1 task-3] ERROR ConfigSys - Could not process request
javax.servlet.ServletException: Could not execute request
at com.tremolosecurity.proxy.ProxySys.doURI(ProxySys.java:112) ~[unison-server-core-1.0.29.jar:?]
at com.tremolosecurity.embedd.NextEmbSys.nextSys(NextEmbSys.java:141) ~[unison-server-core-1.0.29.jar:?]
at com.tremolosecurity.proxy.auth.AuthMgrSys.doAuthMgr(AuthMgrSys.java:138) ~[unison-server-core-1.0.29.jar:?]
at com.tremolosecurity.embedd.NextEmbSys.nextSys(NextEmbSys.java:126) ~[unison-server-core-1.0.29.jar:?]
at com.tremolosecurity.proxy.auth.AzSys.doAz(AzSys.java:139) ~[unison-sdk-1.0.29.jar:?]
at com.tremolosecurity.embedd.NextEmbSys.nextSys(NextEmbSys.java:111) ~[unison-server-core-1.0.29.jar:?]
at com.tremolosecurity.proxy.auth.AuthSys.doAuth(AuthSys.java:140) ~[unison-server-core-1.0.29.jar:?]
at com.tremolosecurity.embedd.NextEmbSys.nextSys(NextEmbSys.java:105) ~[unison-server-core-1.0.29.jar:?]
at com.tremolosecurity.proxy.ConfigSys.doConfig(ConfigSys.java:296) ~[unison-server-core-1.0.29.jar:?]
at com.tremolosecurity.embedd.NextEmbSys.nextSys(NextEmbSys.java:93) ~[unison-server-core-1.0.29.jar:?]
at com.tremolosecurity.filter.UnisonServletFilter.doFilter(UnisonServletFilter.java:299) ~[unison-server-core-1.0.29.jar:?]
at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61) ~[undertow-servlet-2.2.16.Final.jar:2.2.16.Final]
at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131) ~[undertow-servlet-2.2.16.Final.jar:2.2.16.Final]
at io.undertow.servlet.handlers.FilterHandler.handleRequest(FilterHandler.java:84) ~[undertow-servlet-2.2.16.Final.jar:2.2.16.Final]
at io.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:62) ~[undertow-servlet-2.2.16.Final.jar:2.2.16.Final]
at io.undertow.servlet.handlers.ServletChain$1.handleRequest(ServletChain.java:68) ~[undertow-servlet-2.2.16.Final.jar:2.2.16.Final]
at io.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36) ~[undertow-servlet-2.2.16.Final.jar:2.2.16.Final]
at io.undertow.servlet.handlers.RedirectDirHandler.handleRequest(RedirectDirHandler.java:68) ~[undertow-servlet-2.2.16.Final.jar:2.2.16.Final]
at io.undertow.servlet.handlers.security.SSLInformationAssociationHandler.handleRequest(SSLInformationAssociationHandler.java:117) ~[undertow-servlet-2.2.16.Final.jar:2.2.16.Final]
at io.undertow.servlet.handlers.security.ServletAuthenticationCallHandler.handleRequest(ServletAuthenticationCallHandler.java:57) ~[undertow-servlet-2.2.16.Final.jar:2.2.16.Final]
at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) ~[undertow-core-2.2.16.Final.jar:2.2.16.Final]
at io.undertow.security.handlers.AbstractConfidentialityHandler.handleRequest(AbstractConfidentialityHandler.java:46) ~[undertow-core-2.2.16.Final.jar:2.2.16.Final]
at io.undertow.servlet.handlers.security.ServletConfidentialityConstraintHandler.handleRequest(ServletConfidentialityConstraintHandler.java:64) ~[undertow-servlet-2.2.16.Final.jar:2.2.16.Final]
at io.undertow.security.handlers.AuthenticationMechanismsHandler.handleRequest(AuthenticationMechanismsHandler.java:60) ~[undertow-core-2.2.16.Final.jar:2.2.16.Final]
at io.undertow.servlet.handlers.security.CachedAuthenticatedSessionHandler.handleRequest(CachedAuthenticatedSessionHandler.java:77) ~[undertow-servlet-2.2.16.Final.jar:2.2.16.Final]
at io.undertow.security.handlers.AbstractSecurityContextAssociationHandler.handleRequest(AbstractSecurityContextAssociationHandler.java:43) ~[undertow-core-2.2.16.Final.jar:2.2.16.Final]
at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) ~[undertow-core-2.2.16.Final.jar:2.2.16.Final]
at io.undertow.servlet.handlers.SendErrorPageHandler.handleRequest(SendErrorPageHandler.java:52) ~[undertow-servlet-2.2.16.Final.jar:2.2.16.Final]
at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) ~[undertow-core-2.2.16.Final.jar:2.2.16.Final]
at io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:275) ~[undertow-servlet-2.2.16.Final.jar:2.2.16.Final]
at io.undertow.servlet.handlers.ServletInitialHandler.access$100(ServletInitialHandler.java:79) ~[undertow-servlet-2.2.16.Final.jar:2.2.16.Final]
at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:134) ~[undertow-servlet-2.2.16.Final.jar:2.2.16.Final]
at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:131) ~[undertow-servlet-2.2.16.Final.jar:2.2.16.Final]
at io.undertow.servlet.core.ServletRequestContextThreadSetupAction$1.call(ServletRequestContextThreadSetupAction.java:48) ~[undertow-servlet-2.2.16.Final.jar:2.2.16.Final]
at io.undertow.servlet.core.ContextClassLoaderSetupAction$1.call(ContextClassLoaderSetupAction.java:43) ~[undertow-servlet-2.2.16.Final.jar:2.2.16.Final]
at io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:255) ~[undertow-servlet-2.2.16.Final.jar:2.2.16.Final]
at io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:79) ~[undertow-servlet-2.2.16.Final.jar:2.2.16.Final]
at io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:100) ~[undertow-servlet-2.2.16.Final.jar:2.2.16.Final]
at io.undertow.server.Connectors.executeRootHandler(Connectors.java:387) ~[undertow-core-2.2.16.Final.jar:2.2.16.Final]
at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:852) ~[undertow-core-2.2.16.Final.jar:2.2.16.Final]
at org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35) ~[jboss-threads-2.3.6.Final.jar:2.3.6.Final]
at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1982) ~[jboss-threads-2.3.6.Final.jar:2.3.6.Final]
at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1486) ~[jboss-threads-2.3.6.Final.jar:2.3.6.Final]
at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1377) ~[jboss-threads-2.3.6.Final.jar:2.3.6.Final]
at org.xnio.XnioWorker$WorkerThreadFactory$1$1.run(XnioWorker.java:1280) ~[xnio-api-3.8.6.Final.jar:3.8.6.Final]
at java.lang.Thread.run(Thread.java:829) ~[?:?]
Caused by: java.net.UnknownHostException: ouhtml-orchestra-login-portal.openunison.svc: Name or service not known
at java.net.Inet6AddressImpl.lookupAllHostAddr(Native Method) ~[?:?]
at java.net.InetAddress$PlatformNameService.lookupAllHostAddr(InetAddress.java:929) ~[?:?]
at java.net.InetAddress.getAddressesFromNameService(InetAddress.java:1519) ~[?:?]
at java.net.InetAddress$NameServiceAddresses.get(InetAddress.java:848) ~[?:?]
at java.net.InetAddress.getAllByName0(InetAddress.java:1509) ~[?:?]
at java.net.InetAddress.getAllByName(InetAddress.java:1368) ~[?:?]
at java.net.InetAddress.getAllByName(InetAddress.java:1302) ~[?:?]
at org.apache.http.impl.conn.SystemDefaultDnsResolver.resolve(SystemDefaultDnsResolver.java:45) ~[httpclient-4.5.13.jar:4.5.13]
at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:112) ~[httpclient-4.5.13.jar:4.5.13]
at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) ~[httpclient-4.5.13.jar:4.5.13]
at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) ~[httpclient-4.5.13.jar:4.5.13]
at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) ~[httpclient-4.5.13.jar:4.5.13]
at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) ~[httpclient-4.5.13.jar:4.5.13]
at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) ~[httpclient-4.5.13.jar:4.5.13]
at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) ~[httpclient-4.5.13.jar:4.5.13]
at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) ~[httpclient-4.5.13.jar:4.5.13]
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) ~[httpclient-4.5.13.jar:4.5.13]
at com.tremolosecurity.proxy.postProcess.UriRequestProcess.postProcess(UriRequestProcess.java:127) ~[unison-server-core-1.0.29.jar:?]
at com.tremolosecurity.proxy.filter.HttpFilterChainImpl.nextFilter(HttpFilterChainImpl.java:92) ~[unison-server-core-1.0.29.jar:?]
at com.tremolosecurity.proxy.filters.SetNoCacheHeaders.doFilter(SetNoCacheHeaders.java:25) ~[unison-server-core-1.0.29.jar:?]
at com.tremolosecurity.proxy.filter.HttpFilterChainImpl.nextFilter(HttpFilterChainImpl.java:86) ~[unison-server-core-1.0.29.jar:?]
at com.tremolosecurity.proxy.filters.XForward.doFilter(XForward.java:61) ~[unison-server-core-1.0.29.jar:?]
at com.tremolosecurity.proxy.filter.HttpFilterChainImpl.nextFilter(HttpFilterChainImpl.java:86) ~[unison-server-core-1.0.29.jar:?]
at com.tremolosecurity.proxy.ProxySys.doURI(ProxySys.java:97) ~[unison-server-core-1.0.29.jar:?]
... 45 moreCaused by: java.net.UnknownHostException: ouhtml-orchestra-login-portal.openunison.svc: Name or service not known
Is this a certificate issue?
mlbiam
commented
2 years ago Caused by: java.net.UnknownHostException: ouhtml-orchestra-login-portal.openunison.svc: Name or service not known
that sounds like the orchestra-login-service chart didn't deploy correctly? What services and pods are in the openunison namespace?
danielSundsvallSCIT
commented
2 years ago that sounds like the orchestra-login-service chart didn't deploy correctly? What services and pods are in the
openunisonnamespace?
Pods
check-certs-orchestra-27517080--1-9zc8w 0/1 Completed 0 28h
check-certs-orchestra-27518520--1-2mm9j 0/1 Error 0 4h
check-certs-orchestra-27518520--1-6tdjk 0/1 Error 0 3h59m
openunison-operator-58d5458bc4-h8wbk 1/1 Running 0 39h
openunison-orchestra-df9c54654-c7xz4 1/1 Running 0 17h
ouhtml-orchestra-login-portal-55ccc56cc6-txqtd 1/1 RunningServices
openunison-orchestra ClusterIP 10.197.132.144 <none> 443/TCP,80/TCP 39h
ouhtml-orchestra-login-portal ClusterIP 10.197.135.156 <none> 8080/TCP 39hThe Error I get on the check-cert pods
Exception in thread "main" java.net.UnknownHostException: kubernetes.default.svc.cluster.local: Temporary failure in name resolution
at java.net.Inet6AddressImpl.lookupAllHostAddr(Native Method)
at java.net.InetAddress$2.lookupAllHostAddr(InetAddress.java:929)
at java.net.InetAddress.getAddressesFromNameService(InetAddress.java:1324)
at java.net.InetAddress.getAllByName0(InetAddress.java:1277)
at java.net.InetAddress.getAllByName(InetAddress.java:1193)
at java.net.InetAddress.getAllByName(InetAddress.java:1127)
at org.apache.http.impl.conn.SystemDefaultDnsResolver.resolve(SystemDefaultDnsResolver.java:45)
at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:112)
at org.apache.http.impl.conn.BasicHttpClientConnectionManager.connect(BasicHttpClientConnectionManager.java:313)
at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393)
at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236)
at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185)
at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89)
at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110)
at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:108)
at com.tremolosecurity.kubernetes.artifacts.util.K8sUtils.callWS(K8sUtils.java:464)
at com.tremolosecurity.kubernetes.artifacts.util.K8sUtils.callWS(K8sUtils.java:281)
at com.tremolosecurity.kubernetes.artifacts.util.K8sUtils.<init>(K8sUtils.java:226)
at com.tremolosecurity.kubernetes.artifacts.run.RunDeployment.main(RunDeployment.java:75)Or is it OK if only 1 of check-cert job is completed?
mlbiam
commented
2 years ago The check certs failing isn't a big deal, but it looks like you have DNS issues in your cluster. The fact that you can't resolve the html pod or the API server are red flags for DNS issues.
danielSundsvallSCIT
commented
2 years ago You were right about the DNS problem, went on to have problem to do nslookup to internal hostnames from a dnsutils pod. But managed to fix the problem.
But are still getting error log in the orchestrator pod that points to a DNS problem. Is it possible to run a nslookup from inside the orchestrator pod?
Cause as I understand it, it is the orchestrator pod that cant do a name resolution on ouhtml-orchestra-login-portal.openunison.svc??
From inside the dnsutils pod we can to a nslookup to "ouhtml-orchestra-login-portal.openunison.svc"
[2022-05-02 10:15:45,913][XNIO-1 task-1] ERROR ProxySys - Error Executing Request :
[2022-05-02 10:15:45,913][XNIO-1 task-1] INFO AccessLog - [Error] - scale - https://k8sou.sundsvall.se/scale/ - uid=danx-49-xx-49-xost,ou=shadow,o=Tremolo - NONE [10.197.32.98] - [fed783ea095bda75aae7c87b01e2d6e6b39bde370]
[2022-05-02 10:15:45,913][XNIO-1 task-1] ERROR ConfigSys - Could not process request
javax.servlet.ServletException: Could not execute request
at com.tremolosecurity.proxy.ProxySys.doURI(ProxySys.java:112) ~[unison-server-core-1.0.29.jar:?]
at com.tremolosecurity.embedd.NextEmbSys.nextSys(NextEmbSys.java:141) ~[unison-server-core-1.0.29.jar:?]
at com.tremolosecurity.proxy.auth.AuthMgrSys.doAuthMgr(AuthMgrSys.java:138) ~[unison-server-core-1.0.29.jar:?]
at com.tremolosecurity.embedd.NextEmbSys.nextSys(NextEmbSys.java:126) ~[unison-server-core-1.0.29.jar:?]
at com.tremolosecurity.proxy.auth.AzSys.doAz(AzSys.java:139) ~[unison-sdk-1.0.29.jar:?]
at com.tremolosecurity.embedd.NextEmbSys.nextSys(NextEmbSys.java:111) ~[unison-server-core-1.0.29.jar:?]
at com.tremolosecurity.proxy.auth.AuthSys.doAuth(AuthSys.java:140) ~[unison-server-core-1.0.29.jar:?]
at com.tremolosecurity.embedd.NextEmbSys.nextSys(NextEmbSys.java:105) ~[unison-server-core-1.0.29.jar:?]
at com.tremolosecurity.proxy.ConfigSys.doConfig(ConfigSys.java:296) ~[unison-server-core-1.0.29.jar:?]
at com.tremolosecurity.embedd.NextEmbSys.nextSys(NextEmbSys.java:93) ~[unison-server-core-1.0.29.jar:?]
at com.tremolosecurity.filter.UnisonServletFilter.doFilter(UnisonServletFilter.java:299) ~[unison-server-core-1.0.29.jar:?]
at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61) ~[undertow-servlet-2.2.16.Final.jar:2.2.16.Final]
at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131) ~[undertow-servlet-2.2.16.Final.jar:2.2.16.Final]
at io.undertow.servlet.handlers.FilterHandler.handleRequest(FilterHandler.java:84) ~[undertow-servlet-2.2.16.Final.jar:2.2.16.Final]
at io.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:62) ~[undertow-servlet-2.2.16.Final.jar:2.2.16.Final]
at io.undertow.servlet.handlers.ServletChain$1.handleRequest(ServletChain.java:68) ~[undertow-servlet-2.2.16.Final.jar:2.2.16.Final]
at io.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36) ~[undertow-servlet-2.2.16.Final.jar:2.2.16.Final]
at io.undertow.servlet.handlers.RedirectDirHandler.handleRequest(RedirectDirHandler.java:68) ~[undertow-servlet-2.2.16.Final.jar:2.2.16.Final]
at io.undertow.servlet.handlers.security.SSLInformationAssociationHandler.handleRequest(SSLInformationAssociationHandler.java:117) ~[undertow-servlet-2.2.16.Final.jar:2.2.16.Final]
at io.undertow.servlet.handlers.security.ServletAuthenticationCallHandler.handleRequest(ServletAuthenticationCallHandler.java:57) ~[undertow-servlet-2.2.16.Final.jar:2.2.16.Final]
at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) ~[undertow-core-2.2.16.Final.jar:2.2.16.Final]
at io.undertow.security.handlers.AbstractConfidentialityHandler.handleRequest(AbstractConfidentialityHandler.java:46) ~[undertow-core-2.2.16.Final.jar:2.2.16.Final]
at io.undertow.servlet.handlers.security.ServletConfidentialityConstraintHandler.handleRequest(ServletConfidentialityConstraintHandler.java:64) ~[undertow-servlet-2.2.16.Final.jar:2.2.16.Final]
at io.undertow.security.handlers.AuthenticationMechanismsHandler.handleRequest(AuthenticationMechanismsHandler.java:60) ~[undertow-core-2.2.16.Final.jar:2.2.16.Final]
at io.undertow.servlet.handlers.security.CachedAuthenticatedSessionHandler.handleRequest(CachedAuthenticatedSessionHandler.java:77) ~[undertow-servlet-2.2.16.Final.jar:2.2.16.Final]
at io.undertow.security.handlers.AbstractSecurityContextAssociationHandler.handleRequest(AbstractSecurityContextAssociationHandler.java:43) ~[undertow-core-2.2.16.Final.jar:2.2.16.Final]
at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) ~[undertow-core-2.2.16.Final.jar:2.2.16.Final]
at io.undertow.servlet.handlers.SendErrorPageHandler.handleRequest(SendErrorPageHandler.java:52) ~[undertow-servlet-2.2.16.Final.jar:2.2.16.Final]
at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) ~[undertow-core-2.2.16.Final.jar:2.2.16.Final]
at io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:275) ~[undertow-servlet-2.2.16.Final.jar:2.2.16.Final]
at io.undertow.servlet.handlers.ServletInitialHandler.access$100(ServletInitialHandler.java:79) ~[undertow-servlet-2.2.16.Final.jar:2.2.16.Final]
at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:134) ~[undertow-servlet-2.2.16.Final.jar:2.2.16.Final]
at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:131) ~[undertow-servlet-2.2.16.Final.jar:2.2.16.Final]
at io.undertow.servlet.core.ServletRequestContextThreadSetupAction$1.call(ServletRequestContextThreadSetupAction.java:48) ~[undertow-servlet-2.2.16.Final.jar:2.2.16.Final]
at io.undertow.servlet.core.ContextClassLoaderSetupAction$1.call(ContextClassLoaderSetupAction.java:43) ~[undertow-servlet-2.2.16.Final.jar:2.2.16.Final]
at io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:255) ~[undertow-servlet-2.2.16.Final.jar:2.2.16.Final]
at io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:79) ~[undertow-servlet-2.2.16.Final.jar:2.2.16.Final]
at io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:100) ~[undertow-servlet-2.2.16.Final.jar:2.2.16.Final]
at io.undertow.server.Connectors.executeRootHandler(Connectors.java:387) ~[undertow-core-2.2.16.Final.jar:2.2.16.Final]
at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:852) ~[undertow-core-2.2.16.Final.jar:2.2.16.Final]
at org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35) ~[jboss-threads-2.3.6.Final.jar:2.3.6.Final]
at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1982) ~[jboss-threads-2.3.6.Final.jar:2.3.6.Final]
at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1486) ~[jboss-threads-2.3.6.Final.jar:2.3.6.Final]
at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1377) ~[jboss-threads-2.3.6.Final.jar:2.3.6.Final]
at org.xnio.XnioWorker$WorkerThreadFactory$1$1.run(XnioWorker.java:1280) ~[xnio-api-3.8.6.Final.jar:3.8.6.Final]
at java.lang.Thread.run(Thread.java:829) ~[?:?]
Caused by: java.net.UnknownHostException: ouhtml-orchestra-login-portal.openunison.svc: Temporary failure in name resolution
at java.net.Inet6AddressImpl.lookupAllHostAddr(Native Method) ~[?:?]
at java.net.InetAddress$PlatformNameService.lookupAllHostAddr(InetAddress.java:929) ~[?:?]
at java.net.InetAddress.getAddressesFromNameService(InetAddress.java:1519) ~[?:?]
at java.net.InetAddress$NameServiceAddresses.get(InetAddress.java:848) ~[?:?]
at java.net.InetAddress.getAllByName0(InetAddress.java:1509) ~[?:?]
at java.net.InetAddress.getAllByName(InetAddress.java:1368) ~[?:?]
at java.net.InetAddress.getAllByName(InetAddress.java:1302) ~[?:?]
at org.apache.http.impl.conn.SystemDefaultDnsResolver.resolve(SystemDefaultDnsResolver.java:45) ~[httpclient-4.5.13.jar:4.5.13]
at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:112) ~[httpclient-4.5.13.jar:4.5.13]
at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) ~[httpclient-4.5.13.jar:4.5.13]
at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) ~[httpclient-4.5.13.jar:4.5.13]
at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) ~[httpclient-4.5.13.jar:4.5.13]
at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) ~[httpclient-4.5.13.jar:4.5.13]
at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) ~[httpclient-4.5.13.jar:4.5.13]
at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) ~[httpclient-4.5.13.jar:4.5.13]
at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) ~[httpclient-4.5.13.jar:4.5.13]
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) ~[httpclient-4.5.13.jar:4.5.13]
at com.tremolosecurity.proxy.postProcess.UriRequestProcess.postProcess(UriRequestProcess.java:127) ~[unison-server-core-1.0.29.jar:?]
at com.tremolosecurity.proxy.filter.HttpFilterChainImpl.nextFilter(HttpFilterChainImpl.java:92) ~[unison-server-core-1.0.29.jar:?]
at com.tremolosecurity.proxy.filters.SetNoCacheHeaders.doFilter(SetNoCacheHeaders.java:25) ~[unison-server-core-1.0.29.jar:?]
at com.tremolosecurity.proxy.filter.HttpFilterChainImpl.nextFilter(HttpFilterChainImpl.java:86) ~[unison-server-core-1.0.29.jar:?]
at com.tremolosecurity.proxy.filters.XForward.doFilter(XForward.java:61) ~[unison-server-core-1.0.29.jar:?]
at com.tremolosecurity.proxy.filter.HttpFilterChainImpl.nextFilter(HttpFilterChainImpl.java:86) ~[unison-server-core-1.0.29.jar:?]
at com.tremolosecurity.proxy.ProxySys.doURI(ProxySys.java:97) ~[unison-server-core-1.0.29.jar:?]
... 45 moreI don't think netutils is installed on there but I think curl -v shows the DNS lookup
danielSundsvallSCIT
commented
2 years ago I don't think netutils is installed on there but I think
curl -vshows the DNS lookup
Yeah cant resolve with curl -v from orchestrator pod.
Im struggling to understand the traffic flow with using SAML2 and OIDC. What the difference is? By using oidc, when accessing the url to the portal, and trying to log in. The orchestrator uses ldap to check credentials. But from there, what is the next step?
I may have missunderstood it all, so please correct me if Im wrong about the flow.
mlbiam
commented
2 years ago Yeah cant resolve with curl -v from orchestrator pod.
so there seems to be a DNS issue still. Can you resolve other services?
Im struggling to understand the traffic flow with using SAML2 and OIDC. What the difference is? By using oidc, when accessing the url to the portal, and trying to log in. The orchestrator uses ldap to check credentials. But from there, what is the next step?
There's no LDAP in either one (though I understand why error messages make it seam that way). For SAML2 the flow is:
User object and a OidcSession object in the OpenUnison namespaceouhtml-orchestra-login-portal Pod that hosts the HTML content of the portal (APIs are still hosted on openunison-orchestra)While loading your session, the User object created in the openunison namespace is loaded via OpenUnison's built in LDAP virtual directory. That's what lets us talk to Kubernetes, active directory, LDAP, databases, etc. When there's an exception, it's an LDAPException even though we aren't talking to an LDAP server.
The OIDC flow is similar except when the idp returns its token, OpenUnison then needs to call the token service on the identity provider to retrieve the user's tokens.
danielSundsvallSCIT
commented
2 years ago so there seems to be a DNS issue still. Can you resolve other services?
No, cant resolve other services like kubernetes.default. But other pods can resolve. So it looks like the problem is in the orchestrator pod/namespace?
I think it´s all on our end, we have another cluster who is using saml2 and on that we can do a curl -v and even wget inside the orchestrator pod.
So I think we can close this ticket as I dont think there is a problem with the application. I get back to you if there should come up a new error or something.
Thanks for the help as usual.
danielSundsvallSCIT
commented
2 years ago As this issue is still open I continue on this, I manages to get it to work with oidc and access the portal. But now when I access the k8-dashboard I get "504 gateway Time-out" and this error log in the orchestrator pod
[2022-05-05 14:29:54,522][XNIO-1 task-1] INFO AccessLog - [Error] - dashboard - https://k8sdashboard.k8spoc.sundsvall.se/auth/oidc - uid=Anonymous,o=Tremolo - NONE [10.197.32.102] - [f062162c776a22f4526819392b95207196303242b]
[2022-05-05 14:29:54,522][XNIO-1 task-1] ERROR ConfigSys - Could not process request
org.apache.http.conn.HttpHostConnectException: Connect to k8sou.k8spoc.sundsvall.se:443 [k8sou.k8spoc.sundsvall.se/10.197.21.4] failed: Connection timed out (Connection timed out)
at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) ~[httpclient-4.5.13.jar:4.5.13]
at org.apache.http.impl.conn.BasicHttpClientConnectionManager.connect(BasicHttpClientConnectionManager.java:313) ~[httpclient-4.5.13.jar:4.5.13]
at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) ~[httpclient-4.5.13.jar:4.5.13]
at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) ~[httpclient-4.5.13.jar:4.5.13]
at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) ~[httpclient-4.5.13.jar:4.5.13]
at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) ~[httpclient-4.5.13.jar:4.5.13]
at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) ~[httpclient-4.5.13.jar:4.5.13]
at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) ~[httpclient-4.5.13.jar:4.5.13]
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) ~[httpclient-4.5.13.jar:4.5.13]
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:108) ~[httpclient-4.5.13.jar:4.5.13]
at com.tremolosecurity.unison.proxy.auth.openidconnect.OpenIDConnectAuthMech.doGet(OpenIDConnectAuthMech.java:293) ~[unison-auth-openidconnect-1.0.30.jar:?]
at com.tremolosecurity.proxy.auth.AuthMgrSys.doAuthMgr(AuthMgrSys.java:196) ~[unison-server-core-1.0.30.jar:?]
at com.tremolosecurity.embedd.NextEmbSys.nextSys(NextEmbSys.java:126) ~[unison-server-core-1.0.30.jar:?]
at com.tremolosecurity.proxy.auth.AzSys.doAz(AzSys.java:89) ~[unison-sdk-1.0.30.jar:?]
at com.tremolosecurity.embedd.NextEmbSys.nextSys(NextEmbSys.java:111) ~[unison-server-core-1.0.30.jar:?]
at com.tremolosecurity.proxy.auth.AuthSys.doAuth(AuthSys.java:88) ~[unison-server-core-1.0.30.jar:?]
at com.tremolosecurity.embedd.NextEmbSys.nextSys(NextEmbSys.java:105) ~[unison-server-core-1.0.30.jar:?]
at com.tremolosecurity.proxy.ConfigSys.doConfig(ConfigSys.java:296) ~[unison-server-core-1.0.30.jar:?]
at com.tremolosecurity.embedd.NextEmbSys.nextSys(NextEmbSys.java:93) ~[unison-server-core-1.0.30.jar:?]
at com.tremolosecurity.filter.UnisonServletFilter.doFilter(UnisonServletFilter.java:299) ~[unison-server-core-1.0.30.jar:?]
at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61) ~[undertow-servlet-2.2.17.Final.jar:2.2.17.Final]
at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131) ~[undertow-servlet-2.2.17.Final.jar:2.2.17.Final]
at io.undertow.servlet.handlers.FilterHandler.handleRequest(FilterHandler.java:84) ~[undertow-servlet-2.2.17.Final.jar:2.2.17.Final]
at io.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:62) ~[undertow-servlet-2.2.17.Final.jar:2.2.17.Final]
at io.undertow.servlet.handlers.ServletChain$1.handleRequest(ServletChain.java:68) ~[undertow-servlet-2.2.17.Final.jar:2.2.17.Final]
at io.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36) ~[undertow-servlet-2.2.17.Final.jar:2.2.17.Final]
at io.undertow.servlet.handlers.RedirectDirHandler.handleRequest(RedirectDirHandler.java:68) ~[undertow-servlet-2.2.17.Final.jar:2.2.17.Final]
at io.undertow.servlet.handlers.security.SSLInformationAssociationHandler.handleRequest(SSLInformationAssociationHandler.java:117) ~[undertow-servlet-2.2.17.Final.jar:2.2.17.Final]
at io.undertow.servlet.handlers.security.ServletAuthenticationCallHandler.handleRequest(ServletAuthenticationCallHandler.java:57) ~[undertow-servlet-2.2.17.Final.jar:2.2.17.Final]
at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) ~[undertow-core-2.2.17.Final.jar:2.2.17.Final]
at io.undertow.security.handlers.AbstractConfidentialityHandler.handleRequest(AbstractConfidentialityHandler.java:46) ~[undertow-core-2.2.17.Final.jar:2.2.17.Final]
at io.undertow.servlet.handlers.security.ServletConfidentialityConstraintHandler.handleRequest(ServletConfidentialityConstraintHandler.java:64) ~[undertow-servlet-2.2.17.Final.jar:2.2.17.Final]
at io.undertow.security.handlers.AuthenticationMechanismsHandler.handleRequest(AuthenticationMechanismsHandler.java:60) ~[undertow-core-2.2.17.Final.jar:2.2.17.Final]
at io.undertow.servlet.handlers.security.CachedAuthenticatedSessionHandler.handleRequest(CachedAuthenticatedSessionHandler.java:77) ~[undertow-servlet-2.2.17.Final.jar:2.2.17.Final]
at io.undertow.security.handlers.AbstractSecurityContextAssociationHandler.handleRequest(AbstractSecurityContextAssociationHandler.java:43) ~[undertow-core-2.2.17.Final.jar:2.2.17.Final]
at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) ~[undertow-core-2.2.17.Final.jar:2.2.17.Final]
at io.undertow.servlet.handlers.SendErrorPageHandler.handleRequest(SendErrorPageHandler.java:52) ~[undertow-servlet-2.2.17.Final.jar:2.2.17.Final]
at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) ~[undertow-core-2.2.17.Final.jar:2.2.17.Final]
at io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:275) ~[undertow-servlet-2.2.17.Final.jar:2.2.17.Final]
at io.undertow.servlet.handlers.ServletInitialHandler.access$100(ServletInitialHandler.java:79) ~[undertow-servlet-2.2.17.Final.jar:2.2.17.Final]
at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:134) ~[undertow-servlet-2.2.17.Final.jar:2.2.17.Final]
at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:131) ~[undertow-servlet-2.2.17.Final.jar:2.2.17.Final]
at io.undertow.servlet.core.ServletRequestContextThreadSetupAction$1.call(ServletRequestContextThreadSetupAction.java:48) ~[undertow-servlet-2.2.17.Final.jar:2.2.17.Final]
at io.undertow.servlet.core.ContextClassLoaderSetupAction$1.call(ContextClassLoaderSetupAction.java:43) ~[undertow-servlet-2.2.17.Final.jar:2.2.17.Final]
at io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:255) ~[undertow-servlet-2.2.17.Final.jar:2.2.17.Final]
at io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:79) ~[undertow-servlet-2.2.17.Final.jar:2.2.17.Final]
at io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:100) ~[undertow-servlet-2.2.17.Final.jar:2.2.17.Final]
at io.undertow.server.Connectors.executeRootHandler(Connectors.java:387) ~[undertow-core-2.2.17.Final.jar:2.2.17.Final]
at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:852) ~[undertow-core-2.2.17.Final.jar:2.2.17.Final]
at org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35) ~[jboss-threads-2.3.6.Final.jar:2.3.6.Final]
at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1982) ~[jboss-threads-2.3.6.Final.jar:2.3.6.Final]
at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1486) ~[jboss-threads-2.3.6.Final.jar:2.3.6.Final]
at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1377) ~[jboss-threads-2.3.6.Final.jar:2.3.6.Final]
at org.xnio.XnioWorker$WorkerThreadFactory$1$1.run(XnioWorker.java:1282) ~[xnio-api-3.8.7.Final.jar:3.8.7.Final]
at java.lang.Thread.run(Thread.java:829) ~[?:?]
Caused by: java.net.ConnectException: Connection timed out (Connection timed out)
at java.net.PlainSocketImpl.socketConnect(Native Method) ~[?:?]
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:412) ~[?:?]
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:255) ~[?:?]
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:237) ~[?:?]
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) ~[?:?]
at java.net.Socket.connect(Socket.java:609) ~[?:?]
at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) ~[httpclient-4.5.13.jar:4.5.13]
at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ~[httpclient-4.5.13.jar:4.5.13]
... 54 moreI set it up to use our internal certificate and that works, but do I have to change the cert_name: under dashboard in the value file?
dashboard:
namespace: "kubernetes-dashboard"
cert_name: "kubernetes-dashboard-certs" # use the ou-tls-certificate??
label: "k8s-app=kubernetes-dashboard"
service_name: kubernetes-dashboard
certs:
use_k8s_cm: false
[2022-05-05 14:29:54,522][XNIO-1 task-1] ERROR ConfigSys - Could not process request org.apache.http.conn.HttpHostConnectException: Connect to k8sou.k8spoc.sundsvall.se:443 [k8sou.k8spoc.sundsvall.se/10.197.21.4] failed: Connection timed out (Connection timed out)
The issue is that openunison can't talk to it's self via the Ingress. This is needed for SSO. This is a common issue and can be fixed by following : https://openunison.github.io/knowledgebase/broken_dashboard/
I set it up to use our internal certificate and that works, but do I have to change the cert_name: under dashboard in the value file?
You want to use an internal certificate for the dashboard instead of the one generated by openunison? since openunison is the only thing that will consume that certificate it's best to let OpenUnison manage it.
danielSundsvallSCIT
commented
2 years ago So I changed to SAML2 and get access to the portal, but I´m "unauthorized" both in the Dashboard and using Kubernetes Token. As for my last I can se that there is a certificate issue, but this message is a new one for me.
E0509 07:30:43.105792 1 authentication.go:63] "Unable to authenticate the request" err="[invalid bearer token, oidc: verify token: failed to verify signature: fetching keys oidc: get keys failed Get \"https://k8sou.k8spoc.sundsvall.se/auth/idp/k8sIdp/certs\": x509: certificate signed by unknown authority]"I did change the OpenUnison certificate following [https://openunison.github.io/knowledgebase/certificates/].
https://k8sou.k8spoc.sundsvall.se/auth/idp/k8sIdp/certs is showing
{"keys":[{"kty":"RSA","kid":"C=SE, ST=Medelpad, L=Vasternorrland, O=Servicecenter IT, OU=Kubernetes, CN=unison-saml2-rp-sig-C=SE, ST=Medelpad, L=Vasternorrland, O=Servicecenter IT, OU=Kubernetes, CN=unison-saml2-rp-sig-1651688724091","use":"sig","alg":"RS256","n":"k2ZtEuF7KqhwZBJ9UiUHUKb-qYHz3wiG-GuIny6Zh-GdQ2wUW_pSVBqukiuuMLmhuKCx2cDjo4A5gm9wHszvA_QkrznKaeHfCrqXhtmDnLHqeASD8C_m527-9GlOxGl2zEJFoRkizVVe8cUXYSlA_UAtLGzohom_uX0hUJHk2IG3HbbOPVMYDAMUIHTufHeSbe1YrnSMhnCegvCleYaHDrh9O2JMG7x0J7Xn0hEBlxoGf4cz0FI2G7f7hGgJVmaPaiBA5rdqcSXxp5XAkA8trwtOdjKmpaHKGvgzwqmUhXvhoA1Ru5cspqut0M4-OeHtkmS4rWsB3GYkyPavJ8-zFw","e":"AQAB"}]}
E0509 07:30:43.105792 1 authentication.go:63] "Unable to authenticate the request" err="[invalid bearer token, oidc: verify token: failed to verify signature: fetching keys oidc: get keys failed Get \"https://k8sou.k8spoc.sundsvall.se/auth/idp/k8sIdp/certs\": x509: certificate signed by unknown authority]"
When you integrated the api-server into your OpenUnison, did you copy the generated cert and copy it to your API servers and reference it in your oidc configuraiton?
danielSundsvallSCIT
commented
2 years ago Well........when I integrated the api-server into our Openunison. I did copy the new ou-tls-certificate tls.crt that was generetated to every API servers.
Like this:
But should I change the https://k8sou.k8spoc.sundsvall.se/auth/idp/k8sIdp/ to https://k8sou.k8spoc.sundsvall.se/auth/idp/k8sIdp/certs ?? As the error URL points to?
And in the certificate I use I have DNS for the network.api_server_host but I do not have a DNS record for it in our DNS. But that should not matter cause we are not using impersonation.
mlbiam
commented
2 years ago Oh, thats a new one. No, you want to keep it https://k8sou.k8spoc.sundsvall.se/auth/idp/k8sIdp/ because that's the issuer. What's really strange about this error is that to get ther the API server had to make two HTTPS calls:
https://k8sou.k8spoc.sundsvall.se/auth/idp/k8sIdp/well-known/openid-configuiration which returns the discovery document.https://k8sou.k8spoc.sundsvall.se/auth/idp/k8sIdp/certs to get your token signing certificate. It appears that DNS resolved k8sou.k8spoc.sundsvall.se to an IP that wasn't your Ingress controller. In your openunison server logs, do your see requests to https://k8sou.k8spoc.sundsvall.se/auth/idp/k8sIdp/well-known/openid-configuiration?
danielSundsvallSCIT
commented
2 years ago [2022-05-09 14:04:01,490][XNIO-1 task-7] INFO AccessLog - [AzSuccess] - k8sidp - https://127.0.0.1:8443/auth/idp/k8sIdp/.well-known/openid-configuration - uid=Anonymous,o=Tremolo - NONE [127.0.0.1] - [f0a4867a091019b0f38b3feee1da5e42cd4ff8e67]
[2022-05-09 14:04:01,492][XNIO-1 task-7] INFO AccessLog - [AzSuccess] - k8sidp - https://127.0.0.1:8443/auth/idp/k8sIdp/.well-known/openid-configuration - uid=Anonymous,o=Tremolo - NONE [127.0.0.1] - [f1b9d85c82db08d3f83a43e06bd24587cc8b07a5c]
[2022-05-09 14:04:01,574][XNIO-1 task-7] INFO AccessLog - [AzSuccess] - k8sidp - https://k8sou.k8spoc.sundsvall.se/auth/idp/k8sIdp/.well-known/openid-configuration - uid=Anonymous,o=Tremolo - NONE [10.197.32.102] - [fb3b7374106bf2ee98f8a5df468f6b6aa0d83873b]
[2022-05-09 14:04:11,489][XNIO-1 task-7] INFO AccessLog - [AzSuccess] - k8sidp - https://127.0.0.1:8443/auth/idp/k8sIdp/.well-known/openid-configuration - uid=Anonymous,o=Tremolo - NONE [127.0.0.1] - [f5e0a5e0b9e256ae44a45c9f2b93ecac10826ea57]
[2022-05-09 14:04:11,489][XNIO-1 task-1] INFO AccessLog - [AzSuccess] - k8sidp - https://127.0.0.1:8443/auth/idp/k8sIdp/.well-known/openid-configuration - uid=Anonymous,o=Tremolo - NONE [127.0.0.1] - [f29751d4d63fa19237f1099f6d678a0b697751b53]
[2022-05-09 14:04:21,492][XNIO-1 task-1] INFO AccessLog - [AzSuccess] - k8sidp - https://127.0.0.1:8443/auth/idp/k8sIdp/.well-known/openid-configuration - uid=Anonymous,o=Tremolo - NONE [127.0.0.1] - [feec8825e1ff84ab7e7949ef925d7f5fd13a9584d]
[2022-05-09 14:04:21,495][XNIO-1 task-1] INFO AccessLog - [AzSuccess] - k8sidp - https://127.0.0.1:8443/auth/idp/k8sIdp/.well-known/openid-configuration - uid=Anonymous,o=Tremolo - NONE [127.0.0.1] - [f341fa2082b0aa7be33e1aab63cea66ebb1e8f04d]
[2022-05-09 14:04:31,501][XNIO-1 task-1] INFO AccessLog - [AzSuccess] - k8sidp - https://127.0.0.1:8443/auth/idp/k8sIdp/.well-known/openid-configuration - uid=Anonymous,o=Tremolo - NONE [127.0.0.1] - [ff387c7d043119efe2cfb3a5dc8eabc72472bb684]
[2022-05-09 14:04:31,502][XNIO-1 task-7] INFO AccessLog - [AzSuccess] - k8sidp - https://127.0.0.1:8443/auth/idp/k8sIdp/.well-known/openid-configuration - uid=Anonymous,o=Tremolo - NONE [127.0.0.1] - [f50cfa63b27343989de17ae21563ab231d7b4e808]
[2022-05-09 14:04:41,495][XNIO-1 task-7] INFO AccessLog - [AzSuccess] - k8sidp - https://127.0.0.1:8443/auth/idp/k8sIdp/.well-known/openid-configuration - uid=Anonymous,o=Tremolo - NONE [127.0.0.1] - [f8dd7c43d9cfd2d7389f56547858372ca738bec0f]
[2022-05-09 14:04:41,531][XNIO-1 task-7] INFO AccessLog - [AzSuccess] - k8sidp - https://127.0.0.1:8443/auth/idp/k8sIdp/.well-known/openid-configuration - uid=Anonymous,o=Tremolo - NONE [127.0.0.1] - [fbd4092a4cb2f741195489d45ea8a12423706c082]
[2022-05-09 14:04:51,486][XNIO-1 task-1] INFO AccessLog - [AzSuccess] - k8sidp - https://127.0.0.1:8443/auth/idp/k8sIdp/.well-known/openid-configuration - uid=Anonymous,o=Tremolo - NONE [127.0.0.1] - [fe06f9ad51807de9c240a25cbf55dbf1a1fc66fb3]
[2022-05-09 14:04:51,489][XNIO-1 task-7] INFO AccessLog - [AzSuccess] - k8sidp - https://127.0.0.1:8443/auth/idp/k8sIdp/.well-known/openid-configuration - uid=Anonymous,o=Tremolo - NONE [127.0.0.1] - [f210050525de9045167e4f3a22fc740c1965df98a]
[2022-05-09 14:04:57,467][XNIO-1 task-7] INFO AccessLog - [AzSuccess] - k8sidp - https://k8sou.k8spoc.sundsvall.se/auth/idp/k8sIdp/.well-known/openid-configuration - uid=Anonymous,o=Tremolo - NONE [10.197.32.102] - [f2508ab77a2938beb00cc713f70829524361b3d83]
[2022-05-09 14:05:01,475][XNIO-1 task-1] INFO AccessLog - [AzSuccess] - k8sidp - https://127.0.0.1:8443/auth/idp/k8sIdp/.well-known/openid-configuration - uid=Anonymous,o=Tremolo - NONE [127.0.0.1] - [f493d68d962bb3b81e9467ce1591385f92ad899e3]I´m bit confussed, now it all seems to work with the dashboard. I get to see everything as my cluster-admin user. But, using the "kubernetes token" I can run commands sometimes and get result but other times I get error: You must be logged in to the server (Unauthorized)
As you can see
Cluster "kubernetes-poc" set.
Context "kubernetes-poc" modified.
User "danx-49-xx-49-xost@kubernetes-poc" set.
Switched to context "kubernetes-poc".
root@WB10950:~# kubectl get nodes
NAME STATUS ROLES AGE VERSION
masternode Ready control-plane,master 5d6h v1.22.8
masternode2 Ready control-plane,master 5d6h v1.22.8
masternode3 Ready control-plane,master 5d6h v1.22.8
workernode Ready <none> 5d6h v1.22.8
workernode2 Ready <none> 5d6h v1.22.8
workernode3 Ready <none> 5d6h v1.22.8
root@WB10950:~# kubectl get pods
error: You must be logged in to the server (Unauthorized)
root@WB10950:~# kubectl get nodes
error: You must be logged in to the server (Unauthorized)
root@WB10950:~# kubectl get pods
NAME READY STATUS RESTARTS AGE
dnsutils 1/1 Running 125 (32m ago) 5d5h
root@WB10950:~# kubectl get nodes
error: You must be logged in to the server (Unauthorized)
root@WB10950:~# kubectl get pods
error: You must be logged in to the server (Unauthorized)
root@WB10950:~# kubectl get nodes
NAME STATUS ROLES AGE VERSION
masternode Ready control-plane,master 5d6h v1.22.8
masternode2 Ready control-plane,master 5d6h v1.22.8
masternode3 Ready control-plane,master 5d6h v1.22.8
workernode Ready <none> 5d6h v1.22.8
workernode2 Ready <none> 5d6h v1.22.8
workernode3 Ready <none> 5d6h v1.22.8I see now that it is using Cluster "kubernetes-poc". My cluster is named "Kubernetes"
clusters:
- cluster:
certificate-authority-data: DATA+OMITTED
server: https://10.197.20.250:6443
name: kubernetesIn my value file I have "Kubernetes-Poc" I see now, can cause some problems?
image: docker.io/tremolosecurity/openunison-k8s
myvd_config_path: "WEB-INF/myvd.conf"
k8s_cluster_name: kubernetes-poc
enable_impersonation: falseI´m bit confussed, now it all seems to work with the dashboard. I get to see everything as my cluster-admin user. But, using the "kubernetes token" I can run commands sometimes and get result but other times I get error: You must be logged in to the server (Unauthorized)
this generally happens when either one of the API servers is misconfigured or there's a global DNS issue.
danielSundsvallSCIT
commented
2 years ago this generally happens when either one of the API servers is misconfigured or there's a global DNS issue.
Yeah looks like you are right, It looks likes it is only masternode1 it gets responses from. For masternode 2 and 3 I get
E0510 07:23:45.438736 1 authentication.go:63] "Unable to authenticate the request" err="[invalid bearer token, oidc: verify token: failed to verify signature: fetching keys oidc: get keys failed Get \"https://k8sou.k8spoc.sundsvall.se/auth/idp/k8sIdp/certs\": x509: certificate signed by unknown authority]"I sometimes get this error logs i openunison-orchestrator pod.
[2022-05-10 07:30:46,659][Thread-11] ERROR K8sWatcher - Could not run watch, waiting 10 seconds
java.net.SocketException: Connection reset
at java.net.SocketInputStream.read(SocketInputStream.java:186) ~[?:?]
at java.net.SocketInputStream.read(SocketInputStream.java:140) ~[?:?]
at sun.security.ssl.SSLSocketInputRecord.read(SSLSocketInputRecord.java:478) ~[?:?]
at sun.security.ssl.SSLSocketInputRecord.readHeader(SSLSocketInputRecord.java:472) ~[?:?]
at sun.security.ssl.SSLSocketInputRecord.bytesInCompletePacket(SSLSocketInputRecord.java:70) ~[?:?]
at sun.security.ssl.SSLSocketImpl.readApplicationRecord(SSLSocketImpl.java:1454) ~[?:?]
at sun.security.ssl.SSLSocketImpl$AppInputStream.read(SSLSocketImpl.java:1065) ~[?:?]
at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:137) ~[httpcore-4.4.15.jar:4.4.15]
at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:153) ~[httpcore-4.4.15.jar:4.4.15]
at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:280) ~[httpcore-4.4.15.jar:4.4.15]
at org.apache.http.impl.io.ChunkedInputStream.getChunkSize(ChunkedInputStream.java:261) ~[httpcore-4.4.15.jar:4.4.15]
at org.apache.http.impl.io.ChunkedInputStream.nextChunk(ChunkedInputStream.java:222) ~[httpcore-4.4.15.jar:4.4.15]
at org.apache.http.impl.io.ChunkedInputStream.read(ChunkedInputStream.java:183) ~[httpcore-4.4.15.jar:4.4.15]
at org.apache.http.conn.EofSensorInputStream.read(EofSensorInputStream.java:135) ~[httpclient-4.5.13.jar:4.5.13]
at sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:284) ~[?:?]
at sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:326) ~[?:?]
at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:178) ~[?:?]
at java.io.InputStreamReader.read(InputStreamReader.java:181) ~[?:?]
at java.io.BufferedReader.fill(BufferedReader.java:161) ~[?:?]
at java.io.BufferedReader.readLine(BufferedReader.java:326) ~[?:?]
at java.io.BufferedReader.readLine(BufferedReader.java:392) ~[?:?]
at com.tremolosecurity.k8s.watch.K8sWatcher.runWatch(K8sWatcher.java:234) ~[unison-applications-k8s-1.0.30.jar:?]
at com.tremolosecurity.k8s.watch.K8sWatcher.run(K8sWatcher.java:206) ~[unison-applications-k8s-1.0.30.jar:?]
at java.lang.Thread.run(Thread.java:829) ~[?:?]Is that something that can point to the problem?
mlbiam
commented
2 years ago Sounds like a networking issue. Connection resets are usually caused by the server closing the connection abrubtly.
danielSundsvallSCIT
commented
2 years ago Sounds like a networking issue. Connection resets are usually caused by the server closing the connection abrubtly.
Yeah you are right, it looks lika a network issue. I tried to restart our coredns pods. Now I get authorized to 2 masternodes out of 3.
I´m closing this ticket with the original problem solved.
Hello again!
We managed to get everything to work using saml2, now we are trying to get it to work using oidc as "K8S_DB_SSO" but we are getting two error logs in the orchestrator pod pointing to a ldap problem.
Second Error
They both seems to point to same cause,
UnknownHostException: SADC016.domainbut im not sure what it really means.I can run
openssl s_client -connect 'hostname.domain.se:636in the orchestrator pod so if I understand correct that means it has connection to my domain controller.Hope to get some tips and trixx I can try.