search

GRuuuuu / GRuuuuu.github.io

hololy
https://gruuuuu.github.io
MIT License
20 stars 12 forks source link

ocp/ocp-operate-error/ #56

Open utterances-bot opened 3 weeks ago

utterances-bot commented 3 weeks ago

Openshift4 Operations -Troubleshooting - 호롤리한 하루

Overview Openshift를 다루며 생겼던 에러들에 대해서 정리해둔 문서입니다. 설치할때 만난 에러들은 여기 -> 설치 Troubleshooting Errors 1. remote error: tls: internal error rsh logs exec 했을때 발생한 에러. Error from server: error dialing backend: remote error: tls: internal error 해결: oc get csr해서 pending되어있는것들 확인 후 approve oc get csr -o name | xargs oc adm certificate approve 2. error: x509: certificate signed by unknown authority openshift클러스터에 로그인을 하려고할때 발생 I0911 01:24:55.482060 27088 loader.go:375] Config loaded from file: /root/dir/auth/kubeconfig I0911 01:24:55.488669 27088 round_trippers.go:443] HEAD https://api.gru.hololy-dev.com:6443/

https://gruuuuu.github.io/ocp/ocp-operate-error/

Hyun-June-Choi commented 3 weeks ago

기억하실지 모르겠지만 오랜만에 댓글(? 질문) 남깁니다. 잘 지내시나요?

error: x509 관련 질문이 있습니다. oauth-openshift-pod의 이름은 노드에 직접 접근해서 알아낼수 있지만 oc 로그인이 안된 상태에서 "oc rsh"를 실행할수가 없어서 질문드립니다.

GRuuuuu commented 3 weeks ago

@Hyun-June-Choi 안녕하세요!
아래 옵션을 넣고 oc로그인을 다시 시도해보시겠어요? --insecure-skip-tls-verify=true

Hyun-June-Choi commented 2 weeks ago

네, --insecure-skip-tls-verify=true 를 넣어도 로그인은 안되는 상태입니다. 개발용 클러스터라서 일단 다른 클러스터로 돌려놓은 상태인데 비슷한 상황 개발 환경에서만 몇번 경험해서 여쭤보았습니다.

다른 이슈인데 일전에 cluster proxy 설정 질문 드렸었는데 proxy설정만 되어도 mirror registry없이 cluster가 설치 되는것은 이전에 확인했습니다. 테스트되면 공유해달라고 하셨었는데 이제서야 공유드리네요...ㅠㅠ

[itdev@bastion ~]$ ./oc login -u admin -p **** --insecure-skip-tls-verify=true --loglevel 10 I0826 16:03:27.618421 144088 loader.go:373] Config loaded from file: /home/itdev/hobis-dev/auth/kubeconfig I0826 16:03:27.618894 144088 round_trippers.go:466] curl -v -XHEAD 'https://api.dev.hobis.hana:6443/' I0826 16:03:27.619787 144088 round_trippers.go:495] HTTP Trace: DNS Lookup for api.dev.hobis.hana resolved to [{10.25.90.41 }] I0826 16:03:27.620719 144088 round_trippers.go:510] HTTP Trace: Dial to tcp:10.25.90.41:6443 succeed I0826 16:03:27.635338 144088 round_trippers.go:553] HEAD https://api.dev.hobis.hana:6443/ 403 Forbidden in 16 milliseconds I0826 16:03:27.635407 144088 round_trippers.go:570] HTTP Statistics: DNSLookup 0 ms Dial 0 ms TLSHandshake 11 ms ServerProcessing 2 ms Duration 16 ms I0826 16:03:27.635462 144088 round_trippers.go:577] Response Headers: I0826 16:03:27.635511 144088 round_trippers.go:580] Strict-Transport-Security: max-age=31536000; includeSubDomains; preload I0826 16:03:27.635560 144088 round_trippers.go:580] Date: Mon, 26 Aug 2024 09:03:27 GMT I0826 16:03:27.635636 144088 round_trippers.go:580] Audit-Id: 4675ee9a-0c56-4547-8f39-1de4579a046e I0826 16:03:27.635674 144088 round_trippers.go:580] Cache-Control: no-cache, private I0826 16:03:27.635699 144088 round_trippers.go:580] X-Kubernetes-Pf-Flowschema-Uid: 023ac55f-6f83-4ae0-be76-56765b295cec I0826 16:03:27.635730 144088 round_trippers.go:580] X-Kubernetes-Pf-Prioritylevel-Uid: 690c1c08-cf08-4dc8-9bdc-8d8c49de1158 I0826 16:03:27.635753 144088 round_trippers.go:580] Content-Length: 186 I0826 16:03:27.635783 144088 round_trippers.go:580] Content-Type: application/json I0826 16:03:27.635823 144088 round_trippers.go:580] X-Content-Type-Options: nosniff WARNING: Using insecure TLS client config. Setting this option is not supported!

I0826 16:03:27.635901 144088 request_token.go:93] GSSAPI Enabled I0826 16:03:27.635969 144088 round_trippers.go:466] curl -v -XGET -H "X-Csrf-Token: 1" 'https://api.dev.hobis.hana:6443/.well-known/oauth-authorization-server' I0826 16:03:27.638057 144088 round_trippers.go:553] GET https://api.dev.hobis.hana:6443/.well-known/oauth-authorization-server 200 OK in 2 milliseconds I0826 16:03:27.638240 144088 round_trippers.go:570] HTTP Statistics: GetConnection 0 ms ServerProcessing 1 ms Duration 2 ms I0826 16:03:27.638340 144088 round_trippers.go:577] Response Headers: I0826 16:03:27.638439 144088 round_trippers.go:580] Strict-Transport-Security: max-age=31536000; includeSubDomains; preload I0826 16:03:27.638553 144088 round_trippers.go:580] X-Kubernetes-Pf-Flowschema-Uid: 023ac55f-6f83-4ae0-be76-56765b295cec I0826 16:03:27.638723 144088 round_trippers.go:580] X-Kubernetes-Pf-Prioritylevel-Uid: 690c1c08-cf08-4dc8-9bdc-8d8c49de1158 I0826 16:03:27.638817 144088 round_trippers.go:580] Content-Length: 582 I0826 16:03:27.638936 144088 round_trippers.go:580] Date: Mon, 26 Aug 2024 09:03:27 GMT I0826 16:03:27.639027 144088 round_trippers.go:580] Audit-Id: 48fbaed0-b5a1-4e02-9d57-9b91fc19dd74 I0826 16:03:27.639117 144088 round_trippers.go:580] Cache-Control: no-cache, private I0826 16:03:27.639207 144088 round_trippers.go:580] Content-Type: application/json I0826 16:03:27.688961 144088 request_token.go:467] falling back to kubeconfig CA due to possible x509 error: x509: certificate signed by unknown authority I0826 16:03:27.689032 144088 round_trippers.go:466] curl -v -XGET -H "X-Csrf-Token: 1" 'https://oauth-openshift.apps.dev.hobis.hana/oauth/authorize?client_id=openshift-challenging-client&code_challenge=3Dj0R356zcM762Z2pzG-jo1MHHcMHx2faXwYHGYDUcw&code_challenge_method=S256&redirect_uri=https%3A%2F%2Foauth-openshift.apps.dev.hobis.hana%2Foauth%2Ftoken%2Fimplicit&response_type=code' I0826 16:03:27.690290 144088 round_trippers.go:495] HTTP Trace: DNS Lookup for oauth-openshift.apps.dev.hobis.hana resolved to [{10.25.90.45 }] I0826 16:03:27.690915 144088 round_trippers.go:510] HTTP Trace: Dial to tcp:10.25.90.45:443 succeed I0826 16:03:27.732731 144088 round_trippers.go:553] GET https://oauth-openshift.apps.dev.hobis.hana/oauth/authorize?client_id=openshift-challenging-client&code_challenge=3Dj0R356zcM762Z2pzG-jo1MHHcMHx2faXwYHGYDUcw&code_challenge_method=S256&redirect_uri=https%3A%2F%2Foauth-openshift.apps.dev.hobis.hana%2Foauth%2Ftoken%2Fimplicit&response_type=code 401 Unauthorized in 43 milliseconds I0826 16:03:27.732805 144088 round_trippers.go:570] HTTP Statistics: DNSLookup 1 ms Dial 0 ms TLSHandshake 9 ms ServerProcessing 31 ms Duration 43 ms I0826 16:03:27.732835 144088 round_trippers.go:577] Response Headers: I0826 16:03:27.732860 144088 round_trippers.go:580] Expires: 0 I0826 16:03:27.732889 144088 round_trippers.go:580] Pragma: no-cache I0826 16:03:27.732920 144088 round_trippers.go:580] Referrer-Policy: strict-origin-when-cross-origin I0826 16:03:27.732947 144088 round_trippers.go:580] X-Content-Type-Options: nosniff I0826 16:03:27.732973 144088 round_trippers.go:580] Content-Length: 0 I0826 16:03:27.733000 144088 round_trippers.go:580] X-Xss-Protection: 1; mode=block I0826 16:03:27.733027 144088 round_trippers.go:580] Date: Mon, 26 Aug 2024 09:03:27 GMT I0826 16:03:27.733054 144088 round_trippers.go:580] Audit-Id: 3f8972a2-a9e8-4d71-a94c-b049f3c51909 I0826 16:03:27.733076 144088 round_trippers.go:580] Cache-Control: no-cache, no-store, max-age=0, must-revalidate I0826 16:03:27.733097 144088 round_trippers.go:580] Www-Authenticate: Basic realm="openshift" I0826 16:03:27.733124 144088 round_trippers.go:580] X-Dns-Prefetch-Control: off I0826 16:03:27.733151 144088 round_trippers.go:580] X-Frame-Options: DENY I0826 16:03:27.733272 144088 round_trippers.go:466] curl -v -XGET -H "Authorization: Basic " -H "X-Csrf-Token: 1" 'https://oauth-openshift.apps.dev.hobis.hana/oauth/authorize?client_id=openshift-challenging-client&code_challenge=3Dj0R356zcM762Z2pzG-jo1MHHcMHx2faXwYHGYDUcw&code_challenge_method=S256&redirect_uri=https%3A%2F%2Foauth-openshift.apps.dev.hobis.hana%2Foauth%2Ftoken%2Fimplicit&response_type=code' I0826 16:03:27.830094 144088 round_trippers.go:553] GET https://oauth-openshift.apps.dev.hobis.hana/oauth/authorize?client_id=openshift-challenging-client&code_challenge=3Dj0R356zcM762Z2pzG-jo1MHHcMHx2faXwYHGYDUcw&code_challenge_method=S256&redirect_uri=https%3A%2F%2Foauth-openshift.apps.dev.hobis.hana%2Foauth%2Ftoken%2Fimplicit&response_type=code 302 Found in 96 milliseconds I0826 16:03:27.830161 144088 round_trippers.go:570] HTTP Statistics: GetConnection 0 ms ServerProcessing 96 ms Duration 96 ms I0826 16:03:27.830193 144088 round_trippers.go:577] Response Headers: I0826 16:03:27.830225 144088 round_trippers.go:580] Date: Mon, 26 Aug 2024 09:03:27 GMT I0826 16:03:27.830261 144088 round_trippers.go:580] Pragma: no-cache I0826 16:03:27.830292 144088 round_trippers.go:580] Pragma: no-cache I0826 16:03:27.830334 144088 round_trippers.go:580] Location: https://oauth-openshift.apps.dev.hobis.hana/oauth/token/implicit?code=sha256~OtYYxgpeYmFON5lsq0AhkSi5Zc0BIYzA0ccIvLgyYDc&state= I0826 16:03:27.830381 144088 round_trippers.go:580] Set-Cookie: ssn=MTcyNDY2MzAwN3xhVENia1BKQ3VLcHhTZ3ZRUm5za2RyNVlVY19icG9ockNmNERSb19QbW5hNmJ2Uk10TVA1dUQ2TUl6QmxMSFdoVlc5UlhfS1BWT2Z0M3VvSUxGMEtJU3A2RU9UTGZvTXk4QWd5ZnJFOFNzZjltUjRrcUYxUnV5R2lkZzJrZGFmNTBRRmx0Q2IwTzl2QnNQcm1zLS1NTGZWcS1HUE4yOHVQREE9PXysGzwwlqAVNrEX8LAMaeHMxTwnzBGKi1YM8yDdne7phA==; Path=/; HttpOnly; Secure I0826 16:03:27.830429 144088 round_trippers.go:580] X-Dns-Prefetch-Control: off I0826 16:03:27.830478 144088 round_trippers.go:580] X-Frame-Options: DENY I0826 16:03:27.830506 144088 round_trippers.go:580] X-Xss-Protection: 1; mode=block I0826 16:03:27.830527 144088 round_trippers.go:580] Content-Length: 0 I0826 16:03:27.830554 144088 round_trippers.go:580] Audit-Id: 76beffab-7ac8-4e8a-8935-8739c3cd57b0 I0826 16:03:27.830642 144088 round_trippers.go:580] Cache-Control: no-cache, no-store, max-age=0, must-revalidate I0826 16:03:27.830694 144088 round_trippers.go:580] Cache-Control: no-cache, no-store, max-age=0, must-revalidate I0826 16:03:27.830723 144088 round_trippers.go:580] Expires: 0 I0826 16:03:27.830749 144088 round_trippers.go:580] Expires: Fri, 01 Jan 1990 00:00:00 GMT I0826 16:03:27.830770 144088 round_trippers.go:580] Referrer-Policy: strict-origin-when-cross-origin I0826 16:03:27.830803 144088 round_trippers.go:580] X-Content-Type-Options: nosniff I0826 16:03:27.830982 144088 round_trippers.go:466] curl -v -XPOST -H "Authorization: Basic " -H "Content-Type: application/x-www-form-urlencoded" -H "Accept: application/json" 'https://oauth-openshift.apps.dev.hobis.hana/oauth/token' I0826 16:03:27.932707 144088 round_trippers.go:553] POST https://oauth-openshift.apps.dev.hobis.hana/oauth/token 200 OK in 101 milliseconds I0826 16:03:27.932867 144088 round_trippers.go:570] HTTP Statistics: GetConnection 0 ms ServerProcessing 101 ms Duration 101 ms I0826 16:03:27.932957 144088 round_trippers.go:577] Response Headers: I0826 16:03:27.933045 144088 round_trippers.go:580] X-Xss-Protection: 1; mode=block I0826 16:03:27.933129 144088 round_trippers.go:580] Referrer-Policy: strict-origin-when-cross-origin I0826 16:03:27.933209 144088 round_trippers.go:580] Pragma: no-cache I0826 16:03:27.933315 144088 round_trippers.go:580] Pragma: no-cache I0826 16:03:27.933357 144088 round_trippers.go:580] X-Content-Type-Options: nosniff I0826 16:03:27.933446 144088 round_trippers.go:580] X-Dns-Prefetch-Control: off I0826 16:03:27.933513 144088 round_trippers.go:580] Cache-Control: no-cache, no-store, max-age=0, must-revalidate I0826 16:03:27.933544 144088 round_trippers.go:580] Cache-Control: no-cache, no-store, max-age=0, must-revalidate I0826 16:03:27.933615 144088 round_trippers.go:580] Content-Type: application/json I0826 16:03:27.933645 144088 round_trippers.go:580] Expires: 0 I0826 16:03:27.933726 144088 round_trippers.go:580] Expires: Fri, 01 Jan 1990 00:00:00 GMT I0826 16:03:27.933808 144088 round_trippers.go:580] X-Frame-Options: DENY I0826 16:03:27.933872 144088 round_trippers.go:580] Date: Mon, 26 Aug 2024 09:03:27 GMT I0826 16:03:27.933901 144088 round_trippers.go:580] Content-Length: 132 I0826 16:03:27.933952 144088 round_trippers.go:580] Audit-Id: e16149a2-efdc-4bd3-9abb-e1885a72bd2d I0826 16:03:27.934562 144088 round_trippers.go:466] curl -v -XGET -H "Accept: application/json, /" -H "User-Agent: oc/v4.2.0 (linux/amd64) kubernetes/e561d37" -H "Authorization: Bearer " 'https://api.dev.hobis.hana:6443/apis/user.openshift.io/v1/users/~' I0826 16:03:27.940856 144088 round_trippers.go:553] GET https://api.dev.hobis.hana:6443/apis/user.openshift.io/v1/users/~ 401 Unauthorized in 6 milliseconds I0826 16:03:27.940927 144088 round_trippers.go:570] HTTP Statistics: GetConnection 0 ms ServerProcessing 5 ms Duration 6 ms I0826 16:03:27.940946 144088 round_trippers.go:577] Response Headers: I0826 16:03:27.940990 144088 round_trippers.go:580] Strict-Transport-Security: max-age=31536000; includeSubDomains; preload I0826 16:03:27.941013 144088 round_trippers.go:580] Content-Length: 129 I0826 16:03:27.941036 144088 round_trippers.go:580] Date: Mon, 26 Aug 2024 09:03:27 GMT I0826 16:03:27.941086 144088 round_trippers.go:580] Audit-Id: be249d1e-0dc4-4345-b34d-903a45b3f496 I0826 16:03:27.941107 144088 round_trippers.go:580] Cache-Control: no-cache, private I0826 16:03:27.941127 144088 round_trippers.go:580] Content-Type: application/json I0826 16:03:27.941206 144088 request.go:1171] Response Body: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"Unauthorized","reason":"Unauthorized","code":401} I0826 16:03:27.942390 144088 round_trippers.go:466] curl -v -XGET -H "Accept: application/json, /" -H "User-Agent: oc/v4.2.0 (linux/amd64) kubernetes/e561d37" 'https://api.dev.hobis.hana:6443/api/v1/namespaces/openshift/configmaps/motd' I0826 16:03:27.945882 144088 round_trippers.go:553] GET https://api.dev.hobis.hana:6443/api/v1/namespaces/openshift/configmaps/motd 403 Forbidden in 3 milliseconds I0826 16:03:27.945976 144088 round_trippers.go:570] HTTP Statistics: GetConnection 0 ms ServerProcessing 3 ms Duration 3 ms I0826 16:03:27.946034 144088 round_trippers.go:577] Response Headers: I0826 16:03:27.946067 144088 round_trippers.go:580] X-Content-Type-Options: nosniff I0826 16:03:27.946089 144088 round_trippers.go:580] Content-Type: application/json I0826 16:03:27.946143 144088 round_trippers.go:580] Cache-Control: no-cache, private I0826 16:03:27.946170 144088 round_trippers.go:580] Strict-Transport-Security: max-age=31536000; includeSubDomains; preload I0826 16:03:27.946244 144088 round_trippers.go:580] X-Kubernetes-Pf-Flowschema-Uid: 023ac55f-6f83-4ae0-be76-56765b295cec I0826 16:03:27.946337 144088 round_trippers.go:580] X-Kubernetes-Pf-Prioritylevel-Uid: 690c1c08-cf08-4dc8-9bdc-8d8c49de1158 I0826 16:03:27.946359 144088 round_trippers.go:580] Content-Length: 303 I0826 16:03:27.946379 144088 round_trippers.go:580] Date: Mon, 26 Aug 2024 09:03:27 GMT I0826 16:03:27.946418 144088 round_trippers.go:580] Audit-Id: 05d70b75-2c3d-4311-acf1-ab5a7f642788 I0826 16:03:27.946457 144088 request.go:1171] Response Body: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"configmaps \"motd\" is forbidden: User \"system:anonymous\" cannot get resource \"configmaps\" in API group \"\" in the namespace \"openshift\"","reason":"Forbidden","details":{"name":"motd","kind":"configmaps"},"code":403} Login failed (401 Unauthorized) Verify you have provided the correct credentials.

GRuuuuu commented 2 weeks ago

@Hyun-June-Choi 단순히 로그만 보고 판단하기엔 tls문제가 아니라 credential문제로 보입니다. 혹시 admin말고 다른 계정을 시도해보시겠어요? kubeadmin으로 테스트해보시면 좋을 듯 합니다.