search

k8sgpt-ai / k8sgpt-operator

Automatic SRE Superpowers within your Kubernetes cluster
https://k8sgpt.ai
Apache License 2.0
293 stars 80 forks source link

[Bug]: Description field in k8s results is empty #359

Open blueacidification opened 6 months ago

blueacidification commented 6 months ago

Checklist

Affected Components

K8sGPT Version

No response

Kubernetes Version

No response

Host OS and its Version

No response

Steps to reproduce

I followed the installation instructions here https://docs.k8sgpt.ai/getting-started/in-cluster-operator/

Expected behaviour

Suggestions on error fixing show up on description field

Actual behaviour

The description field is empty, probably due to tls certificate errors.

operator logs

{"level":"info","ts":1709049957.7314737,"caller":"server/server.go:83","msg":"binding metrics to 8081"} {"level":"info","ts":1709049957.731643,"caller":"server/server.go:68","msg":"binding api to 8080"} {"level":"info","ts":1709049986.3508625,"caller":"server/log.go:50","msg":"request failed. failed while calling AI provider openai: Post \"https://api.openai.com/v1/chat/completions\": tls: failed to verify certificate: x509: certificate signed by unknown authority","duration_ms":905,"method":"/schema.v1.ServerService/Analyze","request":"backend:\"openai\" explain:true anonymize:true language:\"english\" max_concurrency:10 output:\"json\"","remote_addr":"10.226.255.151:46874","status_code":2} {"level":"info","ts":1709049987.3689353,"caller":"server/log.go:50","msg":"request failed. failed while calling AI provider openai: Post \"https://api.openai.com/v1/chat/completions\": tls: failed to verify certificate: x509: certificate signed by unknown authority","duration_ms":871,"method":"/schema.v1.ServerService/Analyze","request":"backend:\"openai\" explain:true anonymize:true language:\"english\" max_concurrency:10 output:\"json\"","remote_addr":"10.226.255.151:46890","status_code":2} {"level":"info","ts":1709049988.2832956,"caller":"server/log.go:50","msg":"request failed. failed while calling AI provider openai: Post \"https://api.openai.com/v1/chat/completions\": tls: failed to verify certificate: x509: certificate signed by unknown authority","duration_ms":873,"method":"/schema.v1.ServerService/Analyze","request":"backend:\"openai\" explain:true anonymize:true language:\"english\" max_concurrency:10 output:\"json\"","remote_addr":"10.226.255.151:46900","status_code":2} {"level":"info","ts":1709049989.2199733,"caller":"server/log.go:50","msg":"request failed. failed while calling AI provider openai: Post \"https://api.openai.com/v1/chat/completions\": tls: failed to verify certificate: x509: certificate signed by unknown authority","duration_ms":895,"method":"/schema.v1.ServerService/Analyze","request":"backend:\"openai\" explain:true anonymize:true language:\"english\" max_concurrency:10 output:\"json\"","remote_addr":"10.226.255.151:46910","status_code":2} {"level":"info","ts":1709049990.150489,"caller":"server/log.go:50","msg":"request failed. failed while calling AI provider openai: Post \"https://api.openai.com/v1/chat/completions\": tls: failed to verify certificate: x509: certificate signed by unknown authority","duration_ms":869,"method":"/schema.v1.ServerService/Analyze","request":"backend:\"openai\" explain:true anonymize:true language:\"english\" max_concurrency:10 output:\"json\"","remote_addr":"10.226.255.151:46936","status_code":2} {"level":"info","ts":1709049991.131774,"caller":"server/log.go:50","msg":"request failed. failed while calling AI provider openai: Post \"https://api.openai.com/v1/chat/completions\": tls: failed to verify certificate: x509: certificate signed by unknown authority","duration_ms":871,"method":"/schema.v1.ServerService/Analyze","request":"backend:\"openai\" explain:true anonymize:true language:\"english\" max_concurrency:10 output:\"json\"","remote_addr":"10.226.255.151:46954","status_code":2} {"level":"info","ts":1709049992.1812923,"caller":"server/log.go:50","msg":"request failed. failed while calling AI provider openai: Post \"https://api.openai.com/v1/chat/completions\": tls: failed to verify certificate: x509: certificate signed by unknown authority","duration_ms":868,"method":"/schema.v1.ServerService/Analyze","request":"backend:\"openai\" explain:true anonymize:true language:\"english\" max_concurrency:10 output:\"json\"","remote_addr":"10.226.255.151:46970","status_code":2} {"level":"info","ts":1709049993.3672361,"caller":"server/log.go:50","msg":"request completed","duration_ms":840,"method":"/schema.v1.ServerService/Analyze","request":"backend:\"openai\" anonymize:true language:\"english\" max_concurrency:10 output:\"json\"","remote_addr":"10.226.255.151:45074"}

2024-02-27T15:49:44Z INFO controller-runtime.metrics Metrics server is starting to listen {"addr": "127.0.0.1:8080"} 2024-02-27T15:49:44Z INFO setup starting manager 2024-02-27T15:49:44Z INFO Starting server {"kind": "health probe", "addr": "[::]:8081"} 2024-02-27T15:49:44Z INFO starting server {"path": "/metrics", "kind": "metrics", "addr": "127.0.0.1:8080"} I0227 15:49:44.343035 1 leaderelection.go:250] attempting to acquire leader lease k8sgpt-operator-system/ea9c19f7.k8sgpt.ai... I0227 15:50:00.016649 1 leaderelection.go:260] successfully acquired lease k8sgpt-operator-system/ea9c19f7.k8sgpt.ai 2024-02-27T15:50:00Z INFO Starting EventSource {"controller": "k8sgpt", "controllerGroup": "core.k8sgpt.ai", "controllerKind": "K8sGPT", "source": "kind source: v1alpha1.K8sGPT"} 2024-02-27T15:50:00Z INFO Starting Controller {"controller": "k8sgpt", "controllerGroup": "core.k8sgpt.ai", "controllerKind": "K8sGPT"} 2024-02-27T15:50:00Z DEBUG events k8sgpt-operator-controller-manager-5d49cc7c8-bwjvj_30a59862-7521-499d-a00d-4c7c5f2ee21f became leader {"type": "Normal", "object": {"kind":"Lease","namespace":"k8sgpt-operator-system","name":"ea9c19f7.k8sgpt.ai","uid":"b9e6027c-495c-4dbc-92a9-b356deac1c05","apiVersion":"coordination.k8s.io/v1","resourceVersion":"30118309"}, "reason": "LeaderElection"} 2024-02-27T15:50:00Z INFO Starting workers {"controller": "k8sgpt", "controllerGroup": "core.k8sgpt.ai", "controllerKind": "K8sGPT", "worker count": 1} Finished Reconciling k8sGPT E0227 15:50:02.023944 1 reflector.go:147] pkg/mod/k8s.io/client-go@v0.29.1/tools/cache/reflector.go:229: Failed to watch v1alpha1.K8sGPT: the server could not find the requested resource (get k8sgpts.core.k8sgpt.ai) W0227 15:50:03.111307 1 reflector.go:539] pkg/mod/k8s.io/client-go@v0.29.1/tools/cache/reflector.go:229: failed to list v1alpha1.K8sGPT: the server could not find the requested resource (get k8sgpts.core.k8sgpt.ai) E0227 15:50:03.111355 1 reflector.go:147] pkg/mod/k8s.io/client-go@v0.29.1/tools/cache/reflector.go:229: Failed to watch v1alpha1.K8sGPT: failed to list v1alpha1.K8sGPT: the server could not find the requested resource (get k8sgpts.core.k8sgpt.ai) W0227 15:50:05.064488 1 reflector.go:539] pkg/mod/k8s.io/client-go@v0.29.1/tools/cache/reflector.go:229: failed to list v1alpha1.K8sGPT: the server could not find the requested resource (get k8sgpts.core.k8sgpt.ai) E0227 15:50:05.064529 1 reflector.go:147] pkg/mod/k8s.io/client-go@v0.29.1/tools/cache/reflector.go:229: Failed to watch v1alpha1.K8sGPT: failed to list v1alpha1.K8sGPT: the server could not find the requested resource (get k8sgpts.core.k8sgpt.ai) W0227 15:50:10.573187 1 reflector.go:539] pkg/mod/k8s.io/client-go@v0.29.1/tools/cache/reflector.go:229: failed to list v1alpha1.K8sGPT: the server could not find the requested resource (get k8sgpts.core.k8sgpt.ai) E0227 15:50:10.573331 1 reflector.go:147] pkg/mod/k8s.io/client-go@v0.29.1/tools/cache/reflector.go:229: Failed to watch v1alpha1.K8sGPT: failed to list v1alpha1.K8sGPT: the server could not find the requested resource (get k8sgpts.core.k8sgpt.ai) W0227 15:50:17.369624 1 reflector.go:539] pkg/mod/k8s.io/client-go@v0.29.1/tools/cache/reflector.go:229: failed to list v1alpha1.K8sGPT: the server could not find the requested resource (get k8sgpts.core.k8sgpt.ai) E0227 15:50:17.369660 1 reflector.go:147] pkg/mod/k8s.io/client-go@v0.29.1/tools/cache/reflector.go:229: Failed to watch v1alpha1.K8sGPT: failed to list v1alpha1.K8sGPT: the server could not find the requested resource (get k8sgpts.core.k8sgpt.ai) W0227 15:50:42.174958 1 reflector.go:539] pkg/mod/k8s.io/client-go@v0.29.1/tools/cache/reflector.go:229: failed to list v1alpha1.K8sGPT: the server could not find the requested resource (get k8sgpts.core.k8sgpt.ai) E0227 15:50:42.175050 1 reflector.go:147] pkg/mod/k8s.io/client-go@v0.29.1/tools/cache/reflector.go:229: Failed to watch v1alpha1.K8sGPT: failed to list v1alpha1.K8sGPT: the server could not find the requested resource (get k8sgpts.core.k8sgpt.ai) W0227 15:51:21.105496 1 reflector.go:539] pkg/mod/k8s.io/client-go@v0.29.1/tools/cache/reflector.go:229: failed to list v1alpha1.K8sGPT: the server could not find the requested resource (get k8sgpts.core.k8sgpt.ai) E0227 15:51:21.105534 1 reflector.go:147] pkg/mod/k8s.io/client-go@v0.29.1/tools/cache/reflector.go:229: Failed to watch v1alpha1.K8sGPT: failed to list v1alpha1.K8sGPT: the server could not find the requested resource (get k8sgpts.core.k8sgpt.ai) W0227 15:52:13.226463 1 reflector.go:539] pkg/mod/k8s.io/client-go@v0.29.1/tools/cache/reflector.go:229: failed to list v1alpha1.K8sGPT: the server could not find the requested resource (get k8sgpts.core.k8sgpt.ai) E0227 15:52:13.226508 1 reflector.go:147] pkg/mod/k8s.io/client-go@v0.29.1/tools/cache/reflector.go:229: Failed to watch v1alpha1.K8sGPT: failed to list v1alpha1.K8sGPT: the server could not find the requested resource (get k8sgpts.core.k8sgpt.ai) W0227 15:53:06.562863 1 reflector.go:539] pkg/mod/k8s.io/client-go@v0.29.1/tools/cache/reflector.go:229: failed to list v1alpha1.K8sGPT: the server could not find the requested resource (get k8sgpts.core.k8sgpt.ai) E0227 15:53:06.563055 1 reflector.go:147] pkg/mod/k8s.io/client-go@v0.29.1/tools/cache/reflector.go:229: Failed to watch v1alpha1.K8sGPT: failed to list v1alpha1.K8sGPT: the server could not find the requested resource (get k8sgpts.core.k8sgpt.ai) W0227 15:53:38.078142 1 reflector.go:539] pkg/mod/k8s.io/client-go@v0.29.1/tools/cache/reflector.go:229: failed to list v1alpha1.K8sGPT: the server could not find the requested resource (get k8sgpts.core.k8sgpt.ai) E0227 15:53:38.078208 1 reflector.go:147] pkg/mod/k8s.io/client-go@v0.29.1/tools/cache/reflector.go:229: Failed to watch v1alpha1.K8sGPT: failed to list v1alpha1.K8sGPT: the server could not find the requested resource (get k8sgpts.core.k8sgpt.ai) W0227 15:54:12.222873 1 reflector.go:539] pkg/mod/k8s.io/client-go@v0.29.1/tools/cache/reflector.go:229: failed to list v1alpha1.K8sGPT: the server could not find the requested resource (get k8sgpts.core.k8sgpt.ai) E0227 15:54:12.222941 1 reflector.go:147] pkg/mod/k8s.io/client-go@v0.29.1/tools/cache/reflector.go:229: Failed to watch v1alpha1.K8sGPT: failed to list v1alpha1.K8sGPT: the server could not find the requested resource (get k8sgpts.core.k8sgpt.ai) W0227 15:55:08.535654 1 reflector.go:539] pkg/mod/k8s.io/client-go@v0.29.1/tools/cache/reflector.go:229: failed to list v1alpha1.K8sGPT: the server could not find the requested resource (get k8sgpts.core.k8sgpt.ai) E0227 15:55:08.535782 1 reflector.go:147] pkg/mod/k8s.io/client-go@v0.29.1/tools/cache/reflector.go:229: Failed to watch v1alpha1.K8sGPT: failed to list v1alpha1.K8sGPT: the server could not find the requested resource (get k8sgpts.core.k8sgpt.ai) W0227 15:56:01.411178 1 reflector.go:539] pkg/mod/k8s.io/client-go@v0.29.1/tools/cache/reflector.go:229: failed to list v1alpha1.K8sGPT: the server could not find the requested resource (get k8sgpts.core.k8sgpt.ai) E0227 15:56:01.411298 1 reflector.go:147] pkg/mod/k8s.io/client-go@v0.29.1/tools/cache/reflector.go:229: Failed to watch v1alpha1.K8sGPT: failed to list v1alpha1.K8sGPT: the server could not find the requested resource (get k8sgpts.core.k8sgpt.ai) W0227 15:56:51.807140 1 reflector.go:539] pkg/mod/k8s.io/client-go@v0.29.1/tools/cache/reflector.go:229: failed to list v1alpha1.K8sGPT: the server could not find the requested resource (get k8sgpts.core.k8sgpt.ai) E0227 15:56:51.807178 1 reflector.go:147] pkg/mod/k8s.io/client-go@v0.29.1/tools/cache/reflector.go:229: Failed to watch v1alpha1.K8sGPT: failed to list v1alpha1.K8sGPT: the server could not find the requested resource (get k8sgpts.core.k8sgpt.ai) W0227 15:57:41.546606 1 reflector.go:539] pkg/mod/k8s.io/client-go@v0.29.1/tools/cache/reflector.go:229: failed to list v1alpha1.K8sGPT: the server could not find the requested resource (get k8sgpts.core.k8sgpt.ai) E0227 15:57:41.546641 1 reflector.go:147] pkg/mod/k8s.io/client-go@v0.29.1/tools/cache/reflector.go:229: Failed to watch v1alpha1.K8sGPT: failed to list v1alpha1.K8sGPT: the server could not find the requested resource (get k8sgpts.core.k8sgpt.ai) W0227 15:58:11.673180 1 reflector.go:539] pkg/mod/k8s.io/client-go@v0.29.1/tools/cache/reflector.go:229: failed to list v1alpha1.K8sGPT: the server could not find the requested resource (get k8sgpts.core.k8sgpt.ai) E0227 15:58:11.673221 1 reflector.go:147] pkg/mod/k8s.io/client-go@v0.29.1/tools/cache/reflector.go:229: Failed to watch v1alpha1.K8sGPT: failed to list v1alpha1.K8sGPT: the server could not find the requested resource (get k8sgpts.core.k8sgpt.ai) W0227 15:58:42.450552 1 reflector.go:539] pkg/mod/k8s.io/client-go@v0.29.1/tools/cache/reflector.go:229: failed to list v1alpha1.K8sGPT: the server could not find the requested resource (get k8sgpts.core.k8sgpt.ai) E0227 15:58:42.450684 1 reflector.go:147] pkg/mod/k8s.io/client-go@v0.29.1/tools/cache/reflector.go:229: Failed to watch v1alpha1.K8sGPT: failed to list v1alpha1.K8sGPT: the server could not find the requested resource (get k8sgpts.core.k8sgpt.ai) W0227 15:59:29.725643 1 reflector.go:539] pkg/mod/k8s.io/client-go@v0.29.1/tools/cache/reflector.go:229: failed to list v1alpha1.K8sGPT: the server could not find the requested resource (get k8sgpts.core.k8sgpt.ai) E0227 15:59:29.725698 1 reflector.go:147] pkg/mod/k8s.io/client-go@v0.29.1/tools/cache/reflector.go:229: Failed to watch v1alpha1.K8sGPT: failed to list v1alpha1.K8sGPT: the server could not find the requested resource (get k8sgpts.core.k8sgpt.ai) W0227 16:00:27.683481 1 reflector.go:539] pkg/mod/k8s.io/client-go@v0.29.1/tools/cache/reflector.go:229: failed to list v1alpha1.K8sGPT: the server could not find the requested resource (get k8sgpts.core.k8sgpt.ai) E0227 16:00:27.683516 1 reflector.go:147] pkg/mod/k8s.io/client-go@v0.29.1/tools/cache/reflector.go:229: Failed to watch v1alpha1.K8sGPT: failed to list v1alpha1.K8sGPT: the server could not find the requested resource (get k8sgpts.core.k8sgpt.ai) W0227 16:01:19.342470 1 reflector.go:539] pkg/mod/k8s.io/client-go@v0.29.1/tools/cache/reflector.go:229: failed to list v1alpha1.K8sGPT: the server could not find the requested resource (get k8sgpts.core.k8sgpt.ai) E0227 16:01:19.342507 1 reflector.go:147] pkg/mod/k8s.io/client-go@v0.29.1/tools/cache/reflector.go:229: Failed to watch v1alpha1.K8sGPT: failed to list v1alpha1.K8sGPT: the server could not find the requested resource (get k8sgpts.core.k8sgpt.ai) W0227 16:01:56.953314 1 reflector.go:539] pkg/mod/k8s.io/client-go@v0.29.1/tools/cache/reflector.go:229: failed to list v1alpha1.K8sGPT: the server could not find the requested resource (get k8sgpts.core.k8sgpt.ai) E0227 16:01:56.953351 1 reflector.go:147] pkg/mod/k8s.io/client-go@v0.29.1/tools/cache/reflector.go:229: Failed to watch v1alpha1.K8sGPT: failed to list v1alpha1.K8sGPT: the server could not find the requested resource (get k8sgpts.core.k8sgpt.ai) W0227 16:02:50.674812 1 reflector.go:539] pkg/mod/k8s.io/client-go@v0.29.1/tools/cache/reflector.go:229: failed to list v1alpha1.K8sGPT: the server could not find the requested resource (get k8sgpts.core.k8sgpt.ai) E0227 16:02:50.674849 1 reflector.go:147] pkg/mod/k8s.io/client-go@v0.29.1/tools/cache/reflector.go:229: Failed to watch v1alpha1.K8sGPT: failed to list v1alpha1.K8sGPT: the server could not find the requested resource (get k8sgpts.core.k8sgpt.ai) W0227 16:03:47.504501 1 reflector.go:539] pkg/mod/k8s.io/client-go@v0.29.1/tools/cache/reflector.go:229: failed to list v1alpha1.K8sGPT: the server could not find the requested resource (get k8sgpts.core.k8sgpt.ai) E0227 16:03:47.504538 1 reflector.go:147] pkg/mod/k8s.io/client-go@v0.29.1/tools/cache/reflector.go:229: Failed to watch v1alpha1.K8sGPT: failed to list v1alpha1.K8sGPT: the server could not find the requested resource (get k8sgpts.core.k8sgpt.ai) W0227 16:04:37.636919 1 reflector.go:539] pkg/mod/k8s.io/client-go@v0.29.1/tools/cache/reflector.go:229: failed to list v1alpha1.K8sGPT: the server could not find the requested resource (get k8sgpts.core.k8sgpt.ai) E0227 16:04:37.636957 1 reflector.go:147] pkg/mod/k8s.io/client-go@v0.29.1/tools/cache/reflector.go:229: Failed to watch v1alpha1.K8sGPT: failed to list v1alpha1.K8sGPT: the server could not find the requested resource (get k8sgpts.core.k8sgpt.ai) Finished Reconciling k8sGPT Finished Reconciling k8sGPT Creating new client for 172.20.136.189:8080 Connection established between 172.20.136.189:8080 and localhost with time out of 1 seconds. Remote Address : 172.20.136.189:8080 K8sGPT address: 172.20.136.189:8080 Finished Reconciling k8sGPT with error: failed to call Analyze RPC: rpc error: code = Unknown desc = failed while calling AI provider openai: Post "https://api.openai.com/v1/chat/completions": tls: failed to verify certificate: x509: certificate signed by unknown authority 2024-02-27T16:06:26Z ERROR Reconciler error {"controller": "k8sgpt", "controllerGroup": "core.k8sgpt.ai", "controllerKind": "K8sGPT", "K8sGPT": {"name":"k8sgpt-sample","namespace":"k8sgpt-operator-system"}, "namespace": "k8sgpt-operator-system", "name": "k8sgpt-sample", "reconcileID": "58b4b702-d097-4980-ad89-4c752f004f58", "error": "failed to call Analyze RPC: rpc error: code = Unknown desc = failed while calling AI provider openai: Post \"https://api.openai.com/v1/chat/completions\": tls: failed to verify certificate: x509: certificate signed by unknown authority"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(Controller).reconcileHandler /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.15.0/pkg/internal/controller/controller.go:324 sigs.k8s.io/controller-runtime/pkg/internal/controller.(Controller).processNextWorkItem /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.15.0/pkg/internal/controller/controller.go:265 sigs.k8s.io/controller-runtime/pkg/internal/controller.(Controller).Start.func2.2 /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.15.0/pkg/internal/controller/controller.go:226 Creating new client for 172.20.136.189:8080 Connection established between 172.20.136.189:8080 and localhost with time out of 1 seconds. Remote Address : 172.20.136.189:8080 K8sGPT address: 172.20.136.189:8080 Finished Reconciling k8sGPT with error: failed to call Analyze RPC: rpc error: code = Unknown desc = failed while calling AI provider openai: Post "https://api.openai.com/v1/chat/completions": tls: failed to verify certificate: x509: certificate signed by unknown authority 2024-02-27T16:06:27Z ERROR Reconciler error {"controller": "k8sgpt", "controllerGroup": "core.k8sgpt.ai", "controllerKind": "K8sGPT", "K8sGPT": {"name":"k8sgpt-sample","namespace":"k8sgpt-operator-system"}, "namespace": "k8sgpt-operator-system", "name": "k8sgpt-sample", "reconcileID": "65756d07-0ca7-457e-ac5f-d3943ee45a5c", "error": "failed to call Analyze RPC: rpc error: code = Unknown desc = failed while calling AI provider openai: Post \"https://api.openai.com/v1/chat/completions\": tls: failed to verify certificate: x509: certificate signed by unknown authority"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(Controller).reconcileHandler /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.15.0/pkg/internal/controller/controller.go:324 sigs.k8s.io/controller-runtime/pkg/internal/controller.(Controller).processNextWorkItem /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.15.0/pkg/internal/controller/controller.go:265 sigs.k8s.io/controller-runtime/pkg/internal/controller.(Controller).Start.func2.2 /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.15.0/pkg/internal/controller/controller.go:226 Creating new client for 172.20.136.189:8080 Connection established between 172.20.136.189:8080 and localhost with time out of 1 seconds. Remote Address : 172.20.136.189:8080 K8sGPT address: 172.20.136.189:8080 Finished Reconciling k8sGPT with error: failed to call Analyze RPC: rpc error: code = Unknown desc = failed while calling AI provider openai: Post "https://api.openai.com/v1/chat/completions": tls: failed to verify certificate: x509: certificate signed by unknown authority 2024-02-27T16:06:28Z ERROR Reconciler error {"controller": "k8sgpt", "controllerGroup": "core.k8sgpt.ai", "controllerKind": "K8sGPT", "K8sGPT": {"name":"k8sgpt-sample","namespace":"k8sgpt-operator-system"}, "namespace": "k8sgpt-operator-system", "name": "k8sgpt-sample", "reconcileID": "b45a3b35-0870-440b-ab46-a11b6e7f0add", "error": "failed to call Analyze RPC: rpc error: code = Unknown desc = failed while calling AI provider openai: Post \"https://api.openai.com/v1/chat/completions\": tls: failed to verify certificate: x509: certificate signed by unknown authority"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(Controller).reconcileHandler /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.15.0/pkg/internal/controller/controller.go:324 sigs.k8s.io/controller-runtime/pkg/internal/controller.(Controller).processNextWorkItem /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.15.0/pkg/internal/controller/controller.go:265 sigs.k8s.io/controller-runtime/pkg/internal/controller.(Controller).Start.func2.2 /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.15.0/pkg/internal/controller/controller.go:226 Creating new client for 172.20.136.189:8080 Connection established between 172.20.136.189:8080 and localhost with time out of 1 seconds. Remote Address : 172.20.136.189:8080 K8sGPT address: 172.20.136.189:8080 Finished Reconciling k8sGPT with error: failed to call Analyze RPC: rpc error: code = Unknown desc = failed while calling AI provider openai: Post "https://api.openai.com/v1/chat/completions": tls: failed to verify certificate: x509: certificate signed by unknown authority 2024-02-27T16:06:29Z ERROR Reconciler error {"controller": "k8sgpt", "controllerGroup": "core.k8sgpt.ai", "controllerKind": "K8sGPT", "K8sGPT": {"name":"k8sgpt-sample","namespace":"k8sgpt-operator-system"}, "namespace": "k8sgpt-operator-system", "name": "k8sgpt-sample", "reconcileID": "9f4b2992-46d6-4503-bd23-69e6b4b68832", "error": "failed to call Analyze RPC: rpc error: code = Unknown desc = failed while calling AI provider openai: Post \"https://api.openai.com/v1/chat/completions\": tls: failed to verify certificate: x509: certificate signed by unknown authority"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(Controller).reconcileHandler /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.15.0/pkg/internal/controller/controller.go:324 sigs.k8s.io/controller-runtime/pkg/internal/controller.(Controller).processNextWorkItem /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.15.0/pkg/internal/controller/controller.go:265 sigs.k8s.io/controller-runtime/pkg/internal/controller.(Controller).Start.func2.2 /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.15.0/pkg/internal/controller/controller.go:226 Creating new client for 172.20.136.189:8080 Connection established between 172.20.136.189:8080 and localhost with time out of 1 seconds. Remote Address : 172.20.136.189:8080 K8sGPT address: 172.20.136.189:8080 Finished Reconciling k8sGPT with error: failed to call Analyze RPC: rpc error: code = Unknown desc = failed while calling AI provider openai: Post "https://api.openai.com/v1/chat/completions": tls: failed to verify certificate: x509: certificate signed by unknown authority 2024-02-27T16:06:30Z ERROR Reconciler error {"controller": "k8sgpt", "controllerGroup": "core.k8sgpt.ai", "controllerKind": "K8sGPT", "K8sGPT": {"name":"k8sgpt-sample","namespace":"k8sgpt-operator-system"}, "namespace": "k8sgpt-operator-system", "name": "k8sgpt-sample", "reconcileID": "f8c17ec0-7935-44a4-bf8f-e12320b78aab", "error": "failed to call Analyze RPC: rpc error: code = Unknown desc = failed while calling AI provider openai: Post \"https://api.openai.com/v1/chat/completions\": tls: failed to verify certificate: x509: certificate signed by unknown authority"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(Controller).reconcileHandler /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.15.0/pkg/internal/controller/controller.go:324 sigs.k8s.io/controller-runtime/pkg/internal/controller.(Controller).processNextWorkItem /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.15.0/pkg/internal/controller/controller.go:265 sigs.k8s.io/controller-runtime/pkg/internal/controller.(Controller).Start.func2.2 /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.15.0/pkg/internal/controller/controller.go:226 Creating new client for 172.20.136.189:8080 Connection established between 172.20.136.189:8080 and localhost with time out of 1 seconds. Remote Address : 172.20.136.189:8080 K8sGPT address: 172.20.136.189:8080 Finished Reconciling k8sGPT with error: failed to call Analyze RPC: rpc error: code = Unknown desc = failed while calling AI provider openai: Post "https://api.openai.com/v1/chat/completions": tls: failed to verify certificate: x509: certificate signed by unknown authority 2024-02-27T16:06:31Z ERROR Reconciler error {"controller": "k8sgpt", "controllerGroup": "core.k8sgpt.ai", "controllerKind": "K8sGPT", "K8sGPT": {"name":"k8sgpt-sample","namespace":"k8sgpt-operator-system"}, "namespace": "k8sgpt-operator-system", "name": "k8sgpt-sample", "reconcileID": "a79dc8f0-550e-4f79-8740-c30c8df289d7", "error": "failed to call Analyze RPC: rpc error: code = Unknown desc = failed while calling AI provider openai: Post \"https://api.openai.com/v1/chat/completions\": tls: failed to verify certificate: x509: certificate signed by unknown authority"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(Controller).reconcileHandler /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.15.0/pkg/internal/controller/controller.go:324 sigs.k8s.io/controller-runtime/pkg/internal/controller.(Controller).processNextWorkItem /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.15.0/pkg/internal/controller/controller.go:265 sigs.k8s.io/controller-runtime/pkg/internal/controller.(Controller).Start.func2.2 /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.15.0/pkg/internal/controller/controller.go:226 Creating new client for 172.20.136.189:8080 Connection established between 172.20.136.189:8080 and localhost with time out of 1 seconds. Remote Address : 172.20.136.189:8080 K8sGPT address: 172.20.136.189:8080 Disabled AI backend openai due to failures exceeding max retries Finished Reconciling k8sGPT with error: failed to call Analyze RPC: rpc error: code = Unknown desc = failed while calling AI provider openai: Post "https://api.openai.com/v1/chat/completions": tls: failed to verify certificate: x509: certificate signed by unknown authority 2024-02-27T16:06:32Z ERROR Reconciler error {"controller": "k8sgpt", "controllerGroup": "core.k8sgpt.ai", "controllerKind": "K8sGPT", "K8sGPT": {"name":"k8sgpt-sample","namespace":"k8sgpt-operator-system"}, "namespace": "k8sgpt-operator-system", "name": "k8sgpt-sample", "reconcileID": "c9ce2d27-9558-4270-97b8-01f2c2aa7f7c", "error": "failed to call Analyze RPC: rpc error: code = Unknown desc = failed while calling AI provider openai: Post \"https://api.openai.com/v1/chat/completions\": tls: failed to verify certificate: x509: certificate signed by unknown authority"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(Controller).reconcileHandler /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.15.0/pkg/internal/controller/controller.go:324 sigs.k8s.io/controller-runtime/pkg/internal/controller.(Controller).processNextWorkItem /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.15.0/pkg/internal/controller/controller.go:265 sigs.k8s.io/controller-runtime/pkg/internal/controller.(Controller).Start.func2.2 /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.15.0/pkg/internal/controller/controller.go:226 Creating new client for 172.20.136.189:8080 Connection established between 172.20.136.189:8080 and localhost with time out of 1 seconds. Remote Address : 172.20.136.189:8080 K8sGPT address: 172.20.136.189:8080 Created result testrbalcloudnativeutilities Created result datadogdatadogagent584bb Created result testrbalcloudnativeutilities6d88f948cf7fb9v Created result sonarqubesonarqubesonarqube0 Created result datadogdatadogagent Created result defaultrbalcloudnativeutilities Created result kubesystemawsloadbalancerwebhookservice Created result sonarqubesonarqubesonarqube Created result argocdargocdapplicationcontroller Finished Reconciling k8sGPT Creating new client for 172.20.136.189:8080 Connection established between 172.20.136.189:8080 and localhost with time out of 1 seconds. Remote Address : 172.20.136.189:8080 K8sGPT address: 172.20.136.189:8080 Checking if datadogdatadogagent584bb is still relevant Checking if sonarqubesonarqubesonarqube0 is still relevant Checking if kubesystemawsloadbalancerwebhookservice is still relevant Checking if sonarqubesonarqubesonarqube is still relevant Checking if argocdargocdapplicationcontroller is still relevant Checking if testrbalcloudnativeutilities is still relevant Checking if datadogdatadogagent is still relevant Checking if defaultrbalcloudnativeutilities is still relevant Checking if testrbalcloudnativeutilities6d88f948cf7fb9v is still relevant Created result kubesystemawsloadbalancercontroller748d4978b5trbvj Finished Reconciling k8sGPT Creating new client for 172.20.136.189:8080 Connection established between 172.20.136.189:8080 and localhost with time out of 1 seconds. Remote Address : 172.20.136.189:8080 K8sGPT address: 172.20.136.189:8080 Checking if testrbalcloudnativeutilities is still relevant Checking if datadogdatadogagent584bb is still relevant Checking if sonarqubesonarqubesonarqube0 is still relevant Checking if kubesystemawsloadbalancerwebhookservice is still relevant Checking if sonarqubesonarqubesonarqube is still relevant Checking if argocdargocdapplicationcontroller is still relevant Checking if kubesystemawsloadbalancercontroller748d4978b5trbvj is still relevant Checking if testrbalcloudnativeutilities6d88f948cf7fb9v is still relevant Checking if datadogdatadogagent is still relevant Checking if defaultrbalcloudnativeutilities is still relevant Finished Reconciling k8sGPT Creating new client for 172.20.136.189:8080 Connection established between 172.20.136.189:8080 and localhost with time out of 1 seconds. Remote Address : 172.20.136.189:8080 K8sGPT address: 172.20.136.189:8080 Checking if datadogdatadogagent584bb is still relevant Checking if sonarqubesonarqubesonarqube0 is still relevant Checking if kubesystemawsloadbalancerwebhookservice is still relevant Checking if sonarqubesonarqubesonarqube is still relevant Checking if argocdargocdapplicationcontroller is still relevant Checking if kubesystemawsloadbalancercontroller748d4978b5trbvj is still relevant Checking if testrbalcloudnativeutilities is still relevant Checking if datadogdatadogagent is still relevant Checking if defaultrbalcloudnativeutilities is still relevant Checking if testrbalcloudnativeutilities6d88f948cf7fb9v is still relevant Finished Reconciling k8sGPT Creating new client for 172.20.136.189:8080 Connection established between 172.20.136.189:8080 and localhost with time out of 1 seconds. Remote Address : 172.20.136.189:8080 K8sGPT address: 172.20.136.189:8080 Checking if testrbalcloudnativeutilities is still relevant Checking if datadogdatadogagent584bb is still relevant Checking if sonarqubesonarqubesonarqube0 is still relevant Checking if kubesystemawsloadbalancerwebhookservice is still relevant Checking if sonarqubesonarqubesonarqube is still relevant Checking if argocdargocdapplicationcontroller is still relevant Checking if kubesystemawsloadbalancercontroller748d4978b5trbvj is still relevant Checking if testrbalcloudnativeutilities6d88f948cf7fb9v is still relevant Checking if datadogdatadogagent is still relevant Checking if defaultrbalcloudnativeutilities is still relevant Finished Reconciling k8sGPT Creating new client for 172.20.136.189:8080 Connection established between 172.20.136.189:8080 and localhost with time out of 1 seconds. Remote Address : 172.20.136.189:8080 K8sGPT address: 172.20.136.189:8080 Checking if datadogdatadogagent is still relevant Checking if defaultrbalcloudnativeutilities is still relevant Checking if testrbalcloudnativeutilities6d88f948cf7fb9v is still relevant Checking if datadogdatadogagent584bb is still relevant Checking if sonarqubesonarqubesonarqube0 is still relevant Checking if kubesystemawsloadbalancerwebhookservice is still relevant Checking if sonarqubesonarqubesonarqube is still relevant Checking if argocdargocdapplicationcontroller is still relevant Checking if kubesystemawsloadbalancercontroller748d4978b5trbvj is still relevant Checking if testrbalcloudnativeutilities is still relevant Finished Reconciling k8sGPT Creating new client for 172.20.136.189:8080 Connection established between 172.20.136.189:8080 and localhost with time out of 1 seconds. Remote Address : 172.20.136.189:8080 K8sGPT address: 172.20.136.189:8080 Checking if testrbalcloudnativeutilities6d88f948cf7fb9v is still relevant Checking if datadogdatadogagent is still relevant Checking if defaultrbalcloudnativeutilities is still relevant Checking if kubesystemawsloadbalancerwebhookservice is still relevant Checking if sonarqubesonarqubesonarqube is still relevant Checking if argocdargocdapplicationcontroller is still relevant Checking if kubesystemawsloadbalancercontroller748d4978b5trbvj is still relevant Checking if testrbalcloudnativeutilities is still relevant Checking if datadogdatadogagent584bb is still relevant Checking if sonarqubesonarqubesonarqube0 is still relevant Finished Reconciling k8sGPT Creating new client for 172.20.136.189:8080 Connection established between 172.20.136.189:8080 and localhost with time out of 1 seconds. Remote Address : 172.20.136.189:8080 K8sGPT address: 172.20.136.189:8080 Checking if testrbalcloudnativeutilities6d88f948cf7fb9v is still relevant Checking if datadogdatadogagent is still relevant Checking if defaultrbalcloudnativeutilities is still relevant Checking if testrbalcloudnativeutilities is still relevant Checking if datadogdatadogagent584bb is still relevant Checking if sonarqubesonarqubesonarqube0 is still relevant Checking if kubesystemawsloadbalancerwebhookservice is still relevant Checking if sonarqubesonarqubesonarqube is still relevant Checking if argocdargocdapplicationcontroller is still relevant Checking if kubesystemawsloadbalancercontroller748d4978b5trbvj is still relevant Finished Reconciling k8sGPT Creating new client for 172.20.136.189:8080 Connection established between 172.20.136.189:8080 and localhost with time out of 1 seconds. Remote Address : 172.20.136.189:8080 K8sGPT address: 172.20.136.189:8080 Checking if datadogdatadogagent is still relevant Checking if defaultrbalcloudnativeutilities is still relevant Checking if testrbalcloudnativeutilities6d88f948cf7fb9v is still relevant Checking if datadogdatadogagent584bb is still relevant Checking if kubesystemawsloadbalancerwebhookservice is still relevant Checking if sonarqubesonarqubesonarqube is still relevant Checking if argocdargocdapplicationcontroller is still relevant Checking if kubesystemawsloadbalancercontroller748d4978b5trbvj is still relevant Checking if testrbalcloudnativeutilities is still relevant Finished Reconciling k8sGPT Creating new client for 172.20.136.189:8080 Connection established between 172.20.136.189:8080 and localhost with time out of 1 seconds. Remote Address : 172.20.136.189:8080 K8sGPT address: 172.20.136.189:8080 Checking if defaultrbalcloudnativeutilities is still relevant Checking if datadogdatadogagent is still relevant Checking if kubesystemawsloadbalancerwebhookservice is still relevant Checking if sonarqubesonarqubesonarqube is still relevant Checking if argocdargocdapplicationcontroller is still relevant Checking if kubesystemawsloadbalancercontroller748d4978b5trbvj is still relevant Checking if testrbalcloudnativeutilities is still relevant Checking if datadogdatadogagent584bb is still relevant Created result sonarqubesonarqubesonarqube0 Created result testrbalcloudnativeutilities6d88f948cf7fb9v Finished Reconciling k8sGPT Creating new client for 172.20.136.189:8080 Connection established between 172.20.136.189:8080 and localhost with time out of 1 seconds. Remote Address : 172.20.136.189:8080 K8sGPT address: 172.20.136.189:8080 Checking if sonarqubesonarqubesonarqube0 is still relevant Checking if kubesystemawsloadbalancerwebhookservice is still relevant Checking if sonarqubesonarqubesonarqube is still relevant Checking if argocdargocdapplicationcontroller is still relevant Checking if kubesystemawsloadbalancercontroller748d4978b5trbvj is still relevant Checking if testrbalcloudnativeutilities is still relevant Checking if datadogdatadogagent584bb is still relevant Checking if defaultrbalcloudnativeutilities is still relevant Checking if testrbalcloudnativeutilities6d88f948cf7fb9v is still relevant Checking if datadogdatadogagent is still relevant Finished Reconciling k8sGPT Creating new client for 172.20.136.189:8080 Connection established between 172.20.136.189:8080 and localhost with time out of 1 seconds. Remote Address : 172.20.136.189:8080 K8sGPT address: 172.20.136.189:8080 Checking if testrbalcloudnativeutilities6d88f948cf7fb9v is still relevant Checking if datadogdatadogagent is still relevant Checking if defaultrbalcloudnativeutilities is still relevant Checking if argocdargocdapplicationcontroller is still relevant Checking if testrbalcloudnativeutilities is still relevant Checking if datadogdatadogagent584bb is still relevant Checking if sonarqubesonarqubesonarqube0 is still relevant Checking if kubesystemawsloadbalancerwebhookservice is still relevant Checking if sonarqubesonarqubesonarqube is still relevant Created result kubesystemawsloadbalancercontroller748d4978b5trbvj Finished Reconciling k8sGPT

Additional Information

Is there some way to disable this tls certificate verification ?

arbreezy commented 6 months ago

hey @blueacidification I am not sure if it's k8sgpt related, can you share you CR manifest please?

Make sure also your openai keys are valid or are you in a VPN, it looks related to your environment

blueacidification commented 6 months ago

Hi, yes, this is most probably on my side. I still would like to know if there's some way to disable tls certificate verification or to be able to mount a certificate in the pod. Neither am i able to exec inside the pod.

Thanks.

JuHyung-Son commented 6 months ago

would you provide, k8sgpt-operator value and your k8sgpt yaml?

blueacidification commented 6 months ago

sure :) k8sgpt.zip

JuHyung-Son commented 6 months ago

i guess there is no problem in yaml. and maybe backoff is triggered because of llm request failed. would you check controller logs. and is there log that llm request is paused?

blueacidification commented 6 months ago

k8sgpt-logs.zip i've added the logs in the description above, what exactly do you mean with llm request is paused ? i can not find any logs related to that. generated new logs and adding them here

JuHyung-Son commented 6 months ago

hmm yes, backoff is not triggered. then maybe you should check your llm request.

blueacidification commented 5 months ago

hi, can you please clarify whats llm request

JuHyung-Son commented 5 months ago

i mean your llm generation request might not response well.