search

devtron-labs / devtron

The only Kubernetes dashboard you need
https://devtron.ai
Apache License 2.0
4.45k stars 484 forks source link

Bug: Not able to configure gitops, getting authentication handshake failed. #4695

Open Kaushiks4 opened 9 months ago

Kaushiks4 commented 9 months ago

📜 Description

Unable to configure GitOps in the latest released version (v0.6.27). Getting "msg":"error in getting jwt token with username & password","err":"rpc error: code = Unavailable desc = connection error: desc = \"transport: authentication handshake failed: tls: failed to verify certificate: x509: certificate signed by unknown authority\"

Criticality

P0 - Critical/Blocking

Impact on Enterprise

  1. Unable to deploy applications as gitops cannot be configured in devtron.

👟 Steps to replicate the Issue

  1. Install latest version of devtron into a kubernetes cluster.
  2. Setup gitops with gitlab using PAT.
  3. Error

👍 Expected behavior

Gitops should be configured successfully.

👎 Actual Behavior

Getting the following error: [Feb 16 2024 12:49:08 GMT+0530] devtron-68566796c8-5l4k9: {"level":"debug","ts":1708067948.0019996,"caller":"pipeline/CdHandler.go:289","msg":"received argo cd pipelines stuck at kubectl apply synced stage","pipelines":null} [Feb 16 2024 12:49:08 GMT+0530] devtron-68566796c8-5l4k9: {"level":"debug","ts":1708067948.0031743,"caller":"pipeline/CdHandler.go:297","msg":"received argo appStore application stuck at kubectl apply synced stage","pipelines":null} [Feb 16 2024 12:49:13 GMT+0530] devtron-68566796c8-5l4k9: {"level":"info","ts":1708067953.3510125,"caller":"restHandler/GitOpsConfigRestHandler.go:95","msg":"request payload, CreateGitOpsConfig","err":null,"payload":{"provider":"GITLAB","username":"john@doe.com","token":"********","gitLabGroupId":"70435","gitHubOrgId":"","host":"https://gitlab.com/","active":true,"azureProjectName":"","bitBucketWorkspaceId":"","bitBucketProjectKey":"","gitRepoName":"","userEmailId":"","description":""}} [Feb 16 2024 12:49:15 GMT+0530] devtron-68566796c8-5l4k9: {"level":"debug","ts":1708067955.2478583,"caller":"util/GitServiceGitlab.go:76","msg":"gitlab config","config":{"GitlabGroupId":"70435","GitlabGroupPath":"testGroup","GitToken":"******","GitUserName":"john@doe.com","GitWorkingDir":"/tmp/gitops/","GithubOrganization":"","GitProvider":"GITLAB","GitHost":"https://gitlab.com/","AzureToken":"**********","AzureProject":"","BitbucketWorkspaceId":"","BitbucketProjectKey":""}} [Feb 16 2024 12:49:15 GMT+0530] devtron-68566796c8-5l4k9: {"level":"info","ts":1708067955.2479286,"caller":"util/GitService.go:170","msg":"client changed successfully","cfg":{"GitlabGroupId":"70435","GitlabGroupPath":"testGroup","GitToken":"********","GitUserName":"john@doe.com","GitWorkingDir":"/tmp/gitops/","GithubOrganization":"","GitProvider":"GITLAB","GitHost":"https://gitlab.com/","AzureToken":"********","AzureProject":"","BitbucketWorkspaceId":"","BitbucketProjectKey":""}} [Feb 16 2024 12:49:15 GMT+0530] devtron-68566796c8-5l4k9: {"level":"error","ts":1708067955.2487638,"caller":"util/ChartService.go:643","msg":"error in getting emailId from active gitOps config","err":"pg: no rows in result set","stacktrace":"github.com/devtron-labs/devtron/internal/util.(*ChartTemplateServiceImpl).GetUserEmailIdAndNameForGitOpsCommit\n\t/go/src/github.com/devtron-labs/devtron/internal/util/ChartService.go:643\ngithub.com/devtron-labs/devtron/pkg/gitops.(*GitOpsConfigServiceImpl).GitOpsValidateDryRun\n\t/go/src/github.com/devtron-labs/devtron/pkg/gitops/GitOpsConfigService.go:724\ngithub.com/devtron-labs/devtron/pkg/gitops.(*GitOpsConfigServiceImpl).ValidateAndCreateGitOpsConfig\n\t/go/src/github.com/devtron-labs/devtron/pkg/gitops/GitOpsConfigService.go:129\ngithub.com/devtron-labs/devtron/api/restHandler.GitOpsConfigRestHandlerImpl.CreateGitOpsConfig\n\t/go/src/github.com/devtron-labs/devtron/api/restHandler/GitOpsConfigRestHandler.go:102\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2122\ngithub.com/devtron-labs/common-lib/middlewares.Recovery.func1\n\t/go/src/github.com/devtron-labs/devtron/vendor/github.com/devtron-labs/common-lib/middlewares/recovery.go:26\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2122\ngithub.com/devtron-labs/devtron/internal/middleware.PrometheusMiddleware.func1\n\t/go/src/github.com/devtron-labs/devtron/internal/middleware/instrument.go:132\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2122\ngithub.com/devtron-labs/devtron/api/util.LoggingMiddlewareImpl.LoggingMiddleware.func1\n\t/go/src/github.com/devtron-labs/devtron/api/util/logger.go:66\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2122\ngithub.com/gorilla/mux.(*Router).ServeHTTP\n\t/go/src/github.com/devtron-labs/devtron/vendor/github.com/gorilla/mux/mux.go:210\ngithub.com/devtron-labs/authenticator/middleware.Authorizer.func1.1\n\t/go/src/github.com/devtron-labs/devtron/vendor/github.com/devtron-labs/authenticator/middleware/AuthMiddleware.go:71\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2122\nnet/http.serverHandler.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2936\nnet/http.(*conn).serve\n\t/usr/local/go/src/net/http/server.go:1995"} [Feb 16 2024 12:49:15 GMT+0530] devtron-68566796c8-5l4k9: {"level":"debug","ts":1708067955.2488568,"caller":"util/GitServiceGitlab.go:95","msg":"gitlab app create request ","name":"devtron-sample-repo-dryrun-dxx16o","description":""} [Feb 16 2024 12:49:15 GMT+0530] devtron-68566796c8-5l4k9: {"level":"debug","ts":1708067955.5890741,"caller":"util/GitServiceGitlab.go:216","msg":"gitlab get project err","pid":"testGroup/devtron-sample-repo-dryrun-dxx16o","err":"GET https://gitlab.com/api/v4/projects/testGroup/devtron-sample-repo-dryrun-dxx16o: 404 {message: 404 Project Not Found}"} [Feb 16 2024 12:49:17 GMT+0530] devtron-68566796c8-5l4k9: {"level":"info","ts":1708067957.4906795,"caller":"util/GitServiceGitlab.go:172","msg":"gitlab app created","name":"devtron-sample-repo-dryrun-dxx16o","url":"https://gitlab.com/testGroup/devtron-sample-repo-dryrun-dxx16o.git"} [Feb 16 2024 12:49:19 GMT+0530] devtron-68566796c8-5l4k9: {"level":"debug","ts":1708067959.4057343,"caller":"util/GitService.go:312","msg":"git checkout ","url":"https://gitlab.com/testGroup/devtron-sample-repo-dryrun-dxx16o.git","dir":"/ensure-clone/devtron-sample-repo-dryrun-dxx16o"} [Feb 16 2024 12:49:19 GMT+0530] devtron-68566796c8-5l4k9: {"level":"info","ts":1708067959.4057734,"caller":"util/GitCliUtil.go:133","msg":"git clone request","rootDir":"/tmp/gitops/ensure-clone/devtron-sample-repo-dryrun-dxx16o","remoteUrl":"https://gitlab.com/testGroup/devtron-sample-repo-dryrun-dxx16o.git","username":"john@doe.com"} [Feb 16 2024 12:49:19 GMT+0530] devtron-68566796c8-5l4k9: {"level":"debug","ts":1708067959.4067495,"caller":"util/GitCliUtil.go:33","msg":"git fetch ","location":"/tmp/gitops/ensure-clone/devtron-sample-repo-dryrun-dxx16o"} [Feb 16 2024 12:49:20 GMT+0530] devtron-68566796c8-5l4k9: {"level":"debug","ts":1708067960.2595594,"caller":"util/GitCliUtil.go:36","msg":"fetch output","root":"/tmp/gitops/ensure-clone/devtron-sample-repo-dryrun-dxx16o","opt":"From https://gitlab.com/testGroup/devtron-sample-repo-dryrun-dxx16o\n * [new branch] master -> origin/master","errMsg":"","error":null} [Feb 16 2024 12:49:20 GMT+0530] devtron-68566796c8-5l4k9: {"level":"warn","ts":1708067960.2596066,"caller":"util/GitCliUtil.go:140","msg":"git fetch completed, pulling master branch data from remote origin"} [Feb 16 2024 12:49:20 GMT+0530] devtron-68566796c8-5l4k9: {"level":"debug","ts":1708067960.2596138,"caller":"util/GitCliUtil.go:69","msg":"git branch ","location":"/tmp/gitops/ensure-clone/devtron-sample-repo-dryrun-dxx16o"} [Feb 16 2024 12:49:20 GMT+0530] devtron-68566796c8-5l4k9: {"level":"debug","ts":1708067960.2608624,"caller":"util/GitCliUtil.go:72","msg":"branch output","root":"/tmp/gitops/ensure-clone/devtron-sample-repo-dryrun-dxx16o","opt":"origin/master","errMsg":"","error":null} [Feb 16 2024 12:49:20 GMT+0530] devtron-68566796c8-5l4k9: {"level":"info","ts":1708067960.2608852,"caller":"util/GitCliUtil.go:147","msg":"total branch available in git repo","branches":["origin/master"]} [Feb 16 2024 12:49:20 GMT+0530] devtron-68566796c8-5l4k9: {"level":"debug","ts":1708067960.260894,"caller":"util/GitCliUtil.go:45","msg":"git pull ","location":"/tmp/gitops/ensure-clone/devtron-sample-repo-dryrun-dxx16o"} [Feb 16 2024 12:49:20 GMT+0530] devtron-68566796c8-5l4k9: {"level":"debug","ts":1708067960.997014,"caller":"util/GitCliUtil.go:48","msg":"pull output","root":"/tmp/gitops/ensure-clone/devtron-sample-repo-dryrun-dxx16o","opt":"From https://gitlab.com/testGroup/devtron-sample-repo-dryrun-dxx16o\n * branch master -> FETCH_HEAD","errMsg":"","error":null} [Feb 16 2024 12:49:20 GMT+0530] devtron-68566796c8-5l4k9: {"level":"info","ts":1708067960.997083,"caller":"util/GitServiceGitlab.go:201","msg":"gitlab ensureProjectAvailability clone passed","try count":1,"repoUrl":"https://gitlab.com/testGroup/devtron-sample-repo-dryrun-dxx16o.git"} [Feb 16 2024 12:49:20 GMT+0530] devtron-68566796c8-5l4k9: {"level":"debug","ts":1708067960.9971483,"caller":"util/GitService.go:312","msg":"git checkout ","url":"https://gitlab.com/testGroup/devtron-sample-repo-dryrun-dxx16o.git","dir":"devtron-sample-repo-dryrun-dxx16o-6906717860577957873"} [Feb 16 2024 12:49:20 GMT+0530] devtron-68566796c8-5l4k9: {"level":"info","ts":1708067960.9971642,"caller":"util/GitCliUtil.go:133","msg":"git clone request","rootDir":"/tmp/gitops/devtron-sample-repo-dryrun-dxx16o-6906717860577957873","remoteUrl":"https://gitlab.com/testGroup/devtron-sample-repo-dryrun-dxx16o.git","username":"john@doe.com"} [Feb 16 2024 12:49:20 GMT+0530] devtron-68566796c8-5l4k9: {"level":"debug","ts":1708067960.997634,"caller":"util/GitCliUtil.go:33","msg":"git fetch ","location":"/tmp/gitops/devtron-sample-repo-dryrun-dxx16o-6906717860577957873"} [Feb 16 2024 12:49:21 GMT+0530] devtron-68566796c8-5l4k9: {"level":"debug","ts":1708067961.8244953,"caller":"util/GitCliUtil.go:36","msg":"fetch output","root":"/tmp/gitops/devtron-sample-repo-dryrun-dxx16o-6906717860577957873","opt":"From https://gitlab.com/testGroup/devtron-sample-repo-dryrun-dxx16o\n * [new branch] master -> origin/master","errMsg":"","error":null} [Feb 16 2024 12:49:21 GMT+0530] devtron-68566796c8-5l4k9: {"level":"warn","ts":1708067961.8245409,"caller":"util/GitCliUtil.go:140","msg":"git fetch completed, pulling master branch data from remote origin"} [Feb 16 2024 12:49:21 GMT+0530] devtron-68566796c8-5l4k9: {"level":"debug","ts":1708067961.8245478,"caller":"util/GitCliUtil.go:69","msg":"git branch ","location":"/tmp/gitops/devtron-sample-repo-dryrun-dxx16o-6906717860577957873"} [Feb 16 2024 12:49:21 GMT+0530] devtron-68566796c8-5l4k9: {"level":"debug","ts":1708067961.825689,"caller":"util/GitCliUtil.go:72","msg":"branch output","root":"/tmp/gitops/devtron-sample-repo-dryrun-dxx16o-6906717860577957873","opt":"origin/master","errMsg":"","error":null} [Feb 16 2024 12:49:21 GMT+0530] devtron-68566796c8-5l4k9: {"level":"info","ts":1708067961.8257127,"caller":"util/GitCliUtil.go:147","msg":"total branch available in git repo","branches":["origin/master"]} [Feb 16 2024 12:49:21 GMT+0530] devtron-68566796c8-5l4k9: {"level":"debug","ts":1708067961.8257205,"caller":"util/GitCliUtil.go:45","msg":"git pull ","location":"/tmp/gitops/devtron-sample-repo-dryrun-dxx16o-6906717860577957873"} [Feb 16 2024 12:49:22 GMT+0530] devtron-68566796c8-5l4k9: {"level":"debug","ts":1708067962.5576692,"caller":"util/GitCliUtil.go:48","msg":"pull output","root":"/tmp/gitops/devtron-sample-repo-dryrun-dxx16o-6906717860577957873","opt":"From https://gitlab.com/testGroup/devtron-sample-repo-dryrun-dxx16o\n * branch master -> FETCH_HEAD","errMsg":"","error":null} [Feb 16 2024 12:49:22 GMT+0530] devtron-68566796c8-5l4k9: {"level":"debug","ts":1708067962.560679,"caller":"util/GitService.go:354","msg":"git hash","repo":"/tmp/gitops/devtron-sample-repo-dryrun-dxx16o-6906717860577957873","hash":"a0c8f69c7c3983832b11486a9d302fb0108485b8"} [Feb 16 2024 12:49:24 GMT+0530] devtron-68566796c8-5l4k9: {"level":"info","ts":1708067964.1384392,"caller":"util/GitServiceGitlab.go:147","msg":"deleting project ","gitlab project name":"devtron-sample-repo-dryrun-dxx16o"} [Feb 16 2024 12:49:24 GMT+0530] devtron-68566796c8-5l4k9: {"level":"debug","ts":1708067964.3441439,"caller":"k8s/K8sUtil.go:694","msg":"getting k8s rest config"} [Feb 16 2024 12:49:24 GMT+0530] devtron-68566796c8-5l4k9: time="2024-02-16T07:19:24Z" level=info msg="Starting configmap/secret informers" [Feb 16 2024 12:49:24 GMT+0530] devtron-68566796c8-5l4k9: time="2024-02-16T07:19:24Z" level=info msg="Configmap/secret informer synced" [Feb 16 2024 12:49:24 GMT+0530] devtron-68566796c8-5l4k9: {"level":"error","ts":1708067964.6606488,"caller":"argo/ArgoUserService.go:304","msg":"error in getting jwt token with username & password","err":"rpc error: code = Unavailable desc = connection error: desc = \"transport: authentication handshake failed: tls: failed to verify certificate: x509: certificate signed by unknown authority\"","stacktrace":"github.com/devtron-labs/devtron/util/argo.(*ArgoUserServiceImpl).createTokenForArgoCdUser\n\t/go/src/github.com/devtron-labs/devtron/util/argo/ArgoUserService.go:304\ngithub.com/devtron-labs/devtron/util/argo.(*ArgoUserServiceImpl).createNewArgoCdTokenForDevtron\n\t/go/src/github.com/devtron-labs/devtron/util/argo/ArgoUserService.go:146\ngithub.com/devtron-labs/devtron/util/argo.(*ArgoUserServiceImpl).GetOrUpdateArgoCdUserDetail\n\t/go/src/github.com/devtron-labs/devtron/util/argo/ArgoUserService.go:113\ngithub.com/devtron-labs/devtron/pkg/gitops.(*GitOpsConfigServiceImpl).ValidateAndCreateGitOpsConfig\n\t/go/src/github.com/devtron-labs/devtron/pkg/gitops/GitOpsConfigService.go:132\ngithub.com/devtron-labs/devtron/api/restHandler.GitOpsConfigRestHandlerImpl.CreateGitOpsConfig\n\t/go/src/github.com/devtron-labs/devtron/api/restHandler/GitOpsConfigRestHandler.go:102\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2122\ngithub.com/devtron-labs/common-lib/middlewares.Recovery.func1\n\t/go/src/github.com/devtron-labs/devtron/vendor/github.com/devtron-labs/common-lib/middlewares/recovery.go:26\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2122\ngithub.com/devtron-labs/devtron/internal/middleware.PrometheusMiddleware.func1\n\t/go/src/github.com/devtron-labs/devtron/internal/middleware/instrument.go:132\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2122\ngithub.com/devtron-labs/devtron/api/util.LoggingMiddlewareImpl.LoggingMiddleware.func1\n\t/go/src/github.com/devtron-labs/devtron/api/util/logger.go:66\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2122\ngithub.com/gorilla/mux.(*Router).ServeHTTP\n\t/go/src/github.com/devtron-labs/devtron/vendor/github.com/gorilla/mux/mux.go:210\ngithub.com/devtron-labs/authenticator/middleware.Authorizer.func1.1\n\t/go/src/github.com/devtron-labs/devtron/vendor/github.com/devtron-labs/authenticator/middleware/AuthMiddleware.go:71\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2122\nnet/http.serverHandler.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2936\nnet/http.(*conn).serve\n\t/usr/local/go/src/net/http/server.go:1995"} [Feb 16 2024 12:49:24 GMT+0530] devtron-68566796c8-5l4k9: {"level":"error","ts":1708067964.6607323,"caller":"argo/ArgoUserService.go:148","msg":"error in creating new argocd user","err":"rpc error: code = Unavailable desc = connection error: desc = \"transport: authentication handshake failed: tls: failed to verify certificate: x509: certificate signed by unknown authority\"","stacktrace":"github.com/devtron-labs/devtron/util/argo.(*ArgoUserServiceImpl).createNewArgoCdTokenForDevtron\n\t/go/src/github.com/devtron-labs/devtron/util/argo/ArgoUserService.go:148\ngithub.com/devtron-labs/devtron/util/argo.(*ArgoUserServiceImpl).GetOrUpdateArgoCdUserDetail\n\t/go/src/github.com/devtron-labs/devtron/util/argo/ArgoUserService.go:113\ngithub.com/devtron-labs/devtron/pkg/gitops.(*GitOpsConfigServiceImpl).ValidateAndCreateGitOpsConfig\n\t/go/src/github.com/devtron-labs/devtron/pkg/gitops/GitOpsConfigService.go:132\ngithub.com/devtron-labs/devtron/api/restHandler.GitOpsConfigRestHandlerImpl.CreateGitOpsConfig\n\t/go/src/github.com/devtron-labs/devtron/api/restHandler/GitOpsConfigRestHandler.go:102\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2122\ngithub.com/devtron-labs/common-lib/middlewares.Recovery.func1\n\t/go/src/github.com/devtron-labs/devtron/vendor/github.com/devtron-labs/common-lib/middlewares/recovery.go:26\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2122\ngithub.com/devtron-labs/devtron/internal/middleware.PrometheusMiddleware.func1\n\t/go/src/github.com/devtron-labs/devtron/internal/middleware/instrument.go:132\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2122\ngithub.com/devtron-labs/devtron/api/util.LoggingMiddlewareImpl.LoggingMiddleware.func1\n\t/go/src/github.com/devtron-labs/devtron/api/util/logger.go:66\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2122\ngithub.com/gorilla/mux.(*Router).ServeHTTP\n\t/go/src/github.com/devtron-labs/devtron/vendor/github.com/gorilla/mux/mux.go:210\ngithub.com/devtron-labs/authenticator/middleware.Authorizer.func1.1\n\t/go/src/github.com/devtron-labs/devtron/vendor/github.com/devtron-labs/authenticator/middleware/AuthMiddleware.go:71\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2122\nnet/http.serverHandler.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2936\nnet/http.(*conn).serve\n\t/usr/local/go/src/net/http/server.go:1995"} [Feb 16 2024 12:49:24 GMT+0530] devtron-68566796c8-5l4k9: {"level":"error","ts":1708067964.660774,"caller":"argo/ArgoUserService.go:115","msg":"error in creating new argo cd token for devtron","err":"rpc error: code = Unavailable desc = connection error: desc = \"transport: authentication handshake failed: tls: failed to verify certificate: x509: certificate signed by unknown authority\"","stacktrace":"github.com/devtron-labs/devtron/util/argo.(*ArgoUserServiceImpl).GetOrUpdateArgoCdUserDetail\n\t/go/src/github.com/devtron-labs/devtron/util/argo/ArgoUserService.go:115\ngithub.com/devtron-labs/devtron/pkg/gitops.(*GitOpsConfigServiceImpl).ValidateAndCreateGitOpsConfig\n\t/go/src/github.com/devtron-labs/devtron/pkg/gitops/GitOpsConfigService.go:132\ngithub.com/devtron-labs/devtron/api/restHandler.GitOpsConfigRestHandlerImpl.CreateGitOpsConfig\n\t/go/src/github.com/devtron-labs/devtron/api/restHandler/GitOpsConfigRestHandler.go:102\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2122\ngithub.com/devtron-labs/common-lib/middlewares.Recovery.func1\n\t/go/src/github.com/devtron-labs/devtron/vendor/github.com/devtron-labs/common-lib/middlewares/recovery.go:26\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2122\ngithub.com/devtron-labs/devtron/internal/middleware.PrometheusMiddleware.func1\n\t/go/src/github.com/devtron-labs/devtron/internal/middleware/instrument.go:132\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2122\ngithub.com/devtron-labs/devtron/api/util.LoggingMiddlewareImpl.LoggingMiddleware.func1\n\t/go/src/github.com/devtron-labs/devtron/api/util/logger.go:66\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2122\ngithub.com/gorilla/mux.(*Router).ServeHTTP\n\t/go/src/github.com/devtron-labs/devtron/vendor/github.com/gorilla/mux/mux.go:210\ngithub.com/devtron-labs/authenticator/middleware.Authorizer.func1.1\n\t/go/src/github.com/devtron-labs/devtron/vendor/github.com/devtron-labs/authenticator/middleware/AuthMiddleware.go:71\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2122\nnet/http.serverHandler.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2936\nnet/http.(*conn).serve\n\t/usr/local/go/src/net/http/server.go:1995"} [Feb 16 2024 12:49:24 GMT+0530] devtron-68566796c8-5l4k9: {"level":"debug","ts":1708067964.6608114,"caller":"gitops/GitOpsConfigService.go:176","msg":"gitops create request","req":{"provider":"GITLAB","username":"john@doe.com","token":"*******","gitLabGroupId":"70435","gitHubOrgId":"","host":"https://gitlab.com/","active":true,"azureProjectName":"","bitBucketWorkspaceId":"","bitBucketProjectKey":"","gitRepoName":"devtron-sample-repo-dryrun-dxx16o","userEmailId":"devtron-bot@devtron.ai","description":""}} [Feb 16 2024 12:49:24 GMT+0530] devtron-68566796c8-5l4k9: {"level":"debug","ts":1708067964.662436,"caller":"k8s/K8sUtil.go:694","msg":"getting k8s rest config"} [Feb 16 2024 12:49:26 GMT+0530] devtron-68566796c8-5l4k9: {"level":"error","ts":1708067966.9167428,"caller":"gitops/GitOpsConfigService.go:341","msg":"Error while upserting cluster in acd","clusterName":"default_cluster","err":"rpc error: code = Unavailable desc = connection error: desc = \"transport: authentication handshake failed: tls: failed to verify certificate: x509: certificate signed by unknown authority\"","stacktrace":"github.com/devtron-labs/devtron/pkg/gitops.(*GitOpsConfigServiceImpl).CreateGitOpsConfig\n\t/go/src/github.com/devtron-labs/devtron/pkg/gitops/GitOpsConfigService.go:341\ngithub.com/devtron-labs/devtron/pkg/gitops.(*GitOpsConfigServiceImpl).ValidateAndCreateGitOpsConfig\n\t/go/src/github.com/devtron-labs/devtron/pkg/gitops/GitOpsConfigService.go:134\ngithub.com/devtron-labs/devtron/api/restHandler.GitOpsConfigRestHandlerImpl.CreateGitOpsConfig\n\t/go/src/github.com/devtron-labs/devtron/api/restHandler/GitOpsConfigRestHandler.go:102\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2122\ngithub.com/devtron-labs/common-lib/middlewares.Recovery.func1\n\t/go/src/github.com/devtron-labs/devtron/vendor/github.com/devtron-labs/common-lib/middlewares/recovery.go:26\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2122\ngithub.com/devtron-labs/devtron/internal/middleware.PrometheusMiddleware.func1\n\t/go/src/github.com/devtron-labs/devtron/internal/middleware/instrument.go:132\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2122\ngithub.com/devtron-labs/devtron/api/util.LoggingMiddlewareImpl.LoggingMiddleware.func1\n\t/go/src/github.com/devtron-labs/devtron/api/util/logger.go:66\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2122\ngithub.com/gorilla/mux.(*Router).ServeHTTP\n\t/go/src/github.com/devtron-labs/devtron/vendor/github.com/gorilla/mux/mux.go:210\ngithub.com/devtron-labs/authenticator/middleware.Authorizer.func1.1\n\t/go/src/github.com/devtron-labs/devtron/vendor/github.com/devtron-labs/authenticator/middleware/AuthMiddleware.go:71\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2122\nnet/http.serverHandler.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2936\nnet/http.(*conn).serve\n\t/usr/local/go/src/net/http/server.go:1995"} [Feb 16 2024 12:49:26 GMT+0530] devtron-68566796c8-5l4k9: {"level":"error","ts":1708067966.917075,"caller":"gitops/GitOpsConfigService.go:136","msg":"service err, SaveGitRepoConfig","err":"rpc error: code = Unavailable desc = connection error: desc = \"transport: authentication handshake failed: tls: failed to verify certificate: x509: certificate signed by unknown authority\"","payload":{"provider":"GITLAB","username":"john@doe.com","token":"******","gitLabGroupId":"70435","gitHubOrgId":"","host":"https://gitlab.com/testGroup","active":true,"azureProjectName":"","bitBucketWorkspaceId":"","bitBucketProjectKey":"","gitRepoName":"devtron-sample-repo-dryrun-dxx16o","userEmailId":"devtron-bot@devtron.ai","description":""},"stacktrace":"github.com/devtron-labs/devtron/pkg/gitops.(*GitOpsConfigServiceImpl).ValidateAndCreateGitOpsConfig\n\t/go/src/github.com/devtron-labs/devtron/pkg/gitops/GitOpsConfigService.go:136\ngithub.com/devtron-labs/devtron/api/restHandler.GitOpsConfigRestHandlerImpl.CreateGitOpsConfig\n\t/go/src/github.com/devtron-labs/devtron/api/restHandler/GitOpsConfigRestHandler.go:102\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2122\ngithub.com/devtron-labs/common-lib/middlewares.Recovery.func1\n\t/go/src/github.com/devtron-labs/devtron/vendor/github.com/devtron-labs/common-lib/middlewares/recovery.go:26\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2122\ngithub.com/devtron-labs/devtron/internal/middleware.PrometheusMiddleware.func1\n\t/go/src/github.com/devtron-labs/devtron/internal/middleware/instrument.go:132\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2122\ngithub.com/devtron-labs/devtron/api/util.LoggingMiddlewareImpl.LoggingMiddleware.func1\n\t/go/src/github.com/devtron-labs/devtron/api/util/logger.go:66\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2122\ngithub.com/gorilla/mux.(*Router).ServeHTTP\n\t/go/src/github.com/devtron-labs/devtron/vendor/github.com/gorilla/mux/mux.go:210\ngithub.com/devtron-labs/authenticator/middleware.Authorizer.func1.1\n\t/go/src/github.com/devtron-labs/devtron/vendor/github.com/devtron-labs/authenticator/middleware/AuthMiddleware.go:71\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2122\nnet/http.serverHandler.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2936\nnet/http.(*conn).serve\n\t/usr/local/go/src/net/http/server.go:1995"} [Feb 16 2024 12:49:26 GMT+0530] devtron-68566796c8-5l4k9: {"level":"error","ts":1708067966.9171524,"caller":"restHandler/GitOpsConfigRestHandler.go:104","msg":"service err, SaveGitRepoConfig","err":"rpc error: code = Unavailable desc = connection error: desc = \"transport: authentication handshake failed: tls: failed to verify certificate: x509: certificate signed by unknown authority\"","payload":{"provider":"GITLAB","username":"john@doe.com","token":"******","gitLabGroupId":"70435","gitHubOrgId":"","host":"https://gitlab.com/testGroup","active":true,"azureProjectName":"","bitBucketWorkspaceId":"","bitBucketProjectKey":"","gitRepoName":"devtron-sample-repo-dryrun-dxx16o","userEmailId":"devtron-bot@devtron.ai","description":""},"stacktrace":"github.com/devtron-labs/devtron/api/restHandler.GitOpsConfigRestHandlerImpl.CreateGitOpsConfig\n\t/go/src/github.com/devtron-labs/devtron/api/restHandler/GitOpsConfigRestHandler.go:104\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2122\ngithub.com/devtron-labs/common-lib/middlewares.Recovery.func1\n\t/go/src/github.com/devtron-labs/devtron/vendor/github.com/devtron-labs/common-lib/middlewares/recovery.go:26\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2122\ngithub.com/devtron-labs/devtron/internal/middleware.PrometheusMiddleware.func1\n\t/go/src/github.com/devtron-labs/devtron/internal/middleware/instrument.go:132\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2122\ngithub.com/devtron-labs/devtron/api/util.LoggingMiddlewareImpl.LoggingMiddleware.func1\n\t/go/src/github.com/devtron-labs/devtron/api/util/logger.go:66\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2122\ngithub.com/gorilla/mux.(*Router).ServeHTTP\n\t/go/src/github.com/devtron-labs/devtron/vendor/github.com/gorilla/mux/mux.go:210\ngithub.com/devtron-labs/authenticator/middleware.Authorizer.func1.1\n\t/go/src/github.com/devtron-labs/devtron/vendor/github.com/devtron-labs/authenticator/middleware/AuthMiddleware.go:71\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2122\nnet/http.serverHandler.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2936\nnet/http.(*conn).serve\n\t/usr/local/go/src/net/http/server.go:1995"} [Feb 16 2024 12:49:26 GMT+0530] devtron-68566796c8-5l4k9: {"level":"info","ts":1708067966.9171932,"caller":"common/apiError.go:118","msg":"ERROR RES","TYPE":"API-ERROR","RES":500,"ERROR-MSG":[{"code":"000","internalMessage":"[{rpc error: code = Unavailable desc = connection error: desc = \"transport: authentication handshake failed: tls: failed to verify certificate: x509: certificate signed by unknown authority\"}]","userMessage":"rpc error: code = Unavailable desc = connection error: desc = \"transport: authentication handshake failed: tls: failed to verify certificate: x509: certificate signed by unknown authority\""}],"err":null} [Feb 16 2024 12:49:26 GMT+0530] devtron-68566796c8-5l4k9: 2024/02/16 07:19:26 http: superfluous response.WriteHeader call from github.com/devtron-labs/devtron/internal/middleware.(*responseWriterDelegator).WriteHeader (delegator.go:69) [Feb 16 2024 12:49:26 GMT+0530] devtron-68566796c8-5l4k9: 2024/02/16 07:19:26 AUDIT_LOG: urlPath: /orchestrator/gitops/config, queryParams: ,updatedBy: admin, updatedOn: 2024-02-16 07:19:13.349783695 +0000 UTC m=+125.115911002, apiResponseCode: 200,requestPayload: {"id":null,"provider":"GITLAB","username":"john@doe.com","host":"https://gitlab.com/","token":"******","gitLabGroupId":"70435","gitHubOrgId":"","azureProjectName":"","bitBucketWorkspaceId":"","bitBucketProjectKey":"","active":true}

☸ Kubernetes version

OKE v.1.28.2

Cloud provider

Oracle Cloud Infrastructure

🌍 Browser

Chrome

🧱 Your Environment

I use this as our internal developer platform.

✅ Proposed Solution

No fix found yet, but the /validate api to configure gitops works.

👀 Have you spent some time to check if this issue has been raised before?

🏢 Have you read the Code of Conduct?

AB#8584

systemsdt commented 9 months ago

Potential duplicates:

systemsdt commented 9 months ago

Potential duplicates:

systemsdt commented 9 months ago

Potential duplicates:

Kaushiks4 commented 9 months ago

I can confirm this is not duplicate of #4668

abhibhaw commented 6 months ago

Unable to reproduce at our end. Is this resolved or still exist @Kaushiks4 @badal773 ?

abhibhaw commented 6 months ago

Closing this due to inactivity, please feel free to reopen if this issue still exists

saikumar843expleo commented 5 months ago

We are having the same issue.

Steps:

guzmanf commented 3 months ago

We are having the same issue.

Steps:

error:

image

image

image

image

ideas?....

regards

romosa commented 3 weeks ago

I am configure gitops as well using bitbucket but getting the same error. Anyone found a solution?

image

Edit: Additional info: Devtron: 0.7.3 Kubernetes: 1.29.8.eks image image

{"level":"error","ts":1730985847.3232462,"caller":"config/GitOpsConfigReadService.go:96","msg":"error in getting emailId from active gitOps config","err":"pg: no rows in result set","stacktrace":"github.com/devtron-labs/devtron/pkg/deployment/gitOps/config.(*GitOpsConfigReadServiceImpl).GetUserEmailIdAndNameForGitOpsCommit\n\t/go/src/github.com/devtron-labs/devtron/pkg/deployment/gitOps/config/GitOpsConfigReadService.go:96\ngithub.com/devtron-labs/devtron/pkg/deployment/gitOps/validation.(*GitOpsValidationServiceImpl).GitOpsValidateDryRun\n\t/go/src/github.com/devtron-labs/devtron/pkg/deployment/gitOps/validation/gitOpsValidationService.go:101\ngithub.com/devtron-labs/devtron/pkg/gitops.(*GitOpsConfigServiceImpl).GitOpsValidateDryRun\n\t/go/src/github.com/devtron-labs/devtron/pkg/gitops/GitOpsConfigService.go:841\ngithub.com/devtron-labs/devtron/pkg/gitops.(*GitOpsConfigServiceImpl).ValidateAndCreateGitOpsConfig\n\t/go/src/github.com/devtron-labs/devtron/pkg/gitops/GitOpsConfigService.go:110\ngithub.com/devtron-labs/devtron/api/restHandler.GitOpsConfigRestHandlerImpl.CreateGitOpsConfig\n\t/go/src/github.com/devtron-labs/devtron/api/restHandler/GitOpsConfigRestHandler.go:99\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2141\ngithub.com/devtron-labs/common-lib/middlewares.Recovery.func1\n\t/go/src/github.com/devtron-labs/devtron/vendor/github.com/devtron-labs/common-lib/middlewares/recovery.go:52\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2141\ngithub.com/devtron-labs/devtron/internal/middleware.PrometheusMiddleware.func1\n\t/go/src/github.com/devtron-labs/devtron/internal/middleware/instrument.go:137\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2141\ngithub.com/devtron-labs/devtron/api/util.(*LoggingMiddlewareImpl).LoggingMiddleware.LoggingMiddlewareImpl.LoggingMiddleware.func1\n\t/go/src/github.com/devtron-labs/devtron/api/util/logger.go:84\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2141\ngithub.com/gorilla/mux.(*Router).ServeHTTP\n\t/go/src/github.com/devtron-labs/devtron/vendor/github.com/gorilla/mux/mux.go:210\nmain.(*App).Start.(*App).Start.Authorizer.func1.func2\n\t/go/src/github.com/devtron-labs/devtron/vendor/github.com/devtron-labs/authenticator/middleware/AuthMiddleware.go:91\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2141\nnet/http.serverHandler.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2943\nnet/http.(*conn).serve\n\t/usr/local/go/src/net/http/server.go:2014"}
{"level":"info","ts":1730985851.0785055,"caller":"git/GitServiceBitbucket.go:149","msg":"repo created ","repoUrl":"https://bitbucket.org/devopsgitops/devtron-sample-repo-dryrun-b8xam0.git"}
{"level":"info","ts":1730985851.266651,"caller":"git/GitServiceBitbucket.go:207","msg":"repo validated successfully on https"}
{"level":"info","ts":1730985853.7987149,"caller":"git/GitOpsHelper.go:167","msg":"total branch available in git repo","branch length":1}
{"level":"info","ts":1730985854.3371246,"caller":"git/GitServiceBitbucket.go:256","msg":"ensureProjectAvailability clone passed Bitbucket","try count":0,"repoUrl":"https://bitbucket.org/devopsgitops/devtron-sample-repo-dryrun-b8xam0.git"}
{"level":"info","ts":1730985854.9243717,"caller":"git/GitOpsHelper.go:167","msg":"total branch available in git repo","branch length":1}
{"level":"error","ts":1730985857.9748063,"caller":"connection/Connection.go:124","msg":"error on get acd connection","err":"server.secretkey is missing","stacktrace":"github.com/devtron-labs/devtron/client/argocdServer/connection.(*ArgoCDConnectionManagerImpl).getArgoCdSettings\n\t/go/src/github.com/devtron-labs/devtron/client/argocdServer/connection/Connection.go:124\ngithub.com/devtron-labs/devtron/client/argocdServer/connection.(*ArgoCDConnectionManagerImpl).GetConnection\n\t/go/src/github.com/devtron-labs/devtron/client/argocdServer/connection/Connection.go:70\ngithub.com/devtron-labs/devtron/client/argocdServer/session.NewSessionServiceClient\n\t/go/src/github.com/devtron-labs/devtron/client/argocdServer/session/Session.go:36\ngithub.com/devtron-labs/devtron/util/argo.(*ArgoUserServiceImpl).passwordLogin\n\t/go/src/github.com/devtron-labs/devtron/util/argo/ArgoUserService.go:366\ngithub.com/devtron-labs/devtron/util/argo.(*ArgoUserServiceImpl).createTokenForArgoCdUser\n\t/go/src/github.com/devtron-labs/devtron/util/argo/ArgoUserService.go:339\ngithub.com/devtron-labs/devtron/util/argo.(*ArgoUserServiceImpl).createNewArgoCdTokenForDevtron\n\t/go/src/github.com/devtron-labs/devtron/util/argo/ArgoUserService.go:189\ngithub.com/devtron-labs/devtron/util/argo.(*ArgoUserServiceImpl).GetOrUpdateArgoCdUserDetail\n\t/go/src/github.com/devtron-labs/devtron/util/argo/ArgoUserService.go:156\ngithub.com/devtron-labs/devtron/pkg/gitops.(*GitOpsConfigServiceImpl).ValidateAndCreateGitOpsConfig\n\t/go/src/github.com/devtron-labs/devtron/pkg/gitops/GitOpsConfigService.go:113\ngithub.com/devtron-labs/devtron/api/restHandler.GitOpsConfigRestHandlerImpl.CreateGitOpsConfig\n\t/go/src/github.com/devtron-labs/devtron/api/restHandler/GitOpsConfigRestHandler.go:99\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2141\ngithub.com/devtron-labs/common-lib/middlewares.Recovery.func1\n\t/go/src/github.com/devtron-labs/devtron/vendor/github.com/devtron-labs/common-lib/middlewares/recovery.go:52\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2141\ngithub.com/devtron-labs/devtron/internal/middleware.PrometheusMiddleware.func1\n\t/go/src/github.com/devtron-labs/devtron/internal/middleware/instrument.go:137\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2141\ngithub.com/devtron-labs/devtron/api/util.(*LoggingMiddlewareImpl).LoggingMiddleware.LoggingMiddlewareImpl.LoggingMiddleware.func1\n\t/go/src/github.com/devtron-labs/devtron/api/util/logger.go:84\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2141\ngithub.com/gorilla/mux.(*Router).ServeHTTP\n\t/go/src/github.com/devtron-labs/devtron/vendor/github.com/gorilla/mux/mux.go:210\nmain.(*App).Start.(*App).Start.Authorizer.func1.func2\n\t/go/src/github.com/devtron-labs/devtron/vendor/github.com/devtron-labs/authenticator/middleware/AuthMiddleware.go:91\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2141\nnet/http.serverHandler.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2943\nnet/http.(*conn).serve\n\t/usr/local/go/src/net/http/server.go:2014"}
2024/11/07 13:24:17 DEVTRON_PANIC_RECOVERrecovered from panicerrruntime error: invalid memory address or nil pointer dereferencestackgoroutine 36084 [running]:
runtime/debug.Stack()
        /usr/local/go/src/runtime/debug/stack.go:24 +0x5e
github.com/devtron-labs/common-lib/middlewares.Recovery.func1.1()
        /go/src/github.com/devtron-labs/devtron/vendor/github.com/devtron-labs/common-lib/middlewares/recovery.go:42 +0x10a
panic({0x55aa180?, 0x8ba8ba0?})
        /usr/local/go/src/runtime/panic.go:914 +0x21f
github.com/devtron-labs/devtron/client/argocdServer/connection.(*ArgoCDConnectionManagerImpl).GetConnection(0xc00105af90, {0x0, 0x0})
        /go/src/github.com/devtron-labs/devtron/client/argocdServer/connection/Connection.go:72 +0x111
github.com/devtron-labs/devtron/client/argocdServer/session.NewSessionServiceClient({0x67927e0?, 0xc00105af90?})
        /go/src/github.com/devtron-labs/devtron/client/argocdServer/session/Session.go:36 +0x29
github.com/devtron-labs/devtron/util/argo.(*ArgoUserServiceImpl).passwordLogin(0xc00110cbd0?, {0xc00122ebf6, 0x7}, {0xc00122ec00, 0x10})
        /go/src/github.com/devtron-labs/devtron/util/argo/ArgoUserService.go:366 +0x35
github.com/devtron-labs/devtron/util/argo.(*ArgoUserServiceImpl).createTokenForArgoCdUser(0xc000179030, {0xc00122ebf6, 0x7}, {0xc00122ec00?, 0xc00174a240?})
        /go/src/github.com/devtron-labs/devtron/util/argo/ArgoUserService.go:339 +0x3f
github.com/devtron-labs/devtron/util/argo.(*ArgoUserServiceImpl).createNewArgoCdTokenForDevtron(0xc000179030, {0xc00122ebf6?, 0x10?}, {0xc00122ec00?, 0x0?}, 0x0?, 0x5550240?)
        /go/src/github.com/devtron-labs/devtron/util/argo/ArgoUserService.go:189 +0x45
github.com/devtron-labs/devtron/util/argo.(*ArgoUserServiceImpl).GetOrUpdateArgoCdUserDetail(0xc000179030)
        /go/src/github.com/devtron-labs/devtron/util/argo/ArgoUserService.go:156 +0x4f5
github.com/devtron-labs/devtron/pkg/gitops.(*GitOpsConfigServiceImpl).ValidateAndCreateGitOpsConfig(0xc000b10600, 0xc0001de1e0)
        /go/src/github.com/devtron-labs/devtron/pkg/gitops/GitOpsConfigService.go:113 +0x10b
github.com/devtron-labs/devtron/api/restHandler.GitOpsConfigRestHandlerImpl.CreateGitOpsConfig({0xc00091e0f8, {0x67d7e10, 0xc000b10600}, {0x6800960, 0xc0008a2600}, 0xc00090a5a0, {0x67d7558, 0xc0001594a0}, {0x67df4d0, 0xc000db9110}}, ...)
        /go/src/github.com/devtron-labs/devtron/api/restHandler/GitOpsConfigRestHandler.go:99 +0x584
net/http.HandlerFunc.ServeHTTP(0x6794760?, {0x7fe17841ba78?, 0xc00075e870?}, 0xc001105290?)
        /usr/local/go/src/net/http/server.go:2141 +0x29
github.com/devtron-labs/common-lib/middlewares.Recovery.func1({0x7fe17841ba78?, 0xc00075e870?}, 0x7fe17841ba00?)
        /go/src/github.com/devtron-labs/devtron/vendor/github.com/devtron-labs/common-lib/middlewares/recovery.go:52 +0x78
net/http.HandlerFunc.ServeHTTP(0x7fe17841ba78?, {0x7fe17841ba78?, 0xc00075e870?}, 0xc001180e00?)
        /usr/local/go/src/net/http/server.go:2141 +0x29
github.com/devtron-labs/devtron/internal/middleware.PrometheusMiddleware.func1({0x7fe17841ba78, 0xc00075e730}, 0xc000c3ec00)
        /go/src/github.com/devtron-labs/devtron/internal/middleware/instrument.go:137 +0x242
net/http.HandlerFunc.ServeHTTP(0xc001105230?, {0x7fe17841ba78?, 0xc00075e730?}, 0xc00152aa00?)
        /usr/local/go/src/net/http/server.go:2141 +0x29
github.com/devtron-labs/devtron/api/util.(*LoggingMiddlewareImpl).LoggingMiddleware.LoggingMiddlewareImpl.LoggingMiddleware.func1({0x67b6650?, 0xc0005fe9a0?}, 0xc000c3ec00)
        /go/src/github.com/devtron-labs/devtron/api/util/logger.go:84 +0x3b8
net/http.HandlerFunc.ServeHTTP(0xc000c3eb00?, {0x67b6650?, 0xc0005fe9a0?}, 0xc000c31900?)
        /usr/local/go/src/net/http/server.go:2141 +0x29
github.com/gorilla/mux.(*Router).ServeHTTP(0xc0012bc480, {0x67b6650, 0xc0005fe9a0}, 0xc000c3ea00)
        /go/src/github.com/devtron-labs/devtron/vendor/github.com/gorilla/mux/mux.go:210 +0x1c5
main.(*App).Start.(*App).Start.Authorizer.func1.func2({0x67b6650, 0xc0005fe9a0}, 0xc000c3ea00)
        /go/src/github.com/devtron-labs/devtron/vendor/github.com/devtron-labs/authenticator/middleware/AuthMiddleware.go:91 +0x479
net/http.HandlerFunc.ServeHTTP(0x1c5f185?, {0x67b6650?, 0xc0005fe9a0?}, 0xc0005fe901?)
        /usr/local/go/src/net/http/server.go:2141 +0x29
net/http.serverHandler.ServeHTTP({0x67a4fe8?}, {0x67b6650?, 0xc0005fe9a0?}, 0x6?)
        /usr/local/go/src/net/http/server.go:2943 +0x8e
net/http.(*conn).serve(0xc001b55dd0, {0x67cf058, 0xc001068210})
        /usr/local/go/src/net/http/server.go:2014 +0x5f4
created by net/http.(*Server).Serve in goroutine 1
        /usr/local/go/src/net/http/server.go:3091 +0x5cb
abhibhaw commented 3 weeks ago

Is the gitops module installed? @romosa ?

romosa commented 3 weeks ago

Is the gitops module installed? @romosa ?

Hi @abhibhaw

I followed this instructions and added additional configurations for ingress and s3. https://docs.devtron.ai/install/install-devtron-with-cicd

EDIT:

I did not installed with argocd because we already have argocd installed in the cluster.

helm install devtron devtron/devtron-operator --create-namespace --namespace devtroncd \
--set installer.modules={cicd} \
--set configs.BLOB_STORAGE_PROVIDER=S3 \
--set configs.DEFAULT_CACHE_BUCKET=demo-s3-bucket \
--set configs.DEFAULT_CACHE_BUCKET_REGION=us-east-1 \
--set configs.DEFAULT_BUILD_LOGS_BUCKET=demo-s3-bucket \
--set configs.DEFAULT_CD_LOGS_BUCKET_REGION=us-east-1 \
--set secrets.BLOB_STORAGE_S3_ACCESS_KEY=ACCESS_KEY \
--set secrets.BLOB_STORAGE_S3_SECRET_KEY=SECRET_KEY \
--set components.devtron.ingress.enabled=true \
--set components.devtron.ingress.className=nginx \
--set components.devtron.ingress.host=devtron.devops.example.com \
--set components.devtron.ingress.annotations."nginx\.ingress\.kubernetes\.io\/app-root"="/dashboard" \
--set components.devtron.ingress.annotations."external-dns\.alpha\.kubernetes\.io\/hostname"="devtron.devops.example.com" \
--set components.devtron.ingress.annotations."cert-manager\.io\/cluster-issuer"="letsencrypt-prod" \
--set components.devtron.ingress.tls[0].secretName=devtron-tls \
--set components.devtron.ingress.tls[0].hosts[0]=devtron.devops.example.com
abhibhaw commented 3 weeks ago

Was the case same for you as well @guzmanf ?

Shikhar-404 commented 2 weeks ago

Hi @abhibhaw, I'm also facing the same issue when using gitlab, /validate works but upon deployment of any chart it gives this error, though upon checking the repo it creates the respective chart under a new project/repo. Tried refreshing argocd certificates as well but that also doesn't work. https://docs.devtron.ai/resources/devtron-troubleshoot#id-25.-refreshing-argocd-certificates-when-expired [Nov 11 2024 17:04:24 GMT+0530] devtron-5784d949fb-phrns: {"level":"error","ts":1731324864.1663113,"caller":"common/deploymentConfigService.go:303","msg":"error in getting deployment config db object by appId and envId","appId":0,"envId":0,"err":"pg: no rows in result set","stacktrace":"github.com/devtron-labs/devtron/pkg/deployment/common.(*DeploymentConfigServiceImpl).GetConfigDBObj\n\t/go/src/github.com/devtron-labs/devtron/pkg/deployment/common/deploymentConfigService.go:303\ngithub.com/devtron-labs/devtron/pkg/deployment/common.(*DeploymentConfigServiceImpl).CreateOrUpdateConfig\n\t/go/src/github.com/devtron-labs/devtron/pkg/deployment/common/deploymentConfigService.go:64\ngithub.com/devtron-labs/devtron/pkg/appStore/installedApp/service.(*AppStoreDeploymentDBServiceImpl).AppStoreDeployOperationDB\n\t/go/src/github.com/devtron-labs/devtron/pkg/appStore/installedApp/service/AppStoreDeploymentDBService.go:212\ngithub.com/devtron-labs/devtron/pkg/appStore/installedApp/service.(*AppStoreDeploymentServiceImpl).InstallApp\n\t/go/src/github.com/devtron-labs/devtron/pkg/appStore/installedApp/service/AppStoreDeploymentService.go:142\ngithub.com/devtron-labs/devtron/api/appStore/deployment.AppStoreDeploymentRestHandlerImpl.InstallApp\n\t/go/src/github.com/devtron-labs/devtron/api/appStore/deployment/AppStoreDeploymentRestHandler.go:182\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2141\ngithub.com/devtron-labs/common-lib/middlewares.Recovery.func1\n\t/go/src/github.com/devtron-labs/devtron/vendor/github.com/devtron-labs/common-lib/middlewares/recovery.go:52\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2141\ngithub.com/devtron-labs/devtron/internal/middleware.PrometheusMiddleware.func1\n\t/go/src/github.com/devtron-labs/devtron/internal/middleware/instrument.go:137\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2141\ngithub.com/devtron-labs/devtron/api/util.(*LoggingMiddlewareImpl).LoggingMiddleware.LoggingMiddlewareImpl.LoggingMiddleware.func1\n\t/go/src/github.com/devtron-labs/devtron/api/util/logger.go:84\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2141\ngithub.com/gorilla/mux.(*Router).ServeHTTP\n\t/go/src/github.com/devtron-labs/devtron/vendor/github.com/gorilla/mux/mux.go:210\nmain.(*App).Start.(*App).Start.Authorizer.func1.func2\n\t/go/src/github.com/devtron-labs/devtron/vendor/github.com/devtron-labs/authenticator/middleware/AuthMiddleware.go:91\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2141\nnet/http.serverHandler.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2943\nnet/http.(*conn).serve\n\t/usr/local/go/src/net/http/server.go:2014"} [Nov 11 2024 17:04:25 GMT+0530] devtron-5784d949fb-phrns: {"level":"info","ts":1731324865.0349934,"caller":"git/GitOpsHelper.go:167","msg":"total branch available in git repo","branch length":1} [Nov 11 2024 17:04:27 GMT+0530] devtron-5784d949fb-phrns: time="2024-11-11T11:34:27Z" level=info msg="Starting configmap/secret informers" [Nov 11 2024 17:04:27 GMT+0530] devtron-5784d949fb-phrns: time="2024-11-11T11:34:27Z" level=info msg="Configmap/secret informer synced" [Nov 11 2024 17:04:27 GMT+0530] devtron-5784d949fb-phrns: {"level":"error","ts":1731324867.9139314,"caller":"argocdServer/ArgoClientWrapperService.go:326","msg":"error in creating argo Repository","err":"rpc error: code = Unavailable desc = connection error: desc = \"transport: authentication handshake failed: tls: failed to verify certificate: x509: certificate signed by unknown authority\"","stacktrace":"github.com/devtron-labs/devtron/client/argocdServer.(*ArgoClientWrapperServiceImpl).createRepoInArgoCd\n\t/go/src/github.com/devtron-labs/devtron/client/argocdServer/ArgoClientWrapperService.go:326\ngithub.com/devtron-labs/devtron/client/argocdServer.(*ArgoClientWrapperServiceImpl).RegisterGitOpsRepoInArgoWithRetry.func1\n\t/go/src/github.com/devtron-labs/devtron/client/argocdServer/ArgoClientWrapperService.go:244\ngithub.com/devtron-labs/devtron/util/retryFunc.Retry\n\t/go/src/github.com/devtron-labs/devtron/util/retryFunc/RetryFunction.go:36\ngithub.com/devtron-labs/devtron/client/argocdServer.(*ArgoClientWrapperServiceImpl).RegisterGitOpsRepoInArgoWithRetry\n\t/go/src/github.com/devtron-labs/devtron/client/argocdServer/ArgoClientWrapperService.go:246\ngithub.com/devtron-labs/devtron/pkg/appStore/installedApp/service/FullMode/deployment.(*FullModeDeploymentServiceImpl).InstallApp\n\t/go/src/github.com/devtron-labs/devtron/pkg/appStore/installedApp/service/FullMode/deployment/FullModeDeploymentService.go:167\ngithub.com/devtron-labs/devtron/pkg/appStore/installedApp/service.(*AppStoreDeploymentServiceImpl).InstallApp\n\t/go/src/github.com/devtron-labs/devtron/pkg/appStore/installedApp/service/AppStoreDeploymentService.go:213\ngithub.com/devtron-labs/devtron/api/appStore/deployment.AppStoreDeploymentRestHandlerImpl.InstallApp\n\t/go/src/github.com/devtron-labs/devtron/api/appStore/deployment/AppStoreDeploymentRestHandler.go:182\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2141\ngithub.com/devtron-labs/common-lib/middlewares.Recovery.func1\n\t/go/src/github.com/devtron-labs/devtron/vendor/github.com/devtron-labs/common-lib/middlewares/recovery.go:52\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2141\ngithub.com/devtron-labs/devtron/internal/middleware.PrometheusMiddleware.func1\n\t/go/src/github.com/devtron-labs/devtron/internal/middleware/instrument.go:137\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2141\ngithub.com/devtron-labs/devtron/api/util.(*LoggingMiddlewareImpl).LoggingMiddleware.LoggingMiddlewareImpl.LoggingMiddleware.func1\n\t/go/src/github.com/devtron-labs/devtron/api/util/logger.go:84\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2141\ngithub.com/gorilla/mux.(*Router).ServeHTTP\n\t/go/src/github.com/devtron-labs/devtron/vendor/github.com/gorilla/mux/mux.go:210\nmain.(*App).Start.(*App).Start.Authorizer.func1.func2\n\t/go/src/github.com/devtron-labs/devtron/vendor/github.com/devtron-labs/authenticator/middleware/AuthMiddleware.go:91\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2141\nnet/http.serverHandler.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2943\nnet/http.(*conn).serve\n\t/usr/local/go/src/net/http/server.go:2014"} [Nov 11 2024 17:04:27 GMT+0530] devtron-5784d949fb-phrns: {"level":"error","ts":1731324867.9140651,"caller":"argocdServer/ArgoClientWrapperService.go:252","msg":"error in registering GitOps repository","repoName":"https://<confidential>/devspace-projects/devtron-backup-es-v2.git","err":"rpc error: code = Unavailable desc = connection error: desc = \"transport: authentication handshake failed: tls: failed to verify certificate: x509: certificate signed by unknown authority\"","stacktrace":"github.com/devtron-labs/devtron/client/argocdServer.(*ArgoClientWrapperServiceImpl).RegisterGitOpsRepoInArgoWithRetry\n\t/go/src/github.com/devtron-labs/devtron/client/argocdServer/ArgoClientWrapperService.go:252\ngithub.com/devtron-labs/devtron/pkg/appStore/installedApp/service/FullMode/deployment.(*FullModeDeploymentServiceImpl).InstallApp\n\t/go/src/github.com/devtron-labs/devtron/pkg/appStore/installedApp/service/FullMode/deployment/FullModeDeploymentService.go:167\ngithub.com/devtron-labs/devtron/pkg/appStore/installedApp/service.(*AppStoreDeploymentServiceImpl).InstallApp\n\t/go/src/github.com/devtron-labs/devtron/pkg/appStore/installedApp/service/AppStoreDeploymentService.go:213\ngithub.com/devtron-labs/devtron/api/appStore/deployment.AppStoreDeploymentRestHandlerImpl.InstallApp\n\t/go/src/github.com/devtron-labs/devtron/api/appStore/deployment/AppStoreDeploymentRestHandler.go:182\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2141\ngithub.com/devtron-labs/common-lib/middlewares.Recovery.func1\n\t/go/src/github.com/devtron-labs/devtron/vendor/github.com/devtron-labs/common-lib/middlewares/recovery.go:52\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2141\ngithub.com/devtron-labs/devtron/internal/middleware.PrometheusMiddleware.func1\n\t/go/src/github.com/devtron-labs/devtron/internal/middleware/instrument.go:137\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2141\ngithub.com/devtron-labs/devtron/api/util.(*LoggingMiddlewareImpl).LoggingMiddleware.LoggingMiddlewareImpl.LoggingMiddleware.func1\n\t/go/src/github.com/devtron-labs/devtron/api/util/logger.go:84\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2141\ngithub.com/gorilla/mux.(*Router).ServeHTTP\n\t/go/src/github.com/devtron-labs/devtron/vendor/github.com/gorilla/mux/mux.go:210\nmain.(*App).Start.(*App).Start.Authorizer.func1.func2\n\t/go/src/github.com/devtron-labs/devtron/vendor/github.com/devtron-labs/authenticator/middleware/AuthMiddleware.go:91\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2141\nnet/http.serverHandler.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2943\nnet/http.(*conn).serve\n\t/usr/local/go/src/net/http/server.go:2014"} [Nov 11 2024 17:04:27 GMT+0530] devtron-5784d949fb-phrns: /service-worker.js [Nov 11 2024 17:04:27 GMT+0530] devtron-5784d949fb-phrns: 2024/11/11 11:34:27 AUDIT_LOG: requestMethod: GET, urlPath: /dashboard/service-worker.js, queryParams: , updatedBy: admin, updatedOn: 2024-11-11 11:34:27.957166648 +0000 UTC m=+214.900389798, apiResponseCode: 200, requestPayload: [Nov 11 2024 17:04:28 GMT+0530] devtron-5784d949fb-phrns: {"level":"error","ts":1731324868.1274896,"caller":"argocdServer/ArgoClientWrapperService.go:326","msg":"error in creating argo Repository","err":"rpc error: code = Unavailable desc = connection error: desc = \"transport: authentication handshake failed: tls: failed to verify certificate: x509: certificate signed by unknown authority\"","stacktrace":"github.com/devtron-labs/devtron/client/argocdServer.(*ArgoClientWrapperServiceImpl).createRepoInArgoCd\n\t/go/src/github.com/devtron-labs/devtron/client/argocdServer/ArgoClientWrapperService.go:326\ngithub.com/devtron-labs/devtron/client/argocdServer.(*ArgoClientWrapperServiceImpl).handleArgoRepoCreationError\n\t/go/src/github.com/devtron-labs/devtron/client/argocdServer/ArgoClientWrapperService.go:357\ngithub.com/devtron-labs/devtron/client/argocdServer.(*ArgoClientWrapperServiceImpl).RegisterGitOpsRepoInArgoWithRetry\n\t/go/src/github.com/devtron-labs/devtron/client/argocdServer/ArgoClientWrapperService.go:253\ngithub.com/devtron-labs/devtron/pkg/appStore/installedApp/service/FullMode/deployment.(*FullModeDeploymentServiceImpl).InstallApp\n\t/go/src/github.com/devtron-labs/devtron/pkg/appStore/installedApp/service/FullMode/deployment/FullModeDeploymentService.go:167\ngithub.com/devtron-labs/devtron/pkg/appStore/installedApp/service.(*AppStoreDeploymentServiceImpl).InstallApp\n\t/go/src/github.com/devtron-labs/devtron/pkg/appStore/installedApp/service/AppStoreDeploymentService.go:213\ngithub.com/devtron-labs/devtron/api/appStore/deployment.AppStoreDeploymentRestHandlerImpl.InstallApp\n\t/go/src/github.com/devtron-labs/devtron/api/appStore/deployment/AppStoreDeploymentRestHandler.go:182\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2141\ngithub.com/devtron-labs/common-lib/middlewares.Recovery.func1\n\t/go/src/github.com/devtron-labs/devtron/vendor/github.com/devtron-labs/common-lib/middlewares/recovery.go:52\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2141\ngithub.com/devtron-labs/devtron/internal/middleware.PrometheusMiddleware.func1\n\t/go/src/github.com/devtron-labs/devtron/internal/middleware/instrument.go:137\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2141\ngithub.com/devtron-labs/devtron/api/util.(*LoggingMiddlewareImpl).LoggingMiddleware.LoggingMiddlewareImpl.LoggingMiddleware.func1\n\t/go/src/github.com/devtron-labs/devtron/api/util/logger.go:84\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2141\ngithub.com/gorilla/mux.(*Router).ServeHTTP\n\t/go/src/github.com/devtron-labs/devtron/vendor/github.com/gorilla/mux/mux.go:210\nmain.(*App).Start.(*App).Start.Authorizer.func1.func2\n\t/go/src/github.com/devtron-labs/devtron/vendor/github.com/devtron-labs/authenticator/middleware/AuthMiddleware.go:91\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2141\nnet/http.serverHandler.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2943\nnet/http.(*conn).serve\n\t/usr/local/go/src/net/http/server.go:2014"} [Nov 11 2024 17:04:28 GMT+0530] devtron-5784d949fb-phrns: {"level":"error","ts":1731324868.1275816,"caller":"deployment/FullModeDeploymentService.go:169","msg":"error in argo registry","err":"rpc error: code = Unavailable desc = connection error: desc = \"transport: authentication handshake failed: tls: failed to verify certificate: x509: certificate signed by unknown authority\"","stacktrace":"github.com/devtron-labs/devtron/pkg/appStore/installedApp/service/FullMode/deployment.(*FullModeDeploymentServiceImpl).InstallApp\n\t/go/src/github.com/devtron-labs/devtron/pkg/appStore/installedApp/service/FullMode/deployment/FullModeDeploymentService.go:169\ngithub.com/devtron-labs/devtron/pkg/appStore/installedApp/service.(*AppStoreDeploymentServiceImpl).InstallApp\n\t/go/src/github.com/devtron-labs/devtron/pkg/appStore/installedApp/service/AppStoreDeploymentService.go:213\ngithub.com/devtron-labs/devtron/api/appStore/deployment.AppStoreDeploymentRestHandlerImpl.InstallApp\n\t/go/src/github.com/devtron-labs/devtron/api/appStore/deployment/AppStoreDeploymentRestHandler.go:182\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2141\ngithub.com/devtron-labs/common-lib/middlewares.Recovery.func1\n\t/go/src/github.com/devtron-labs/devtron/vendor/github.com/devtron-labs/common-lib/middlewares/recovery.go:52\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2141\ngithub.com/devtron-labs/devtron/internal/middleware.PrometheusMiddleware.func1\n\t/go/src/github.com/devtron-labs/devtron/internal/middleware/instrument.go:137\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2141\ngithub.com/devtron-labs/devtron/api/util.(*LoggingMiddlewareImpl).LoggingMiddleware.LoggingMiddlewareImpl.LoggingMiddleware.func1\n\t/go/src/github.com/devtron-labs/devtron/api/util/logger.go:84\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2141\ngithub.com/gorilla/mux.(*Router).ServeHTTP\n\t/go/src/github.com/devtron-labs/devtron/vendor/github.com/gorilla/mux/mux.go:210\nmain.(*App).Start.(*App).Start.Authorizer.func1.func2\n\t/go/src/github.com/devtron-labs/devtron/vendor/github.com/devtron-labs/authenticator/middleware/AuthMiddleware.go:91\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2141\nnet/http.serverHandler.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2943\nnet/http.(*conn).serve\n\t/usr/local/go/src/net/http/server.go:2014"} [Nov 11 2024 17:04:28 GMT+0530] devtron-5784d949fb-phrns: {"level":"error","ts":1731324868.128001,"caller":"deployment/AppStoreDeploymentRestHandler.go:187","msg":"service err, CreateInstalledApp","err":"rpc error: code = Unavailable desc = connection error: desc = \"transport: authentication handshake failed: tls: failed to verify certificate: x509: certificate signed by unknown authority\"","payload":{"id":10,"appId":21,"appName":"backup-es-v2","teamId":1,"environmentId":1,"installedAppId":10,"installedAppVersionId":10,"installedAppVersionHistoryId":10,"appStoreVersion":16109,"valuesOverrideYaml":"global:\n

ayu-devtron commented 1 week ago

Findings: when we were doing curl argocd-server.devtroncd there is some config issue in ndot or noproxy setup , so there is connectivity issue to argocd-server from devtron pod , after using fully qualified name the issue got resolved argocd-server.devtroncd.svc.cluster.local.

Other things that we tried was deploying same app via argo dashboard and it got deployed which points us in the direction that there is some connectivity issue from devtron pod to argo server

cc: @Shikhar-404

ayu-devtron commented 1 week ago

@romosa in your error logs below log is coming. You need to provide server.secretkey in argocd-secret

"msg":"error on get acd connection","err":"server.secretkey is missing",

Let me know if that resolves your issue