search

weaveworks / weave-gitops-enterprise

This repo provides the enterprise level features for the weave-gitops product, including CAPI cluster creation and team workspaces.
https://docs.gitops.weave.works/
Apache License 2.0
160 stars 29 forks source link

Some times WGE failed to create pull request for capi cluster #544

Open saeedfazal opened 2 years ago

saeedfazal commented 2 years ago

Some times enterprise failed to create pull request for capd cluster. We can clearly see the error in cluster service logs.

{"level":"error","ts":1646839609.5645578,"caller":"server/clusters.go:169","msg":"Failed to create pull request","error":"unable to write files to branch \"br-cli-end-to-end-capd-cluster-2\": unable to create new branch \"br-cli-end-to-end-capd-cluster-2\" from commit \"dc0ffddd277a82ee36325957ce05084cace59f01\" in branch \"main\": POST https://gitlab.com/api/v4/projects/wge-test-org/gitops-capi-template-1957833463-9f60563621de3b88/repository/branches: 400 {message: }","stacktrace":"github.com/weaveworks/weave-gitops-enterprise/cmd/clusters-service/pkg/server.(*server).CreatePullRequest.func1\n\t/home/runner/work/weave-gitops-enterprise/weave-gitops-enterprise/cmd/clusters-service/pkg/server/clusters.go:169\ngorm.io/gorm.(*DB).Transaction\n\t/home/runner/go/pkg/mod/gorm.io/gorm@v1.21.11/finisher_api.go:545\ngithub.com/weaveworks/weave-gitops-enterprise/cmd/clusters-service/pkg/server.(*server).CreatePullRequest\n\t/home/runner/work/weave-gitops-enterprise/weave-gitops-enterprise/cmd/clusters-service/pkg/server/clusters.go:140\ngithub.com/weaveworks/weave-gitops-enterprise/cmd/clusters-service/pkg/protos.local_request_ClustersService_CreatePullRequest_0\n\t/home/runner/work/weave-gitops-enterprise/weave-gitops-enterprise/cmd/clusters-service/pkg/protos/capi_server.pb.gw.go:323\ngithub.com/weaveworks/weave-gitops-enterprise/cmd/clusters-service/pkg/protos.RegisterClustersServiceHandlerServer.func6\n\t/home/runner/work/weave-gitops-enterprise/weave-gitops-enterprise/cmd/clusters-service/pkg/protos/capi_server.pb.gw.go:600\ngithub.com/grpc-ecosystem/grpc-gateway/v2/runtime.(*ServeMux).ServeHTTP\n\t/home/runner/go/pkg/mod/github.com/grpc-ecosystem/grpc-gateway/v2@v2.7.1/runtime/mux.go:300\ngithub.com/weaveworks/weave-gitops/pkg/server/middleware.WithLogging.func1\n\t/home/runner/go/pkg/mod/github.com/weaveworks/weave-gitops@v0.6.3-0.20220224102352-7297ddf944d7/pkg/server/middleware/middleware.go:53\nnet/http.HandlerFunc.ServeHTTP\n\t/opt/hostedtoolcache/go/1.17.7/x64/src/net/http/server.go:2047\ngithub.com/weaveworks/weave-gitops/pkg/server/middleware.WithProviderToken.func1\n\t/home/runner/go/pkg/mod/github.com/weaveworks/weave-gitops@v0.6.3-0.20220224102352-7297ddf944d7/pkg/server/middleware/middleware.go:94\nnet/http.HandlerFunc.ServeHTTP\n\t/opt/hostedtoolcache/go/1.17.7/x64/src/net/http/server.go:2047\ngithub.com/weaveworks/weave-gitops-enterprise/common/entitlement.CheckEntitlementHandler.func1\n\t/home/runner/work/weave-gitops-enterprise/weave-gitops-enterprise/common/entitlement/entitlement.go:77\nnet/http.HandlerFunc.ServeHTTP\n\t/opt/hostedtoolcache/go/1.17.7/x64/src/net/http/server.go:2047\ngithub.com/weaveworks/weave-gitops-enterprise/common/entitlement.EntitlementHandler.func1\n\t/home/runner/work/weave-gitops-enterprise/weave-gitops-enterprise/common/entitlement/entitlement.go:51\nnet/http.HandlerFunc.ServeHTTP\n\t/opt/hostedtoolcache/go/1.17.7/x64/src/net/http/server.go:2047\nnet/http.(*ServeMux).ServeHTTP\n\t/opt/hostedtoolcache/go/1.17.7/x64/src/net/http/server.go:2425\nnet/http.serverHandler.ServeHTTP\n\t/opt/hostedtoolcache/go/1.17.7/x64/src/net/http/server.go:2879\nnet/http.(*conn).serve\n\t/opt/hostedtoolcache/go/1.17.7/x64/src/net/http/server.go:1930"}
E0309 15:26:49.568345       8 middleware.go:39]  "msg"="server error" "error"="unable to create pull request and cluster rows for \"cluster-template-development-observability-0\": unable to write files to branch \"br-cli-end-to-end-capd-cluster-2\": unable to create new branch \"br-cli-end-to-end-capd-cluster-2\" from commit \"dc0ffddd277a82ee36325957ce05084cace59f01\" in branch \"main\": POST https://gitlab.com/api/v4/projects/wge-test-org/gitops-capi-template-1957833463-9f60563621de3b88/repository/branches: 400 {message: }"

https://github.com/weaveworks/weave-gitops-enterprise/runs/5481758380?check_suite_focus=true

yiannistri commented 2 years ago

Consider adding retrying code for PR creation. Also try to investigate and understand why the Gitlab API returns a 400.