search

apache / incubator-devlake

Apache DevLake is an open-source dev data platform to ingest, analyze, and visualize the fragmented data from DevOps tools, extracting insights for engineering excellence, developer experience, and community growth.
https://devlake.apache.org/
Apache License 2.0
2.56k stars 510 forks source link

[Bug][GitHub] Pipeline run fails due to x509: certificate issue #7795

Closed sprathod369 closed 1 month ago

sprathod369 commented 1 month ago

Search before asking

What happened

GitHub connection from Config UI successfully tested but when pipelines are executed they fails with below error in one of the stages.

attached stack trace -- stack trace: | github.com/apache/incubator-devlake/core/runner.RunTask.func1 | /app/core/runner/run_task.go:73 | runtime.gopanic | /usr/local/go/src/runtime/panic.go:884 | [...repeated from below...] Wraps: (2) run task failed with panic (github.com/apache/incubator-devlake/helpers/pluginhelper/api.CreateAsyncGraphqlClient:71) Wraps: (3) attached stack trace -- stack trace: | github.com/apache/incubator-devlake/plugins/github_graphql/impl.GithubGraphql.PrepareTaskData.func1 | /app/plugins/github_graphql/impl/impl.go:223 | github.com/apache/incubator-devlake/helpers/pluginhelper/api.CreateAsyncGraphqlClient | /app/helpers/pluginhelper/api/graphql_async_client.go:69 | github.com/apache/incubator-devlake/plugins/github_graphql/impl.GithubGraphql.PrepareTaskData | /app/plugins/github_graphql/impl/impl.go:220 | github.com/apache/incubator-devlake/core/runner.RunPluginSubTasks | /app/core/runner/run_task.go:246 | github.com/apache/incubator-devlake/core/runner.RunPluginTask | /app/core/runner/run_task.go:163 | github.com/apache/incubator-devlake/core/runner.RunTask | /app/core/runner/run_task.go:137 | github.com/apache/incubator-devlake/server/services.runTaskStandalone | /app/server/services/task_runner.go:113 | github.com/apache/incubator-devlake/server/services.RunTasksStandalone.func1 | /app/server/services/task.go:187 | runtime.goexit | /usr/local/go/src/runtime/asm_amd64.s:1598 Wraps: (4) Post "https://api.github.com/graphql": tls: failed to verify certificate: x509: certificate signed by unknown authority Wraps: (5) Post "https://api.github.com/graphql" Wraps: (6) tls: failed to verify certificate Wraps: (7) x509: certificate signed by unknown authority Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *withstack.withStack (4) *errutil.withPrefix (5) *url.Error (6) *tls.CertificateVerificationError (7) x509.UnknownAuthorityError

What do you expect to happen

Pipeline execution should have completed successfully.

How to reproduce

1) Spin up the devlake using docker-compose on an on-prem Ubuntu 22x VM 2) Add the GitHub connection, provide the required details and PAT key, successfully test the GitHub connection 3) Create a project that has the GitHub connection and the required repo scope 4) Run the pipeline

Anything else

Currently moved to the latest version v1.0.1-beta4 but pipeline executions are blocked due to X509 certificate issue. Note: The earlier version that was setup using v1.0.0-beta11 with the same GitHub configuration, PAT key and repo scope, the pipelines completed successfully.

The IN_SECURE_SKIP_VERIFY is set to true.

Version

v1.0.1-beta4

Are you willing to submit PR?

Code of Conduct

dosubot[bot] commented 1 month ago

Hey @sprathod369! I'm here to assist with any bugs, questions, or contributions. Let's get this issue sorted out together!

To resolve the x509 certificate error in DevLake v1.0.1-beta4, despite having IN_SECURE_SKIP_VERIFY set to true, you should add your root CA to the devlake container. Here is how you can do it:

  1. Mount your rootCA.crt into the devlake container.
  2. Add a command node to install the mounted certificate.

Here is an example of the docker-compose installation:

devlake:
  image: apache/devlake:v1.0.1-beta4
  ...
  volumes:
    ...
    - /path/to/your/rootCA.crt:/usr/local/share/ca-certificates/rootCA.crt
  command: [ "sh", "-c", "update-ca-certificates; lake" ]
  ...

This approach ensures that the root CA is recognized by the devlake container, thereby resolving the certificate verification issue. Note that setting IN_SECURE_SKIP_VERIFY=true only solves the "Test Connection" error, but not the "GitExtractor" failure due to certificate verification. Adding the root CA is a more comprehensive solution.

Additionally, it is important to note that neither gogit nor git2go, which are used by DevLake, support insecure HTTPS. Therefore, adding the root CA is the recommended approach to resolve the certificate verification issue. For more details, you can refer to the Configuration and Blueprint Troubleshooting [1][2][3][4][5].

To continue talking to Dosu, mention @dosu.

sprathod369 commented 1 month ago

Issue resolved and this worked like charm. Thanks @dosu

dosubot[bot] commented 1 month ago

You're welcome! I'm glad to hear that it worked for you. If you have any more questions or run into any other issues, feel free to reach out. Happy coding!

To continue talking to Dosu, mention @dosu.