Git repository: context deadline exceeded

updatecli / updatecli

A Declarative Dependency Management tool

https://www.updatecli.io

Apache License 2.0

539 stars 62 forks source link

Git repository: context deadline exceeded #559

Open olblak opened 2 years ago

olblak commented 2 years ago

Is there an existing issue for this?

[X] I have searched the existing issues

Current Behavior

While testing a manifest with three git repository. I notice that from time to time that one git clone is failing with error message "context deadline exceeded"

scenario-3
----------
DEBUG: stage: git-checkout

DEBUG: checkout branch "main", based on "main" to directory "/tmp/updatecli/gitgithub_com_updatecli_updatecli_git"
ERROR: ✗ context deadline exceeded

Expected Behavior

It shouldn't

Steps To Reproduce

No response

Environment

- OS:Linux
- updatecli:v0.20.1

Anything else?

No response

dduportal commented 2 years ago

Can you share the manifest, so we can work on reproducing (and ensure it is not only a network issue) ?

olblak commented 2 years ago

I couldn't reproduce the issue reliably.

olblak commented 2 years ago

After a discussion with @dduportal today, we identify the root cause of the issue. It comes from the way we identify what should be clone in parallel. In the context of GitHub scm, we use the combo owner/repository if a repository need to be cloned, but when two similar scm configurations needs to be clone, we end up in a situation where different Go routines try to clone the same repository in the same location

https://github.com/updatecli/updatecli/blob/main/pkg/core/engine/main.go#L91

Ideally each scm should provide a function such as CacheID so we could clearly identify if two scm configuration are similar enough to be clone once in the same location