concourse / git-resource

tracks commits in a branch of a Git repository
Apache License 2.0
193 stars 289 forks source link

git resource should be resilient to occasional dns resolution errors #209

Open cppforlife opened 6 years ago

cppforlife commented 6 years ago

encountered on wings env. created issue per @vito's suggestion

Cloning into '/tmp/build/get'...
fatal: unable to access 'https://github.com/cppforlife/knctl/': Could not resolve host: github.com
lrstanley commented 6 years ago

This has been open for awhile, but.. I don't think so, for a few reasons.

You can't expect a utility or resource to be resilient to it's inputs and outputs, that's just way too much overhead. E.g. should we make the git resource resilient to the disk somehow filling up? cpu/disk utilization issues? the timezone being incorrectly specified? random network drops? All these things I think are unreasonable expectations. If git itself cannot, I don't see why the resource should. If you are bringing concourse (and it's resources) into a broken environment, I wouldn't expect it to handle putting bandages on things, especially ones outside of it's control.

Secondly, doing so could cause major issues for others. There is already a configurable re-check time for Concourse resources. Making the git resource "resilient to occasional dns resolution errors" implies that the resource itself would also implement it's own re-check system. This is not only duplicated effort that has to be implemented, but could cause a cascading issue where a downed bitbucket server (example) or temporary service discovery issue leads to 10x the amount of request attempts to a potentially already-failing system. The alternative is for some form of dns caching system, but this has it's own large list of caveats, living inside of Concourse.

Resources, at least how they're used today, are run in a one-off fashion, not long lived. I think this makes them easier to manage and maintain, if a bit resource wasting.

95rade commented 4 years ago

This same error happened to me today running 3.10.