search

sous-chefs / apt

Development repository for the apt cookbook
https://supermarket.chef.io/cookbooks/apt
Apache License 2.0
202 stars 266 forks source link

apt-get update: retries and error handling #164

Closed rmoriz closed 7 years ago

rmoriz commented 8 years ago

Sometimes, for whatever reasons, apt-get update fails. This might be caused by mirrors that are updated right in the moment when apt-get update was executed or due to network issues. Usually this later causes various obscure problems like "WARNING: The following packages cannot be authenticated!" and crash the chef run.

I suggest to implement some shell logic to retry apt-get update if it fails (retries: attribute, maybe 3?) in all resources that have the command 'apt-get update'`.

I'm also not sure if the ignorance of errors (ignore_failure true) is a good idea, but at least a retry would probably reduce errors a lot.

The Debian/Ubuntu autopkgtest maintainers had the same issue and implemented a very simple static solution: http://anonscm.debian.org/cgit/autopkgtest/autopkgtest.git/diff/?id=7f667754

AlexanderStohr commented 7 years ago

I am sometimes living behind a (really not small) proxy service and I am getting quite often such sequences of failure as you can see below. When manually invoked for me it can take some 1 to >10 attempts to get all querys succeeding.

I suspect that opening this bigger bunch of locations is just giving up a tiny little bit too fast - you might know from your browser... trying the refresh keystroke after an "endlesse loading" situation will nearly instantly succeed, for whatever non-evident reason this happens. Care should be taken to allow similar behaviour either by default or by special expressed desire. An option for changing (=increasing) the retry count value on the command line that applies to the any individual open for a location should be made availabel.

See here an expample what currently happens to me:

# apt update [...] Get:6 http://de.archive.ubuntu.com/ubuntu xenial-updates/main amd64 Packages [391 kB] Err:6 http://de.archive.ubuntu.com/ubuntu xenial-updates/main amd64 Packages Hash Sum mismatch [...] Get:10 http://de.archive.ubuntu.com/ubuntu xenial-updates/universe Translation-en [114 kB] Fetched 562 kB in 3s (182 kB/s) Reading package lists... Done E: Failed to fetch http://de.archive.ubuntu.com/ubuntu/dists/xenial-updates/main/binary-amd64/by-hash/SHA256/6a4dfa61229a6e9d1e82f5fb3434c9b650bc856aca68add08343380e3fb48afe Hash Sum mismatch E: Failed to fetch http://de.archive.ubuntu.com/ubuntu/dists/xenial-updates/main/binary-i386/by-hash/SHA256/17dc81048807986276d6314abbacc968efe9da255679863b06667b5ffa60ffd8 E: Some index files failed to download. They have been ignored, or old ones used instead.

tas50 commented 7 years ago

We've removed the apt_update and apt_repository resources from this cookbook as they were merged into core chef. I'm going to close this issue out now. If you think it's still valid against a new chef-client release please open it up against the chef/chef repo so we can work on it where the code now resides.

rmoriz commented 7 years ago

the resource change is not related, but the original bug seems to be "fixed" by setting "ignore_failure" to true in https://github.com/chef-cookbooks/apt/commit/24a68c0dbf96df3be4348e8b22693abde56cbd88 anyhow