project-everest / everest-ci

CI scripts for project everest
3 stars 8 forks source link

understand why we're getting hash sum mismatch #17

Closed msprotz closed 7 years ago

msprotz commented 7 years ago

This is the Docker/Ubuntu from scratch scenario.

Probably a corrupt mirror somewhere on the network? Can we make CI retry several times when fetching packages?

irinasp commented 7 years ago

Retrying several time is not helping to fetch the package. Recent problem was with apt-get install fstar package. The workaround is to get package manually from different mirror. It is temporary workaround until corrupted mirror mirrors.kernel.org/Ubuntu will be fixed. Workaround for dockerfile: RUN wget http://mirrors.xmission.com/ubuntu/pool/main/m/mono/libmono-microsoft-web-infrastructure1.0-cil_4.2.1.102+dfsg2-7ubuntu4_all.deb RUN apt-get install --yes fsharp

msprotz commented 7 years ago

Retrying several time is not helping to fetch the package. What method exactly are you using to retry? There is a built-in facility for APT to retry, which you can specify using apt.conf (see https://linux.die.net/man/5/apt.conf)

More concretely: something like echo "APT::Acquire::Retries=5" >> /etc/apt.conf should be added to the script (untested, please read man page above).

apt-get install fstar I don't think we install fstar via APT, do we?

corrupted mirror mirrors.kernel.org/Ubuntu Is it the mirror that's corrupted or just a failure to download? What I observed when I wrote that bug report a couple months ago was that, most of the time, it was a network error and retrying did solve the problem.

More generally, I would strongly advise against putting that sort of hardcoded URLs in the dockerfile; the next time there is a minor update (e.g. .103) the package will go away and the Docker file won't be future-proof. Also, if this is a network hash sum mismatch, then tomorrow the package that fails may end up being a different one.

Thanks,

Jonathan

irinasp commented 7 years ago

Adding following parts into dockerfile fixed the issue:

Try to overcome the "Hash Sum Mismatch" failure by retrying if failed

RUN echo "Acquire::Retries \"16\";" > /etc/apt/apt.conf.d/99acquire-retries

And also Tahina added the loop for installing the packages until they got installed: RUN until apt-get install --no-install-recommends --yes \