search

JuliaCI / julia-buildkite-plugin

Buildkite plugin to install Julia for use in a pipeline.
4 stars 5 forks source link

Put the "download Julia" step inside a bounded retry loop? #5

Closed DilumAluthge closed 3 years ago

DilumAluthge commented 3 years ago

Sometimes, the "download Julia" step will fail due to network flakiness.

Here's an example (from https://buildkite.com/julialang/kernelabstractions-dot-jl/builds/116#904bc928-8bef-4901-a972-7f30dfac4943):

Downloading Julia nightly
Source URL: https://julialangnightlies-s3.julialang.org/bin/linux/x64/julia-latest-linux64.tar.gz

gzip: stdin: unexpected end of file
tar: Unexpected EOF in archive
tar: Unexpected EOF in archive
tar: Error is not recoverable: exiting now
🚨 Error: The plugin julia pre-command hook exited with status 2

Currently, we only try once to download Julia. Instead, we should put this inside a bounded retry loop, with e.g. a maximum of 10 tries.

maleadt commented 3 years ago

wget already defaults to --tries=20, this is actual invalid stuff being downloaded...

DilumAluthge commented 3 years ago

Hmmm.

DilumAluthge commented 3 years ago

Interesting.

DilumAluthge commented 3 years ago

I kept seeing this on the p6000 machine but not on other machines. But now I can't reproduce.

DilumAluthge commented 3 years ago

Yeah everything seems to be working fine now, see e.g. https://github.com/JuliaGPU/KernelAbstractions.jl/pull/204#issuecomment-770319234

Idk what was causing the issue I was seeing before.