search

JuliaLang / Pkg.jl

Pkg - Package manager for the Julia programming language
https://pkgdocs.julialang.org
Other
609 stars 251 forks source link

Don't retry artifact downloads on `SystemError`, like `no space left on device` #3932

Open IanButterworth opened 1 week ago

IanButterworth commented 1 week ago

Fixes https://github.com/JuliaLang/Pkg.jl/issues/2928

The error that Downloads.download throws when out of space (on a Mac) is 


julia> Downloads.download("https://julialang-s3.julialang.org/bin/mac/aarch64/1.10/julia-1.10.4-macaarch64.dmg", "/Volumes/RAMDisk/file")
ERROR: SystemError: flush: No space left on device
Stacktrace:
  [1] systemerror(p::String, errno::Int32; extrainfo::Nothing)
    @ Base ./error.jl:176
  [2] systemerror
    @ ./error.jl:175 [inlined]
  [3] flush(s::IOStream)
    @ Base ./iostream.jl:70
  [4] arg_write(f::Downloads.var"#8#17"{Base.DevNull, Nothing, Vector{…}, Float64, Nothing, Bool, Nothing, Bool, String, Bool, Bool}, arg::IOStream)
    @ ArgTools ~/.julia/juliaup/julia-1.10.4+0.aarch64.apple.darwin14/share/julia/stdlib/v1.10/ArgTools/src/ArgTools.jl:136
  [5] #7
    @ ~/.julia/juliaup/julia-1.10.4+0.aarch64.apple.darwin14/share/julia/stdlib/v1.10/Downloads/src/Downloads.jl:353 [inlined]
  [6] arg_read
    @ ~/.julia/juliaup/julia-1.10.4+0.aarch64.apple.darwin14/share/julia/stdlib/v1.10/ArgTools/src/ArgTools.jl:76 [inlined]
  [7] request(url::String; input::Nothing, output::IOStream, method::Nothing, headers::Vector{…}, timeout::Float64, progress::Nothing, verbose::Bool, debug::Nothing, throw::Bool, downloader::Nothing)
    @ Downloads ~/.julia/juliaup/julia-1.10.4+0.aarch64.apple.darwin14/share/julia/stdlib/v1.10/Downloads/src/Downloads.jl:352
  [8] request
    @ ~/.julia/juliaup/julia-1.10.4+0.aarch64.apple.darwin14/share/julia/stdlib/v1.10/Downloads/src/Downloads.jl:320 [inlined]
  [9] #3
    @ ~/.julia/juliaup/julia-1.10.4+0.aarch64.apple.darwin14/share/julia/stdlib/v1.10/Downloads/src/Downloads.jl:258 [inlined]
 [10] open(f::Downloads.var"#3#4"{…}, args::String; kwargs::@Kwargs{…})
    @ Base ./io.jl:396
 [11] open_nolock
    @ ~/.julia/juliaup/julia-1.10.4+0.aarch64.apple.darwin14/share/julia/stdlib/v1.10/ArgTools/src/ArgTools.jl:35 [inlined]
 [12] arg_write(f::Function, arg::String)
    @ ArgTools ~/.julia/juliaup/julia-1.10.4+0.aarch64.apple.darwin14/share/julia/stdlib/v1.10/ArgTools/src/ArgTools.jl:103
 [13] #download#2
    @ ~/.julia/juliaup/julia-1.10.4+0.aarch64.apple.darwin14/share/julia/stdlib/v1.10/Downloads/src/Downloads.jl:257 [inlined]
 [14] download(url::String, output::String)
    @ Downloads ~/.julia/juliaup/julia-1.10.4+0.aarch64.apple.darwin14/share/julia/stdlib/v1.10/Downloads/src/Downloads.jl:246
 [15] top-level scope
    @ REPL[2]:1
Some type information was truncated. Use `show(err)` to see complete types.
IanButterworth commented 1 week ago

@StefanKarpinski two thoughts:

  1. Should Downloads.download throw a specific error for lack of disk space that we catch here?
  2. Could we check for disk space before starting the download? Would that be possible/valid?
DilumAluthge commented 1 week ago

Out of curiosity, when do we think the "out of disk space" error is occurring?

  1. When downloading the tar.gz file?
  2. When extracting the tar.gz file?
IanButterworth commented 1 week ago

Logically it could be either