JuliaLang / Downloads.jl

MIT License
89 stars 34 forks source link

`easy_hook` invocation can run into worldage issue #240

Closed tanmaykm closed 3 weeks ago

tanmaykm commented 2 months ago

If easy_hook is replaced on the default downloader, tasks that are already running may face world age issue when they try to use the downloader. This can happen quite easily if packages or code modules start tasks that use the default downloader and something sets an easy_hook.

julia> using Downloads

julia> bg_task = @async begin
           for idx in 1:5
               sleep(5)
               try
                   output = Downloads.download("http://www.google.com")
                   @info("downloaded $idx/5 to $output")
               catch ex
                   @error("exception downloading $idx/5 $ex")
               end
           end
       end;

julia> sleep(10);
[ Info: downloaded 1/5 to /tmp/jl_YlIp8rgBKF

julia> Downloads.DOWNLOADER[].easy_hook = (easy, info) -> Downloads.Curl.setopt(easy, Downloads.Curl.CURLOPT_LOW_SPEED_TIME, 0);

julia> @info("easy_hook set")
[ Info: easy_hook set

julia> wait(bg_task)
┌ Error: exception downloading 2/5 MethodError(var"#3#4"(), ...
└ @ Main REPL[2]:8
┌ Error: exception downloading 3/5 MethodError(var"#3#4"(), ...
└ @ Main REPL[2]:8
┌ Error: exception downloading 4/5 MethodError(var"#3#4"(), ...
└ @ Main REPL[2]:8
┌ Error: exception downloading 5/5 MethodError(var"#3#4"(), ...
└ @ Main REPL[2]:8

julia> output = Downloads.download("http://www.google.com");

julia> @info("downloaded successfully in current world age to $output")
[ Info: downloaded successfully in current world age to /tmp/jl_UFEdo9GhCL

Maybe we should be doing a Base.invokelatest here in any case?

Also there could be a race between downloader.easy_hook !== nothing and downloader.easy_hook(easy, info), so would be good to get downloader.easy_hook into a local variable there?

Ref. some related discussion here.