JuliaWeb / HTTP.jl

HTTP for Julia
https://juliaweb.github.io/HTTP.jl/stable/
Other
626 stars 177 forks source link

Improving progress information while downloading #1120

Open mkoculak opened 8 months ago

mkoculak commented 8 months ago

I am writing a package to interact with an online database and download all the user requested files. Since it will be a lot of big files, it would be great to have progress displayed as well as keeping the output relatively constrained.

Right now, the download function reports the progress, but prints a multi-line info block every second. I experimented with ANSI codes a bit and found a combination that overwrites the block rather that printing a new one: print("\e[0J") and print("\e[1F", "\e[9A") placed before and after (respectively) the @info call here makes it work for me. This could also be turned on/off by a keyword if someone would prefer the current behaviour.

Secondly, I tried the simplest way to insert the ProgressMeter functionality instead, replacing the whole writing block with this:

p = Progress(Int(total_bytes), dt=1)
Base.open(file, "w") do fh
    while(!eof(stream))
        downloaded_bytes += write(fh, readavailable(stream))
        update!(p, downloaded_bytes)
    end
end

and it seems to work as expected (it would address things mentioned in #537 and #922). This would probably be a good thing to add as an extension, but that would require some re-factoring of the existing code, right? (to make each progress reporting method interchangeable)

do you think these would be valuable contributions? I could prepare a pull request (although would appreciate suggestions what would be the best way to make it into an extension).