timholy / ProgressMeter.jl

Progress meter for long-running computations
MIT License
694 stars 91 forks source link

Async task issue #331

Closed kobusherbst closed 1 month ago

kobusherbst commented 1 month ago

I have a situation like this:

@sync begin
     @async sometask(nodea)
     @async sometask(nodeb)
end

The task displays a progress meter, created like this:

p = Progress(length(keys(gdf_memberships)); desc="Processing $(node.name) household memberships...")

The output is interlaced with progress updated on multiple lines, rather than just on two lines.

How can I fix this please - wasn't sure how to use the Tips for parallel programming in the documentation in this case.

MarcMush commented 1 month ago

this seems like a job for #157 (not yet merged)

Progress(10; safe_lock=true) might be enough for you though

otherwise a MWE would be useful to help you

kobusherbst commented 1 month ago

Here is the MWE:

using ProgressMeter

function progress_example(node)
    n = 25
    p = Progress(n; desc="$(node) computing...", safe_lock=true)
    for i in 1:n
        sleep(rand(1:2))
        next!(p)
    end
    finish!(p)
end

progress_example("Node A")
progress_example("Node B")

@sync begin
    @async progress_example("Node C")
    @async progress_example("Node D")
end

The first two calls display the "normal" expected behaviour, whereas the async calls show how the progress bars become interlaced. I am expecting (perhaps naively so) that the Node C and D progress bars will be updated in place as the async tasks progress.

MarcMush commented 1 month ago

157 is perfect for that, you can use it with ]add ProgressMeter#5c21f5d

otherwise, you need the same lock for both progressbars and different offset:

using ProgressMeter

function progress_example(node, lock, offset)
    n = 25
    p = Progress(n; desc="$(node) computing...", safe_lock=true, lock, offset)
    for i in 1:n
        sleep(rand(1:2))
        next!(p)
    end
    finish!(p)
end

@sync begin
    lock = Threads.ReentrantLock()
    @async progress_example("Node C", lock, 0)
    @async progress_example("Node D", lock, 1)
end
kobusherbst commented 1 month ago

Works perfectly, that you!