Interrupting Julia disables `@batch` for this session

[efaulhaber]

MWE:

using Polyester

function foo()
    @batch for i in 1:4
        println(Threads.threadid())
        sleep(1)
    end
end

When running this in a new REPL (Julia started with two threads, julia -t 2), the output of foo() is, as expected:

julia> foo2()
1
2
1
2

I get the same output when running this multiple times. Now, when I interrupt Julia before the function is done (that's what the sleep is for), @batch only uses one thread until I restart Julia:

julia> foo2()
1
2
ERROR: InterruptException:
Stacktrace:
  [1] poptask(W::Base.InvasiveLinkedListSynchronized{Task})
    @ Base .\task.jl:760
  [2] wait
    @ .\task.jl:768 [inlined]
  [3] wait(c::Base.GenericCondition{Base.Threads.SpinLock})
    @ Base .\condition.jl:106
  [4] _trywait(t::Timer)
    @ Base .\asyncevent.jl:111
  [5] wait
    @ .\asyncevent.jl:129 [inlined]
  [6] sleep(sec::Int64)
    @ Base .\asyncevent.jl:214
  [7] macro expansion
    @ C:\Users\Erik\git\Trixi.jl\foo.jl:18 [inlined]
  [8] #7
    @ C:\Users\Erik\.julia\packages\Polyester\0DPCU\src\closure.jl:168 [inlined]
  [9] macro expansion
    @ C:\Users\Erik\.julia\packages\Polyester\0DPCU\src\batch.jl:79 [inlined]
 [10] _batch_no_reserve
    @ C:\Users\Erik\.julia\packages\Polyester\0DPCU\src\batch.jl:53 [inlined]
 [11] batch
    @ C:\Users\Erik\.julia\packages\Polyester\0DPCU\src\batch.jl:195 [inlined]
 [12] macro expansion
    @ C:\Users\Erik\.julia\packages\Polyester\0DPCU\src\closure.jl:164 [inlined]
 [13] foo2()
    @ Main C:\Users\Erik\git\Trixi.jl\foo.jl:16
 [14] top-level scope
    @ REPL[2]:1

julia> 2
julia> foo2()
1
1
1
1

julia> foo2()
1
1
1
1

[chriselrod]

You can reset the state via

using Polyester, ThreadingUtilities
Polyester.reset_workers!()
ThreadingUtilities.reinitialize_tasks!()

[efaulhaber]

Yes, that works! I guess that's not supposed to happen, right? One shouldn't need to reset the state manually.

[jlchan]

Was reset_workers! removed in the recent 0.5.1 update? I still need to reset the state manually from time to time but can't seem to access it anymore.

[chriselrod]

It is now in PolyesterWeave.jl

[jlchan]

Thanks!

[jlchan]

Just so its visible for others, this is what I'm using with PolyesterWeave 0.1 and ThreadingUtilities 0.4.6:

PolyesterWeave.reset_workers!()
for i = 1:Threads.nthreads()-1; ThreadingUtilities.initialize_task(i); end

[efaulhaber]

Have there been any developments by now? Threads are still being disabled (without notifying the user), and resetting them with the commands above is not really convenient.

I don't see any situation where disabling threading and not re-enabling it automatically is the desired behavior.

[chriselrod]

try/finally has > 10 ns overhead, but that would be how we could re-enable.

julia> bar(x) = x
bar (generic function with 1 method)

julia> function foo(x)
           try
               bar(x)
           finally
               bar(x)
           end
           nothing
       end
foo (generic function with 2 methods)

julia> foo(2)

julia> @benchmark foo(2)
BenchmarkTools.Trial: 10000 samples with 998 evaluations.
 Range (min … max):  17.132 ns … 28.618 ns  ┊ GC (min … max): 0.00% … 0.00%
 Time  (median):     18.461 ns              ┊ GC (median):    0.00%
 Time  (mean ± σ):   18.522 ns ±  0.338 ns  ┊ GC (mean ± σ):  0.00% ± 0.00%

                        ▆▄   ▄█▁   ▆▂    ▄    ▃     ▁         ▂
  ▃▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▆███▇▇███▅▃▅██▄▄▄▄█▅▅▅▄█▇▅▅▅▅█▄▅▆▆▇▇▆▆ █
  17.1 ns      Histogram: log(frequency) by time      19.9 ns <

 Memory estimate: 0 bytes, allocs estimate: 0.

[Krastanov]

Maybe an exceptionhandling=true argument for the batch macro can be considered for merging if contributed by those that need the feature?

[ranocha]

For people interested in this issue: a function Polyester.reset_threads!() has just been released to make it easier for people to do this manually (without requiring to depend on PolyesterWeave.jl and ThreadingUtilities.jl explicitly).