Unfortunately, #98 turned out to be a bit of a failure. It worked locally, but ran into https://github.com/JuliaLang/Downloads.jl/issues/94 in CI on Julia 1.7.2. Luckily, starting multiple threads actually was a bit dumb because PlutoStaticHTML.jl then starts threads in which Pluto.jl starts distributed workers which can start multiple threads. This PR aims to reduce the number of started threads by starting Pluto.jl's workers from asynchronous tasks (coroutines) instead of threads. As it turns out, there is a nice Julia function asyncmap exactly for this purpose.
This PR also adds a new BuildOptions setting called max_concurrent_runs which specifies how many notebooks can be evaluated concurrently. This defaults to 4 to be on the safe side in case a notebook starts multiple threads too and overwhelms the system.
Unfortunately, #98 turned out to be a bit of a failure. It worked locally, but ran into https://github.com/JuliaLang/Downloads.jl/issues/94 in CI on Julia 1.7.2. Luckily, starting multiple threads actually was a bit dumb because
PlutoStaticHTML.jlthen starts threads in whichPluto.jlstarts distributed workers which can start multiple threads. This PR aims to reduce the number of started threads by startingPluto.jl's workers from asynchronous tasks (coroutines) instead of threads. As it turns out, there is a nice Julia functionasyncmapexactly for this purpose.This PR also adds a new
BuildOptionssetting calledmax_concurrent_runswhich specifies how many notebooks can be evaluated concurrently. This defaults to 4 to be on the safe side in case a notebook starts multiple threads too and overwhelms the system.EDIT: Checked this in a CI run. No problems now.