How to interrupt queued threads?

[sairus7]

I've run the following code to process a batch of files in external program:

ThreadPools.@qthreads for i in 1:length(filelist)
    try
        file = filelist[i]
        run(`some external command on $file`)
    catch err
        @error err #if some task fails, we just log it and continue to the next one
    end
end

The whole process lasts for a coule of hours, but when I tried to cancel execution, I could not do it. Then I exited Julia console, but my machine continued to execute 8 external processes in parallel. When I manually kill a process, it drops CPU usage to zero for a short time, and then continues to work (seems like sheduler just starts the next item). Finally, had to restart my machine to stop it.

So, how should I interrupt queued threads?

[tro3]

This is more of a Julia question, I think. Do you get different behavior with a simple Threads.@threads? Once that external process is launched, Julia will not stop the process for you by default. But you can manually try to shut it down. If the process can be killed, you can try something like:

ThreadPools.@qthreads for i in 1:length(filelist)
    try
        file = filelist[i]
        proc = run(`some external command on $file`)
    catch err # note that a premature kill should trigger this
        kill(proc)
        @error err #if some task fails, we just log it and continue to the next one
    end
end

but as noted in this conversation, it has to be a process that will allow itself to be killed to begin with. (Note, I haven't tested the above at all - just tossing out ideas)