grosser / parallel

Ruby: parallel processing made simple and fast
MIT License
4.16k stars 254 forks source link

Hangs indefinitely when used with EventedFileUpdateChecker #329

Closed bmulholland closed 2 years ago

bmulholland commented 2 years ago

When parallel and EventedFileUpdateChecker are used together, ruby execution never finishes

I originally encountered this while using tapioca, see https://github.com/Shopify/tapioca/issues/1149

I've posted a minimal repro at https://github.com/bmulholland/parallel_repro. See specifically the commit adding the ropro: https://github.com/bmulholland/parallel_repro/commit/9b505ff0fea2c6fd8340fd020a147969aec32944. Removing either line from the repro script will stop the hang.

Unfortunately I haven't been able to repro this outside of rails

bmulholland commented 2 years ago

There's some odd behaviour with the debug gem that I couldn't figure out. I think it's required to repro, which may make it a convergence of three things?

grosser commented 2 years ago

use fork instead of parallel and see how that goes

bmulholland commented 2 years ago

Do you mean replace Parallel.map([1, 2], { in_processes: 2 }) {} with fork {} in my repro? I just tried that and the process exits properly, meaning something about Parallel is required for the bug to occur.

Output with 2x fork:

DEBUGGER: Attaching after process 63103 fork to child process 63104
BUT NEVER EXITS
DEBUGGER[repro.rb#63105]: Attaching after process 63103 fork to child process 63105
[exits]

Output with 2x Parallel processes:

DEBUGGER: Attaching after process 63423 fork to child process 63424
DEBUGGER[repro.rb#63425]: Attaching after process 63423 fork to child process 63425
BUT NEVER EXITS
[does not exit]

I'm not the author of any of this code so I don't understand much of it, TBH. Just a user trying to debug.

grosser commented 2 years ago

this also hangs and is what parallel does under the hood, fork and then wait for the process to finish, so find out what makes these processes never finish and the problem is solved.

pids = 2.times.map do
  fork { puts 'fork' }
end
Process.wait pids[0]
Process.wait pids[1]
bmulholland commented 2 years ago

Thanks, appreciate your reply/help. The problem is beyond my understanding of what's going on. I suspect it's a problem in Rails, and hopefully someone more knowledgeable can improve on my repro and file an issue with them.