gatoatigrado / vimap

Variations on imap
Other
12 stars 7 forks source link

More issues with multiprocessing queue shutdown #14

Open gatoatigrado opened 10 years ago

gatoatigrado commented 10 years ago

The closes_fds_test sometimes flakes. From some real-world testing, this isn't just a test flake, it's a real problem. I added some verbosity [ http://pastebin.com/eDvF1Gm0 ], and then added a loop that checked that the pipes the queue class uses were actually closed [ http://pastebin.com/Yx5qpf3C ]. The loop freezes when checking that the output queue's write pipe is closed.

Fortunately, one can call close() manually on pipe objects, and doing this seems to fix the flakiness. It's ugly, though, and I'm creating this issue so we can track it & reference why there's some ugly-looking code.

gatoatigrado commented 10 years ago

I have a work-around which I'm going to put up a review for now, but for further investigation it looks like the writer pipe should be closed here [ link ], but there are a number of exit paths which don't run that close method.