celery / billiard

Multiprocessing Pool Extensions
Other
417 stars 252 forks source link

Hangs when trying to call `os.close` #153

Open jakirkham opened 9 years ago

jakirkham commented 9 years ago

Not sure what the issue is here.

Traceback (most recent call last):
  File "/zopt/conda/envs/nanshenv/lib/python2.7/site-packages/billiard/context.py", line 138, in Queue
    return Queue(maxsize, ctx=self.get_context())
  File "/zopt/conda/envs/nanshenv/lib/python2.7/site-packages/billiard/queues.py", line 45, in __init__
    self._rlock = ctx.Lock()
  File "/zopt/conda/envs/nanshenv/lib/python2.7/site-packages/billiard/context.py", line 103, in Lock
    return Lock(ctx=self.get_context())
  File "/zopt/conda/envs/nanshenv/lib/python2.7/site-packages/billiard/synchronize.py", line 182, in __init__
    SemLock.__init__(self, SEMAPHORE, 1, 1, ctx=ctx)
  File "/zopt/conda/envs/nanshenv/lib/python2.7/site-packages/billiard/synchronize.py", line 101, in __init__
    register(self._semlock.name)
  File "/zopt/conda/envs/nanshenv/lib/python2.7/site-packages/billiard/semaphore_tracker.py", line 71, in register
    self._send('REGISTER', name)
  File "/zopt/conda/envs/nanshenv/lib/python2.7/site-packages/billiard/semaphore_tracker.py", line 78, in _send
    self.ensure_running()
  File "/zopt/conda/envs/nanshenv/lib/python2.7/site-packages/billiard/semaphore_tracker.py", line 62, in ensure_running
    os.close(w)
OSError: [Errno 9] Bad file descriptor
ask commented 9 years ago

Are you on Windows or are you using FORCE_EXECV/forking_enable?

jakirkham commented 9 years ago

This is on Mac, actually. I should add that I am using billiard.get_context("spawn") to create a spawn context that I use for launching a single billiard.Process.

ask commented 9 years ago

Ok, pretty sure I have only tested fork so far, so that probably requires some work

jakirkham commented 9 years ago

Alright, let me know if I can help, I am very interested in getting this to work.

thedrow commented 7 years ago

@jakirkham Feel free to contribute a PR.

jakirkham commented 7 years ago

Unfortunately I don't know the cause. So I don't know how to fix it either.

thedrow commented 7 years ago

We need a test case at the very least to debug this. I'm not sure you'll be able to provide one since so much time has passed. Sorry about that,

If you can, please do.