Child Process Error

[anand-ar]

I tried to paralleize using process pool for interactive mode - that is I have same question on multiple passages I get the following error while trying to process multiprocessing.pool.RemoteTraceback: """ Traceback (most recent call last): File "/Users/arunachalam/anaconda3/envs/drqa/lib/python3.6/site-packages/ptyprocess/ptyprocess.py", line 698, in isalive pid, status = os.waitpid(self.pid, waitpid_options) ChildProcessError: [Errno 10] No child processes

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/Users/arunachalam/anaconda3/envs/drqa/lib/python3.6/site-packages/pexpect/pty_spawn.py", line 21, in _wrap_ptyprocess_err yield File "/Users/arunachalam/anaconda3/envs/drqa/lib/python3.6/site-packages/pexpect/pty_spawn.py", line 662, in isalive alive = ptyproc.isalive() File "/Users/arunachalam/anaconda3/envs/drqa/lib/python3.6/site-packages/ptyprocess/ptyprocess.py", line 705, in isalive 'on our process?') ptyprocess.ptyprocess.PtyProcessError: isalive() encountered condition where "terminated" is 0, but there was no child process. Did someone else call waitpid() on our process?

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/Users/arunachalam/anaconda3/envs/drqa/lib/python3.6/multiprocessing/pool.py", line 119, in worker result = (True, func(*args, *kwds)) File "/Users/arunachalam/anaconda3/envs/drqa/lib/python3.6/multiprocessing/pool.py", line 44, in mapstar return list(map(args)) File "scripts/reader/interactive.py", line 87, in process_evidence prediction_with_answer = predictor.predict(document_with_answer, question, candidates, 1) File "/Users/arunachalam/drqa/drqa/reader/predictor.py", line 76, in predict results = self.predict_batch([(document, question, candidates,)], top_n) File "/Users/arunachalam/drqa/drqa/reader/predictor.py", line 96, in predict_batch q_tokens = list(map(self.tokenizer.tokenize, questions)) File "/Users/arunachalam/drqa/drqa/tokenizers/corenlp_tokenizer.py", line 96, in tokenize self.corenlp.expect_exact('NLP>', searchwindowsize=100) File "/Users/arunachalam/anaconda3/envs/drqa/lib/python3.6/site-packages/pexpect/spawnbase.py", line 390, in expect_exact return exp.expect_loop(timeout) File "/Users/arunachalam/anaconda3/envs/drqa/lib/python3.6/site-packages/pexpect/expect.py", line 99, in expect_loop incoming = spawn.read_nonblocking(spawn.maxread, timeout) File "/Users/arunachalam/anaconda3/envs/drqa/lib/python3.6/site-packages/pexpect/pty_spawn.py", line 437, in read_nonblocking if not self.isalive(): File "/Users/arunachalam/anaconda3/envs/drqa/lib/python3.6/site-packages/pexpect/pty_spawn.py", line 662, in isalive alive = ptyproc.isalive() File "/Users/arunachalam/anaconda3/envs/drqa/lib/python3.6/contextlib.py", line 99, in exit self.gen.throw(type, value, traceback) File "/Users/arunachalam/anaconda3/envs/drqa/lib/python3.6/site-packages/pexpect/pty_spawn.py", line 23, in _wrap_ptyprocess_err raise ExceptionPexpect(*e.args) pexpect.exceptions.ExceptionPexpect: isalive() encountered condition where "terminated" is 0, but there was no child process. Did someone else call waitpid() on our process?

Thanks for your help

[ajfisch]

If you have written your own code, I can't debug it without seeing it.

In general interactive mode is meant to be just for demonstration. You can look at scripts/pipeline/predict.py for an example about how to make efficient batched predictions on a dataset.

[anand-ar]

Thanks - wondering what is the flow I could use for this scenario - say if I have 10 passages and just one question - I do not want the docs to be retrieved as I already have the passages

[ajfisch]

If you already have the passages, then you should be using the DocReader only. This is equivalent to the SQuAD setup. You should look at the scripts/reader directory.

[ajfisch]

Closing due to being quiet. Feel free to reopen if you still need help!