search

facebookresearch / DrQA

Reading Wikipedia to Answer Open-Domain Questions
Other
4.48k stars 898 forks source link

scripts/pipeline/interactive.py error #213

Open jelly-FF opened 5 years ago

jelly-FF commented 5 years ago

scripts/retriever/interactive.py and scripts/reader/interactive.py all works well. But when I try to run scripts/pipeline/interactive.py.

It shows: python scripts/pipeline/interactive.py 04/03/2019 05:55:35 PM: [ CUDA enabled (GPU -1) ] 04/03/2019 05:55:35 PM: [ Initializing pipeline... ] 04/03/2019 05:55:35 PM: [ Initializing document ranker... ] 04/03/2019 05:55:35 PM: [ Loading /home/xuelif/Documents/NLP/DrQA/data/wikipedia/docs-tfidf-ngram=2-hash=16777216-tokenizer=simple.npz ] 04/03/2019 05:56:28 PM: [ Initializing document reader... ] 04/03/2019 05:56:28 PM: [ Loading model /home/xuelif/Documents/NLP/DrQA/data/reader/multitask.mdl ] 04/03/2019 05:56:33 PM: [ Initializing tokenizers and document retrievers... ] Traceback (most recent call last): File "scripts/pipeline/interactive.py", line 70, in tokenizer=args.tokenizer File "/home/xuelif/Documents/NLP/DrQA/drqa/pipeline/drqa.py", line 146, in init initargs=(tok_class, tok_opts, db_class, db_opts, fixed_candidates) File "/home/xuelif/Documents/anaconda3/envs/autopaper/lib/python3.6/multiprocessing/context.py", line 119, in Pool context=self.get_context()) File "/home/xuelif/Documents/anaconda3/envs/autopaper/lib/python3.6/multiprocessing/pool.py", line 175, in init self._repopulate_pool() File "/home/xuelif/Documents/anaconda3/envs/autopaper/lib/python3.6/multiprocessing/pool.py", line 236, in _repopulate_pool self._wrap_exception) File "/home/xuelif/Documents/anaconda3/envs/autopaper/lib/python3.6/multiprocessing/pool.py", line 255, in _repopulate_pool_static w.start() File "/home/xuelif/Documents/anaconda3/envs/autopaper/lib/python3.6/multiprocessing/process.py", line 105, in start self._popen = self._Popen(self) File "/home/xuelif/Documents/anaconda3/envs/autopaper/lib/python3.6/multiprocessing/context.py", line 277, in _Popen return Popen(process_obj) File "/home/xuelif/Documents/anaconda3/envs/autopaper/lib/python3.6/multiprocessing/popen_fork.py", line 19, in init self._launch(process_obj) File "/home/xuelif/Documents/anaconda3/envs/autopaper/lib/python3.6/multiprocessing/popen_fork.py", line 66, in _launch self.pid = os.fork() OSError: [Errno 12] Cannot allocate memory

P.S. I have downloaded the latest CoreNLPTokenizer from Stanford. (http://nlp.stanford.edu/software/stanford-corenlp-full-2018-10-05.zip) Without the lastest corenlp, running scripts/reader/interactive.py will return TIMEOUT error. python scripts/reader/interactive.py 04/03/2019 04:53:11 PM: [ CUDA enabled (GPU -1) ] 04/03/2019 04:53:11 PM: [ Initializing model... ] 04/03/2019 04:53:11 PM: [ Loading model /home/xuelif/Documents/NLP/DrQA/data/reader/single.mdl ] 04/03/2019 04:53:12 PM: [ Initializing tokenizer... ] Traceback (most recent call last): File "/home/xuelif/Documents/anaconda3/envs/autopaper/lib/python3.6/site-packages/pexpect/expect.py", line 99, in expect_loop incoming = spawn.read_nonblocking(spawn.maxread, timeout) File "/home/xuelif/Documents/anaconda3/envs/autopaper/lib/python3.6/site-packages/pexpect/pty_spawn.py", line 462, in read_nonblocking raise TIMEOUT('Timeout exceeded.') pexpect.exceptions.TIMEOUT: Timeout exceeded.

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "scripts/reader/interactive.py", line 53, in normalize=not args.no_normalize) File "/home/xuelif/Documents/NLP/DrQA/drqa/reader/predictor.py", line 84, in init self.tokenizer = tokenizer_class(annotators=annotators) File "/home/xuelif/Documents/NLP/DrQA/drqa/tokenizers/corenlp_tokenizer.py", line 33, in init self._launch() File "/home/xuelif/Documents/NLP/DrQA/drqa/tokenizers/corenlp_tokenizer.py", line 61, in _launch self.corenlp.expect_exact('NLP>', searchwindowsize=100) File "/home/xuelif/Documents/anaconda3/envs/autopaper/lib/python3.6/site-packages/pexpect/spawnbase.py", line 390, in expect_exact return exp.expect_loop(timeout) File "/home/xuelif/Documents/anaconda3/envs/autopaper/lib/python3.6/site-packages/pexpect/expect.py", line 107, in expect_loop return self.timeout(e) File "/home/xuelif/Documents/anaconda3/envs/autopaper/lib/python3.6/site-packages/pexpect/expect.py", line 70, in timeout raise TIMEOUT(msg) pexpect.exceptions.TIMEOUT: Timeout exceeded. <pexpect.pty_spawn.spawn object at 0x7fac9a550588> command: /bin/bash args: ['/bin/bash'] buffer (last 100 chars): b'cuments/NLP/DrQA\x07\x1b[01;32mxuelif@xuelif-HP-Z4-G4-Workstation\x1b[00m:\x1b[01;34m~/Documents/NLP/DrQA\x1b[00m$ ' before (last 100 chars): b'cuments/NLP/DrQA\x07\x1b[01;32mxuelif@xuelif-HP-Z4-G4-Workstation\x1b[00m:\x1b[01;34m~/Documents/NLP/DrQA\x1b[00m$ ' after: <class 'pexpect.exceptions.TIMEOUT'> match: None match_index: None exitstatus: None flag_eof: False pid: 7694 child_fd: 15 closed: False timeout: 60 delimiter: <class 'pexpect.exceptions.EOF'> logfile: None logfile_read: None logfile_send: None maxread: 100000 ignorecase: False searchwindowsize: None delaybeforesend: 0 delayafterclose: 0.1 delayafterterminate: 0.1 searcher: searcher_string: 0: "b'NLP>'"

anushkmittal commented 4 years ago

230 should solve it.

krutikabapat commented 4 years ago

I was facing the same issue. I solved it by adding the following line of code in my bashrc file. export CLASSPATH=/PATH TO DIRECTORY/stanford-corenlp-3.9.2.jar:/PATH TO DIRECTORY/stanford-corenlp-3.9.2-models.jar

Martin36 commented 3 years ago

230 should solve it.

This did not solve the issue for me. I still get the following stack trace, after downgrading the packages: 

Traceback (most recent call last):
  File "scripts/pipeline/interactive.py", line 70, in <module>
    tokenizer=args.tokenizer
  File "/mnt/d/Documents/NLP/DrQA/drqa/pipeline/drqa.py", line 146, in __init__
    initargs=(tok_class, tok_opts, db_class, db_opts, fixed_candidates)
  File "/home/martin/miniconda3/lib/python3.7/multiprocessing/context.py", line 119, in Pool
    context=self.get_context())
  File "/home/martin/miniconda3/lib/python3.7/multiprocessing/pool.py", line 176, in __init__
    self._repopulate_pool()
  File "/home/martin/miniconda3/lib/python3.7/multiprocessing/pool.py", line 241, in _repopulate_pool
    w.start()
  File "/home/martin/miniconda3/lib/python3.7/multiprocessing/process.py", line 112, in start
    self._popen = self._Popen(self)
  File "/home/martin/miniconda3/lib/python3.7/multiprocessing/context.py", line 277, in _Popen
    return Popen(process_obj)
  File "/home/martin/miniconda3/lib/python3.7/multiprocessing/popen_fork.py", line 20, in __init__
    self._launch(process_obj)
  File "/home/martin/miniconda3/lib/python3.7/multiprocessing/popen_fork.py", line 70, in _launch
    self.pid = os.fork()
OSError: [Errno 12] Cannot allocate memory
anushkmittal commented 3 years ago

@Martin36 looks like a RAM issue to me. try upgrading the memory?

Martin36 commented 3 years ago

@Martin36 looks like a RAM issue to me. try upgrading the memory?

I believe it was a combination of RAM and disk space memory. My main drive had only 128 GB and most of that was allocated. It was probably only 20 GB or so left. So I decided to reinstall Windows on my secondary SSD drive, which is 500 GB, and then it worked.