Hi, I have met errors during preprocessing.

[diebridge]

The error code like this: Traceback (most recent call last): File "/anaconda3/lib/python3.6/multiprocessing/process.py", line 258, in _bootstrap self.run() File "/anaconda3/lib/python3.6/multiprocessing/process.py", line 258, in _bootstrap self.run() File "/anaconda3/lib/python3.6/multiprocessing/process.py", line 93, in run self._target(*self._args, *self._kwargs) File "/anaconda3/lib/python3.6/multiprocessing/process.py", line 93, in run self._target(self._args, *self._kwargs) File "/anaconda3/lib/python3.6/multiprocessing/pool.py", line 103, in worker initializer(initargs) File "/anaconda3/lib/python3.6/multiprocessing/pool.py", line 103, in worker initializer(*initargs) File "preprocess.py", line 23, in init_tokenizer TOK = CoreNLPTokenizer(annota File "preprocess.py", line 23, in init_tokenizer TOK = CoreNLPTokenizer(annotators=annotators) tors=annotators) File "/Users/fzy/question-answering/reader/data/tokenizer.py", line 50, in init self._launch() File "/Users/fzy/question-answering/reader/data/tokenizer.py", line 73, in _launch self.corenlp.expect_exact('NLP>', searchwindowsize=100) File "/Users/fzy/question-answering/reader/data/tokenizer.py", line 50, in init self._launch() File "/anaconda3/lib/python3.6/site-packages/pexpect/spawnbase.py", line 390, in expect_exact return exp.expect_loop(timeout) File "/Users/fzy/question-answering/reader/data/tokenizer.py", line 73, in _launch self.corenlp.expect_exact('NLP>', searchwindowsize=100) File "/anaconda3/lib/python3.6/site-packages/pexpect/expect.py", line 107, in expect_loop return self.timeout(e) File "/anaconda3/lib/python3.6/site-packages/pexpect/spawnbase.py", line 390, in expect_exact return exp.expect_loop(timeout) File "/anaconda3/lib/python3.6/site-packages/pexpect/expect.py", line 70, in timeout raise TIMEOUT(msg) Traceback (most recent call last): File "/anaconda3/lib/python3.6/site-packages/pexpect/expect.py", line 107, in expect_loop return self.timeout(e) File "/anaconda3/lib/python3.6/site-packages/pexpect/expect.py", line 70, in timeout raise TIMEOUT(msg) pexpect.exceptions.TIMEOUT: Timeout exceeded.

How could I debug them? Thanks in advance!

[wzq016]

do you solve this problem? i met same error as yours.

[wzq016]

Oh, i solve this problem by downgrade java version from 11 to 8 :).

[mady143]

Hi @wzq016 , I didn't resolve this by downgrade java version from 11 to 8 again i am getting the same error

(my_qa3) launchship@launchship-ML:~/Downloads/question-answering$ python preprocess.py --data data/squad --embed-path wordvec/glove/glove.840B.300d.txt --restrict-vocab --num-characters 300 [2020-03-02 12:43:19] COMMAND: preprocess.py --data data/squad --embed-path wordvec/glove/glove.840B.300d.txt --restrict-vocab --num-characters 300 [2020-03-02 12:43:19] Arguments: {'data': 'data/squad', 'dest_dir': 'data', 'tokenizer': 'corenlp', 'num_workers': None, 'threshold': 0, 'num_words': -1, 'num_characters': 300, 'embed_path': 'wordvec/glove/glove.840B.300d.txt', 'restrict_vocab': True} [2020-03-02 12:43:19] Loaded dataset data/squad/train-v1.1.json (87599 questions, 18896 contexts) Process ForkPoolWorker-2: Process ForkPoolWorker-4: Traceback (most recent call last): Traceback (most recent call last): File "/home/launchship/Downloads/my_qa3/lib/python3.6/site-packages/pexpect/expect.py", line 99, in expect_loop incoming = spawn.read_nonblocking(spawn.maxread, timeout) File "/home/launchship/Downloads/my_qa3/lib/python3.6/site-packages/pexpect/expect.py", line 99, in expect_loop incoming = spawn.read_nonblocking(spawn.maxread, timeout) File "/home/launchship/Downloads/my_qa3/lib/python3.6/site-packages/pexpect/pty_spawn.py", line 462, in read_nonblocking raise TIMEOUT('Timeout exceeded.') File "/home/launchship/Downloads/my_qa3/lib/python3.6/site-packages/pexpect/pty_spawn.py", line 462, in read_nonblocking raise TIMEOUT('Timeout exceeded.') pexpect.exceptions.TIMEOUT: Timeout exceeded. pexpect.exceptions.TIMEOUT: Timeout exceeded.

During handling of the above exception, another exception occurred:

During handling of the above exception, another exception occurred:

Traceback (most recent call last): Traceback (most recent call last): File "/usr/lib/python3.6/multiprocessing/process.py", line 258, in _bootstrap self.run() File "/usr/lib/python3.6/multiprocessing/process.py", line 258, in _bootstrap self.run() File "/usr/lib/python3.6/multiprocessing/process.py", line 93, in run self._target(*self._args, *self._kwargs) File "/usr/lib/python3.6/multiprocessing/process.py", line 93, in run self._target(self._args, *self._kwargs) File "/usr/lib/python3.6/multiprocessing/pool.py", line 103, in worker initializer(initargs) File "/usr/lib/python3.6/multiprocessing/pool.py", line 103, in worker initializer(*initargs) File "preprocess.py", line 23, in init_tokenizer TOK = CoreNLPTokenizer(annotators=annotators) File "preprocess.py", line 23, in init_tokenizer TOK = CoreNLPTokenizer(annotators=annotators) File "/home/launchship/Downloads/question-answering/reader/data/tokenizer.py", line 50, in init self._launch() File "/home/launchship/Downloads/question-answering/reader/data/tokenizer.py", line 50, in init self._launch() File "/home/launchship/Downloads/question-answering/reader/data/tokenizer.py", line 73, in _launch self.corenlp.expect_exact('NLP>', searchwindowsize=100) File "/home/launchship/Downloads/question-answering/reader/data/tokenizer.py", line 73, in _launch self.corenlp.expect_exact('NLP>', searchwindowsize=100) File "/home/launchship/Downloads/my_qa3/lib/python3.6/site-packages/pexpect/spawnbase.py", line 390, in expect_exact return exp.expect_loop(timeout) File "/home/launchship/Downloads/my_qa3/lib/python3.6/site-packages/pexpect/spawnbase.py", line 390, in expect_exact return exp.expect_loop(timeout) File "/home/launchship/Downloads/my_qa3/lib/python3.6/site-packages/pexpect/expect.py", line 107, in expect_loop return self.timeout(e) File "/home/launchship/Downloads/my_qa3/lib/python3.6/site-packages/pexpect/expect.py", line 107, in expect_loop return self.timeout(e) File "/home/launchship/Downloads/my_qa3/lib/python3.6/site-packages/pexpect/expect.py", line 70, in timeout raise TIMEOUT(msg) File "/home/launchship/Downloads/my_qa3/lib/python3.6/site-packages/pexpect/expect.py", line 70, in timeout raise TIMEOUT(msg) pexpect.exceptions.TIMEOUT: Timeout exceeded. <pexpect.pty_spawn.spawn object at 0x7f0d52790630> command: /bin/bash args: ['/bin/bash'] buffer (last 100 chars): b'estion-answering\x07\x1b[01;32mlaunchship@launchship-ML\x1b[00m:\x1b[01;34m~/Downloads/question-answering\x1b[00m$ ' before (last 100 chars): b'estion-answering\x07\x1b[01;32mlaunchship@launchship-ML\x1b[00m:\x1b[01;34m~/Downloads/question-answering\x1b[00m$ ' after: <class 'pexpect.exceptions.TIMEOUT'> match: None match_index: None exitstatus: None flag_eof: False pid: 19267 child_fd: 12 closed: False timeout: 60 delimiter: <class 'pexpect.exceptions.EOF'> logfile: None logfile_read: None logfile_send: None maxread: 100000 ignorecase: False searchwindowsize: None delaybeforesend: 0 delayafterclose: 0.1 delayafterterminate: 0.1 searcher: searcher_string: 0: "b'NLP>'" pexpect.exceptions.TIMEOUT: Timeout exceeded. <pexpect.pty_spawn.spawn object at 0x7f0d52790400> command: /bin/bash args: ['/bin/bash'] buffer (last 100 chars): b'estion-answering\x07\x1b[01;32mlaunchship@launchship-ML\x1b[00m:\x1b[01;34m~/Downloads/question-answering\x1b[00m$ ' before (last 100 chars): b'estion-answering\x07\x1b[01;32mlaunchship@launchship-ML\x1b[00m:\x1b[01;34m~/Downloads/question-answering\x1b[00m$ ' after: <class 'pexpect.exceptions.TIMEOUT'> match: None match_index: None exitstatus: None flag_eof: False pid: 19260 child_fd: 10 closed: False timeout: 60 delimiter: <class 'pexpect.exceptions.EOF'> logfile: None logfile_read: None logfile_send: None maxread: 100000 ignorecase: False searchwindowsize: None delaybeforesend: 0 delayafterclose: 0.1 delayafterterminate: 0.1 searcher: searcher_string: 0: "b'NLP>'" Process ForkPoolWorker-1: Traceback (most recent call last): File "/home/launchship/Downloads/my_qa3/lib/python3.6/site-packages/pexpect/expect.py", line 99, in expect_loop incoming = spawn.read_nonblocking(spawn.maxread, timeout) File "/home/launchship/Downloads/my_qa3/lib/python3.6/site-packages/pexpect/pty_spawn.py", line 462, in read_nonblocking raise TIMEOUT('Timeout exceeded.') pexpect.exceptions.TIMEOUT: Timeout exceeded.

[wzq016]

Hi @wzq016 , I didn't resolve this by downgrade java version from 11 to 8 again i am getting the same error

(my_qa3) launchship@launchship-ML:~/Downloads/question-answering$ python preprocess.py --data data/squad --embed-path wordvec/glove/glove.840B.300d.txt --restrict-vocab --num-characters 300 [2020-03-02 12:43:19] COMMAND: preprocess.py --data data/squad --embed-path wordvec/glove/glove.840B.300d.txt --restrict-vocab --num-characters 300 [2020-03-02 12:43:19] Arguments: {'data': 'data/squad', 'dest_dir': 'data', 'tokenizer': 'corenlp', 'num_workers': None, 'threshold': 0, 'num_words': -1, 'num_characters': 300, 'embed_path': 'wordvec/glove/glove.840B.300d.txt', 'restrict_vocab': True} [2020-03-02 12:43:19] Loaded dataset data/squad/train-v1.1.json (87599 questions, 18896 contexts) Process ForkPoolWorker-2: Process ForkPoolWorker-4: Traceback (most recent call last): Traceback (most recent call last): File "/home/launchship/Downloads/my_qa3/lib/python3.6/site-packages/pexpect/expect.py", line 99, in expect_loop incoming = spawn.read_nonblocking(spawn.maxread, timeout) File "/home/launchship/Downloads/my_qa3/lib/python3.6/site-packages/pexpect/expect.py", line 99, in expect_loop incoming = spawn.read_nonblocking(spawn.maxread, timeout) File "/home/launchship/Downloads/my_qa3/lib/python3.6/site-packages/pexpect/pty_spawn.py", line 462, in read_nonblocking raise TIMEOUT('Timeout exceeded.') File "/home/launchship/Downloads/my_qa3/lib/python3.6/site-packages/pexpect/pty_spawn.py", line 462, in read_nonblocking raise TIMEOUT('Timeout exceeded.') pexpect.exceptions.TIMEOUT: Timeout exceeded. pexpect.exceptions.TIMEOUT: Timeout exceeded.

During handling of the above exception, another exception occurred:

During handling of the above exception, another exception occurred:

Traceback (most recent call last): Traceback (most recent call last): File "/usr/lib/python3.6/multiprocessing/process.py", line 258, in _bootstrap self.run() File "/usr/lib/python3.6/multiprocessing/process.py", line 258, in _bootstrap self.run() File "/usr/lib/python3.6/multiprocessing/process.py", line 93, in run self._target(*self._args, *self._kwargs) File "/usr/lib/python3.6/multiprocessing/process.py", line 93, in run self._target(self._args, self._kwargs) File "/usr/lib/python3.6/multiprocessing/pool.py", line 103, in worker initializer(initargs) File "/usr/lib/python3.6/multiprocessing/pool.py", line 103, in worker initializer(initargs) File "preprocess.py", line 23, in init_tokenizer TOK = CoreNLPTokenizer(annotators=annotators) File "preprocess.py", line 23, in init_tokenizer TOK = CoreNLPTokenizer(annotators=annotators) File "/home/launchship/Downloads/question-answering/reader/data/tokenizer.py", line 50, in init self._launch() File "/home/launchship/Downloads/question-answering/reader/data/tokenizer.py", line 50, in init** self._launch() File "/home/launchship/Downloads/question-answering/reader/data/tokenizer.py", line 73, in _launch self.corenlp.expect_exact('NLP>', searchwindowsize=100) File "/home/launchship/Downloads/question-answering/reader/data/tokenizer.py", line 73, in _launch self.corenlp.expect_exact('NLP>', searchwindowsize=100) File "/home/launchship/Downloads/my_qa3/lib/python3.6/site-packages/pexpect/spawnbase.py", line 390, in expect_exact return exp.expect_loop(timeout) File "/home/launchship/Downloads/my_qa3/lib/python3.6/site-packages/pexpect/spawnbase.py", line 390, in expect_exact return exp.expect_loop(timeout) File "/home/launchship/Downloads/my_qa3/lib/python3.6/site-packages/pexpect/expect.py", line 107, in expect_loop return self.timeout(e) File "/home/launchship/Downloads/my_qa3/lib/python3.6/site-packages/pexpect/expect.py", line 107, in expect_loop return self.timeout(e) File "/home/launchship/Downloads/my_qa3/lib/python3.6/site-packages/pexpect/expect.py", line 70, in timeout raise TIMEOUT(msg) File "/home/launchship/Downloads/my_qa3/lib/python3.6/site-packages/pexpect/expect.py", line 70, in timeout raise TIMEOUT(msg) pexpect.exceptions.TIMEOUT: Timeout exceeded. <pexpect.pty_spawn.spawn object at 0x7f0d52790630> command: /bin/bash args: ['/bin/bash'] buffer (last 100 chars): b'estion-answering\x07\x1b[01;32mlaunchship@launchship-ML\x1b[00m:\x1b[01;34m~/Downloads/question-answering\x1b[00m$ ' before (last 100 chars): b'estion-answering\x07\x1b[01;32mlaunchship@launchship-ML\x1b[00m:\x1b[01;34m~/Downloads/question-answering\x1b[00m$ ' after: <class 'pexpect.exceptions.TIMEOUT'> match: None match_index: None exitstatus: None flag_eof: False pid: 19267 child_fd: 12 closed: False timeout: 60 delimiter: <class 'pexpect.exceptions.EOF'> logfile: None logfile_read: None logfile_send: None maxread: 100000 ignorecase: False searchwindowsize: None delaybeforesend: 0 delayafterclose: 0.1 delayafterterminate: 0.1 searcher: searcher_string: 0: "b'NLP>'" pexpect.exceptions.TIMEOUT: Timeout exceeded. <pexpect.pty_spawn.spawn object at 0x7f0d52790400> command: /bin/bash args: ['/bin/bash'] buffer (last 100 chars): b'estion-answering\x07\x1b[01;32mlaunchship@launchship-ML\x1b[00m:\x1b[01;34m~/Downloads/question-answering\x1b[00m$ ' before (last 100 chars): b'estion-answering\x07\x1b[01;32mlaunchship@launchship-ML\x1b[00m:\x1b[01;34m~/Downloads/question-answering\x1b[00m$ ' after: <class 'pexpect.exceptions.TIMEOUT'> match: None match_index: None exitstatus: None flag_eof: False pid: 19260 child_fd: 10 closed: False timeout: 60 delimiter: <class 'pexpect.exceptions.EOF'> logfile: None logfile_read: None logfile_send: None maxread: 100000 ignorecase: False searchwindowsize: None delaybeforesend: 0 delayafterclose: 0.1 delayafterterminate: 0.1 searcher: searcher_string: 0: "b'NLP>'" Process ForkPoolWorker-1: Traceback (most recent call last): File "/home/launchship/Downloads/my_qa3/lib/python3.6/site-packages/pexpect/expect.py", line 99, in expect_loop incoming = spawn.read_nonblocking(spawn.maxread, timeout) File "/home/launchship/Downloads/my_qa3/lib/python3.6/site-packages/pexpect/pty_spawn.py", line 462, in read_nonblocking raise TIMEOUT('Timeout exceeded.') pexpect.exceptions.TIMEOUT: Timeout exceeded.

Yes, I find such problem will still occur under Java8 sometimes. I finally solve this by using Spacy tokenizer instead corenlp, i.e. try this command python preprocess.py --data data/squad --embed-path wordvec/glove/glove.840B.300d.txt --restrict-vocab --num-characters 300 --tokenizer spacy

[mady143]

Hi @wzq016 , Thank you its worked for me pre-processing is completed successfully without any error but while in training process getting an error that no a directory logs/drqa-log but i run according comannd given in the README.md file

NotADirectoryError: [Errno 20] Not a directory: '/home/launchship/Downloads/question-answering/logs/drqa.log'

[wzq016]

@mady143 this is because you need to make a directory first, i.e. mkdir logs under question-answering folder.

[mady143]

yeah already i had logs i run the command like python train.py --arch drqa --embed-path wordvec/glove/glove.840B.300d.txt --checkpoint-dir checkpoints/drqa --log-file logs

getting an error

word, count = line.rstrip().rsplit(' ', 1) ValueError: not enough values to unpack (expected 2, got 1)

[wzq016]

I don't meet such problem, so i'm not sure what is going wrong here, I only run this code on bidaf with Squad.

Seems like this is because of data problem, maybe you could try printing results here to find out the error.

[SarangSanjayGujar-lilly]

I have the same issues both with bidaf and drqa, would you please check ? @mady143 did you find any solution or work around ?