search

thunlp / OpenQA

The source code of ACL 2018 paper "Denoising Distantly Supervised Open-Domain Question Answering".
MIT License
205 stars 49 forks source link

IndexError: too many indices for array & out of range #1

Closed xyznlp closed 6 years ago

xyznlp commented 6 years ago

When I run python main --batch-size 256 --model-name quasart_reader --num-epochs 10 --dataset quasart --mode reader, I got:

08/20/2018 01:34:57 PM: [ 16 ] 08/20/2018 01:34:57 PM: [ 16 ] 08/20/2018 01:34:57 PM: [ 16 ] 08/20/2018 01:34:57 PM: [ [(1, 1), (9, 9)] ] Traceback (most recent call last): File "\src\reader\model.py", line 253, in update if (tmp1.data.cpu().numpy()[0]<(score_s[i][target_s[i][j][0]]*score_e[i][target_s[i][j][1]]).data.cpu().numpy()[0]): IndexError: too many indices for array

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "main.py", line 820, in main(args) File "main.py", line 744, in main pretrain_reader(args, train_loader_with_doc, model, stats, train_exs_with_doc, train_docs) File "main.py", line 376, in pretrain_reader train_loss.update(*model.update(ex_with_doc[idx_doc], l_list, r_list, HasAnswer_list[idx_doc])) File "\src\reader\model.py", line 264, in update logger.info(target_e[i]) IndexError: list index out of range

Mrlyk423 commented 6 years ago

I did not meet this problem when running with the same command. Maybe the environment does not match? I ran my code with python3.5 and pytorch in anaconda. And I have the following pip-list: certifi (2017.11.5) cffi (1.11.2) nltk (3.2.5) numpy (1.13.3) olefile (0.44) pexpect (4.3.1) Pillow (4.3.0) pip (9.0.1) prettytable (0.7.2) ptyprocess (0.5.2) pycparser (2.18) regex (2017.12.12) scikit-learn (0.19.1) scipy (1.0.0) setuptools (36.5.0.post20170921) six (1.11.0) termcolor (1.1.0) torch (0.3.0.post4) torchvision (0.2.0) tqdm (4.19.5) wheel (0.30.0)

xyznlp commented 6 years ago

Thanks a lot ! In addition, for stanford nlp is not supported, if I use spacy as tokenizer, how about the result?

Mrlyk423 commented 6 years ago

I did not try spacy yet, and I think the result will be comparable with spacy. And if you want to use stanford nlp, you may try the install script (https://github.com/facebookresearch/DrQA/blob/master/install_corenlp.sh) provided by Danqi Chen. I also provide the stanford nlp jar used in our project in our data link.