memray
commented
3 years ago I took a look at it and found no simple way to fix it. The issue is I updated the eval part by loading data from json files directly, but the pred part still relies on the traditional OpenNMT .src/.tgt pairs.
I won't be able to fix it in the near future since I'm trying to migrate things to OpenNMT 2.0 (but it may not be compatible with currently trained models). I would suggest you locally replace the code (kp_gen_eval.py) with a previous version (say this) and this may bypass it.
Let's have this issue open to have people notice this.
BiggyBing
Hi Meng, I encountered the following issue when I ran the evaluation commands ( python kp_gen_eval.py -tasks pred eval report -config config/test/config-test-keyphrase-one2seq.yml -data_dir data/keyphrase/meng17/ -ckpt_dir models/keyphrase/meng17-one2seq-kp20k-topmodels/ -output_dir output/meng17-one2seq-topbeam-selfterminating/meng17-one2many-beam10-maxlen40/ -testsets duc inspec semeval krapivin nus -gpu -1 --verbose --beam_size 10 --batch_size 32 --max_length 40 --onepass --beam_terminate topbeam --eval_topbeam).
[2021-04-22 03:11:54,383 ERROR] Error while translating Traceback (most recent call last): File "kp_gen_eval.py", line 196, in
opt=opt
File "/home/bingyang/keyphrase/transformer_gan/OpenNMT-kpg/onmt/translate/translator.py", line 362, in translate
filter_pred=self._filter_pred
File "/home/bingyang/keyphrase/transformer_gan/OpenNMT-kpg/onmt/inputters/keyphrase_dataset.py", line 197, in init
ex_dict, src_base_field, tgt_base_field)
File "/home/bingyang/keyphrase/transformer_gan/OpenNMT-kpg/onmt/inputters/dataset_base.py", line 43, in _dynamic_dict
src = src_field.tokenize(example["src"])
File "/home/bingyang/keyphrase/transformer_gan/OpenNMT-kpg/onmt/inputters/text_dataset.py", line 63, in _feature_tokenize
tokens = string.split(tok_delim)
AttributeError: 'dict' object has no attribute 'split'
Can you please help to address this issue?