search

yiqingxyq / Eider

Source code for paper "EIDER: Empowering Document-level Relation Extraction with Efficient Evidence Extraction and Inference-stage Fusion", ACL Findings, 2022
39 stars 10 forks source link

IndexError: list index out of range #1

Closed ady-yu closed 2 years ago

ady-yu commented 2 years ago

when running the command bash scripts/test_bert.sh eider test hoi, it occurs the following mistakes. image Besides, when running the command bash scripts/train_bert.sh eider test hoi, the best F1 only get 0.61158. image

yiqingxyq commented 2 years ago

I see. The author of DocRED has updated dev.json after I downloaded it. Please check the original version of dev.json (with 1000 documents): https://drive.google.com/drive/folders/1owp7ZRbrMl_s1ljIh6AvnmniLJSliV6h?usp=sharing Thanks for raising this issue!

It's actually normal to get F1=61.15 after training. You'll likely see another performance gain after running test_bert.sh

ady-yu commented 2 years ago

I see. The author of DocRED has updated dev.json after I downloaded it. Please check the original version of dev.json (with 1000 documents): https://drive.google.com/drive/folders/1owp7ZRbrMl_s1ljIh6AvnmniLJSliV6h?usp=sharing Thanks for raising this issue!

It's actually normal to get F1=61.15 after training. You'll likely see another performance gain after running test_bert.sh

Thanks you for you reply!!! I wonder the command bash scripts/test_bert.sh eider test hoi occupy how much GPU memory. It seems 16G is not enough!

yiqingxyq commented 2 years ago

I see. The author of DocRED has updated dev.json after I downloaded it. Please check the original version of dev.json (with 1000 documents): https://drive.google.com/drive/folders/1owp7ZRbrMl_s1ljIh6AvnmniLJSliV6h?usp=sharing Thanks for raising this issue! It's actually normal to get F1=61.15 after training. You'll likely see another performance gain after running test_bert.sh

Thanks you for you reply!!! I wonder the command bash scripts/test_bert.sh eider test hoi occupy how much GPU memory. It seems 16G is not enough!

Can your set evi_eval_mode=none in test_bert.sh? The memory is 6571M on my computer after setting it.

evi_eval_mode is to evaluate the evidence F1 (which is not necessary)