Open preethiseshadri518 opened 2 years ago
I am interested in evaluating performance on the OntoNotes dataset using the provided trained BERT-base model (I am not training from scratch). In order to evaluate performance, I need to first run
./setup_training.sh <ontonotes/path/ontonotes-release-5.0> $data_dir
. However, I am unsure how to specify<ontonotes/path/ontonotes-release-5.0>
anddata_dir
. What is the difference between the two? I have set data_dir to be '.' and the <ontonotes/path/ontonotes-release-5.0> path to be the full path to where I have stored onotonotes-release-5.0, but I am not sure if this is correct (I also tried making data_dir the path to the directory where ontonotes-release-5.0 is stored and <ontonotes/path/ontonotes-release-5.0> to be 'ontonotes-release-5.0', but that didn't help)? Could someone provide an example of how to correctly specify these paths?I tried running
GPU=0 python evaluate.py 'bert_base'
and find that it evaluates on 0 examples. I assume that is because there is an issue with the data paths. Would appreciate any help in getting evaluate to work!Update: I saw that minimize_partition() in
minimize.py
currently writes 0 documents, which makes sense that we are evaluating on 0 examples. Not sure if this is a data path-related issue as I discussed above, or something else altogether.
Hi there, can I ask how did you manage to produce the temp files? @preethiseshadri518
I am interested in evaluating performance on the OntoNotes dataset using the provided trained BERT-base model (I am not training from scratch). In order to evaluate performance, I need to first run
./setup_training.sh <ontonotes/path/ontonotes-release-5.0> $data_dir
. However, I am unsure how to specify<ontonotes/path/ontonotes-release-5.0>
anddata_dir
. What is the difference between the two? I have set data_dir to be '.' and the <ontonotes/path/ontonotes-release-5.0> path to be the full path to where I have stored onotonotes-release-5.0, but I am not sure if this is correct (I also tried making data_dir the path to the directory where ontonotes-release-5.0 is stored and <ontonotes/path/ontonotes-release-5.0> to be 'ontonotes-release-5.0', but that didn't help)? Could someone provide an example of how to correctly specify these paths?I tried running
GPU=0 python evaluate.py 'bert_base'
and find that it evaluates on 0 examples. I assume that is because there is an issue with the data paths. Would appreciate any help in getting evaluate to work!Update: I saw that minimize_partition() in
minimize.py
currently writes 0 documents, which makes sense that we are evaluating on 0 examples. Not sure if this is a data path-related issue as I discussed above, or something else altogether.