search

ymcui / cmrc2018

A Span-Extraction Dataset for Chinese Machine Reading Comprehension (CMRC 2018)
https://ymcui.github.io/cmrc2018/
Creative Commons Attribution Share Alike 4.0 International
411 stars 87 forks source link

Evaluate error (issues from old repository) #3

Closed ymcui closed 5 years ago

ymcui commented 5 years ago

@chiangyulun0914 https://github.com/ymcui/CMRC2018-DRCD-BERT/issues/1#issue-483718582

When I use the cmrc2018_evaluate.py to get EM/F1 for the DRCD_dev.json, I got this: image

Is there any solution for that?

Here is my evaluate.sh:

#!/bin/bash

#### local path
DRCD_DIR=raw_data/
EVALUATE_DIR=BERT/bert/
PREDICT_RESULT=BERT/experiment/chinese_L-12_H-768_A-12_S-512_B-2/model_ckpt

python $EVALUATE_DIR/cmrc2018_evaluate.py $DRCD_DIR/DRCD_dev.json $PREDICT_RESULT/predictions.json
ymcui commented 5 years ago

It seems that you are using python3. The original code is under python2.

Also, this repository will be read-only and transferred to a new place. If you have any further questions, please submit your issue in https://github.com/ymcui/cmrc2018/issues

ymcui commented 5 years ago

re-open if necessary.