OhadRubin / SmBop

MIT License
88 stars 35 forks source link

Problems using SmBop without values #56

Open CeadeS opened 2 years ago

CeadeS commented 2 years ago

Hello, I recently tried to train and evaluate the method without values and I ran into some problems. You can surely help me with that. I trained the model with :

python exec.py --disable_value_pred

and afterward tried to evaluate with the lines:

python eval.py --archive_path model_name  --output preds.sql 
python smbop/eval_final/evaluation.py --gold dataset/dev_gold.sql --pred preds.sql --etype all --db  dataset/database  --table dataset/tables.json

The preds.sql contained lines like:

concert.concert_id != singer.age    concert_singer
concert.concert_id != *    concert_singer
concert.concert_name != *    concert_singer

The result in every run was:


=====================   EXECUTION ACCURACY     =====================
execution            0.000                0.000                0.000                0.000                0.000               

====================== EXACT MATCHING ACCURACY =====================
exact match          0.000                0.000                0.000                0.000                0.000  

Thank you in advance!

minhkids commented 2 years ago

I think the prob here is your --gold {path} try to enter your path again and it will work!