search

LYH-YF / MWPToolkit

MWPToolkit is an open-source framework for math word problem(MWP) solvers.
MIT License
162 stars 37 forks source link

About the experiment of MWPBert on math23k #21

Open LzhinFdu opened 2 years ago

LzhinFdu commented 2 years ago

I got 'value accu=40.0' and found that the model uses 'bert-base-uncased' as the encoder by default. Could the reason be that I were not using a Chinese bert for math23k?

here is my instruction： python run_mwptoolkit.py --model=MWPBert --dataset=math23k --task_type=single_equation --equation_fix=prefix --test_step=5 --gpu_id=0

I tried to change 'config["pretrained_model"]' to 'bert-base-chinese'，but got some bugs which showed it doesn;t match the model......Is there any built-in method to change it?

LYH-YF commented 2 years ago
LYH-YF commented 2 years ago

you can change it with command line like --model=MWPBert --dataset=math23k --task_type=single_equation --gpu_id=0 --equation_fix=prefix --pretrained_model=bert-base-chinese

i hope this will help you

LzhinFdu commented 2 years ago

Very grateful for your help. The model works fine then. However, I ended up with a score of 66.3, which is still lower than the result given in the paper.

LYH-YF commented 2 years ago

there may be something wrong with my code when i update v0.0.6, i will check it. and i'm so sorry for that.

LYH-YF commented 2 years ago

I got value acc 82.5, the latest result of MWPBert on math23k.

here is my here is my instruction:

python run_mwptoolkit.py --model=MWPBert --dataset=math23k --equation_fix=prefix --task_type=single_equation --pretrained_model=hfl/chinese-bert-wwm-ext --test_step=5 --gpu_id=0 --train_batch_size=32 --epoch_nums=85 --learning_rate=3e-4 --encoding_learning_rate=3e-5 --vocab_level=char

and I publish the result at the result table