This is the offcial repo for the ACL-2022 paper "Learning to Reason Deductively: Math Word Problem Solving as Complex Relation Extraction".
pip3 install transformers
pip3 install accelerate
(for distributed training)Reproduce the results, simply run our scripts under scripts
folder.
For example, reproduce the results for Math23k
dataset with train/val/test setting,
bash scripts/run_math23k.sh
Run the following for the train/test setting
bash scripts/run_math23k_train_test.sh
We reproduce the main results of Roberta-base-DeductiveReasoner in the following table.
Dataset | Value Accuracy |
---|---|
Math23k (train/val/test) | 84.3 |
Math23k (train/test) | 86.0 |
MAWPS (5-fold CV) | 92.0 |
MathQA (train/val/test) | 78.6 |
SVAMP | 48.9 |
More details can be found in Appendix C in our paper.
We also provide the Roberta-base-DeductiveReasoner checkpoints that we have trained on the Math23k, MathQA and SVAMP datasets. We do not provide the 5-fold model checkpoints due to space limitation.
Dataset | Link |
---|---|
Math23k (train/dev/test setting) | Link |
Math23k (train/test setting) | Link |
MathQA | Link |
SVAMP | Link |
The data for Math23k Five-fold is not uploaded to GitHub due to slightly larger dataset size, it is uploaded here in Google Drive.
If you find this work useful, please cite our paper:
@inproceedings{jie2022learning,
title={Learning to Reason Deductively: Math Word Problem Solving as Complex Relation Extraction},
author={Jie, Zhanming and Li, Jierui and Lu, Wei},
booktitle={Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
pages={5944--5955},
year={2022}
}