This pull request aims to introduce a new method, plain3, which employs a newline (\n) as the starting symbol for 3-digit addition baseline tests. Below are the summarized results comparing plain3 against the baseline plain method.
Summary of Results
For plain3: The best accuracy achieved was 95.8%
(Full data is available under identifier: plain3_results)
For baseline plain: The best accuracy achieved was 86.9%
(Full data is available under identifier: baseline_plain_results)
As seen, the plain3 method outperforms the baseline in terms of test accuracy. However, it still falls short of achieving a 97% accuracy rate.
Conclusion
The introduction of the plain3 method offers an improvement over the baseline plain method in terms of test accuracy. Further analysis is needed to understand the remaining gap from a 97% accuracy rate.
Your review and feedback on this pull request would be highly appreciated.
Introduction
This pull request aims to introduce a new method,
plain3
, which employs a newline (\n
) as the starting symbol for 3-digit addition baseline tests. Below are the summarized results comparingplain3
against the baselineplain
method.Summary of Results
For
plain3
: The best accuracy achieved was 95.8%(Full data is available under identifier:
plain3_results
)For baseline
plain
: The best accuracy achieved was 86.9%(Full data is available under identifier:
baseline_plain_results
)As seen, the
plain3
method outperforms the baseline in terms of test accuracy. However, it still falls short of achieving a 97% accuracy rate.Conclusion
The introduction of the
plain3
method offers an improvement over the baselineplain
method in terms of test accuracy. Further analysis is needed to understand the remaining gap from a 97% accuracy rate.Your review and feedback on this pull request would be highly appreciated.
appendix
plain3_results
with the commandpython train.py config2/addition/plain/train_addition_bal.py \ --ckpt_path_name="ckpt_10000.pt" \ --out_dir="out/addition_plain3/$model_root" \ --data_type='text' --data_format='plain3' \ --dataset='bal' --train_data_path="train_3digit_10000.txt" \ --eval_addition=True --start='FILE:data/bal/test_10000.txt' \ --wandb_log=False \ --exp_name="$model_root"
baseline_plain_results
with the commandpython train.py config2/addition/plain/train_addition_bal.py \ --ckpt_path_name="ckpt_10000.pt" \ --out_dir="out/addition_plain/$model_root" \ --data_type='text' --data_format='plain' \ --dataset='bal' --train_data_path="train_3digit_10000.txt" \ --eval_addition=True --start='FILE:data/bal/test_10000.txt' \ --wandb_log=False \ --exp_name="$model_root"