search

google-deepmind / mathematics_dataset

This dataset code generates mathematical question and answer pairs, from a range of question types at roughly school-level difficulty.
Apache License 2.0
1.8k stars 249 forks source link

Question about baseline results #9

Closed reiinakano closed 4 years ago

reiinakano commented 5 years ago

Did you use beam search for the accuracy values reported in the paper? If so, is the accuracy top1 accuracy or top K accuracy?

davidsaxton commented 4 years ago

No beam search; the models output the most probably character sequentially for every output character.

"As is common in sequence-to-sequence models, the models predict the answer autoregressively using a greedy decoder (output majority class at each step)." (section 4.1)

reiinakano commented 4 years ago

Ah thanks, I missed that

On Wed, Nov 6, 2019, 8:10 PM David Saxton notifications@github.com wrote:

No beam search; the models output the most probably character sequentially for every output character.

"As is common in sequence-to-sequence models, the models predict the answer autoregressively using a greedy decoder (output majority class at each step)." (section 4.1)

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/deepmind/mathematics_dataset/issues/9?email_source=notifications&email_token=AEMDKVUTPAQWE2ZREAIY2UDQSKQ3NA5CNFSM4JIDWYYKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEDGFUBA#issuecomment-550263300, or unsubscribe https://github.com/notifications/unsubscribe-auth/AEMDKVRIUJFOK2EPHF2FJHDQSKQ3NANCNFSM4JIDWYYA .