pfnet-research / japanese-lm-fin-harness

Japanese Language Model Financial Evaluation Harness
MIT License
51 stars 4 forks source link

Typo in fp2 dataset's data.json file #2

Closed nishikaz closed 2 months ago

nishikaz commented 4 months ago

I believe there's a typo in the fp2 dataset's data.json file, specifically at this line: fp2/data.json#L1383

The typo is in the context for choice 3. It currently says "3.(ア)80%2(イ)330m(ウ)400m2(エ)50%", but I believe it should be "3.(ア)80%(イ)330m2(ウ)400m2(エ)50%".

Here is the original question for you to check: 問57 of FP2過去問#202105.

Can you please check and fix this? Thanks!

masanorihirano commented 4 months ago

https://github.com/pfnet-research/japanese-lm-fin-harness/pull/4/files

I made a hot fix. Could you check it? I'll merge it in the next release.

nishikaz commented 4 months ago

Thanks for the hot fix branch. I think an additional edit is required to complement your changes. I've forked the repo and added a commit to it. Could you take a look at this? https://github.com/nishikaz/japanese-lm-fin-harness/commit/2aba44da8d5ed769e2fdf8ebd4f4d8f4a342ebb2

I can open a new pull request from my forked branch, or you can add the commit to the hot fix branch yourself. (It seems I'm not allowed to commit directly to the original repo.)

I appreciate your effort in sharing very helpful data with us.

masanorihirano commented 4 months ago

Thank you for pointing out. I modified the PR. I apologize for it and thank you for your suggestion again.

masanorihirano commented 2 months ago

This PR is now merged.