search

instructkr / LogicKor

한국어 언어모델 다분야 사고력 벤치마크
175 stars 31 forks source link

Add reasoning question #25

Closed smartdolphin closed 5 months ago

smartdolphin commented 5 months ago

This is an elementary school level reasoning problem. It is incredibly easy for humans, but no LLM model has provided the correct answer yet.

The problem can be solved by simply filling and emptying each water container once.

(0, 0, 3) (0, 0, 0) (0, 7, 0) (0, 0, 0) (11, 0, 0)

StableFluffy commented 5 months ago

Hi, thank you for your contribution. Unfortunately, we are not updating question list since that can add confusion to users.

Best regards,

smartdolphin commented 5 months ago

I understand your point. However, I cannot accept that the gpt-4-1106-preview reasoning singleturn score is 10 points, which is why I submitted this PR. Thanks.