fix reference answer for task 301 and 302 as it requires a dict, not str

web-arena-x / webarena

Code repo for "WebArena: A Realistic Web Environment for Building Autonomous Agents"

https://webarena.dev

Apache License 2.0

676 stars 103 forks source link

fix reference answer for task 301 and 302 as it requires a dict, not str #51

Closed nicholaschenai closed 10 months ago

nicholaschenai commented 10 months ago

in webarena/evaluation_harness/evaluators.py line 127, in class StringEvaluator, reference_answers expects a dict but task 301 and 302 gives a string. fixed it to be in line with other string match annotations

nicholaschenai commented 10 months ago

alr fixed in latest merge