Closed nicholaschenai closed 10 months ago
task 615-619 labels: all of them have
"locator": "[...document.querySelector('.submission__inner').querySelectorAll('[href],[src]')].map(elem => elem.getAttribute('href') || elem.getAttribute('src')).join(' ')",
however, in webarena/evaluation_harness/evaluators.py class HTMLContentExactEvaluator(Evaluator) line 236 on, locators are expected to be empty or startswith "document." or "func.". Hence, ValueError is raised for 615-619
Good catch! #54 has fixed it!
task 615-619 labels: all of them have
"locator": "[...document.querySelector('.submission__inner').querySelectorAll('[href],[src]')].map(elem => elem.getAttribute('href') || elem.getAttribute('src')).join(' ')",
however, in webarena/evaluation_harness/evaluators.py class HTMLContentExactEvaluator(Evaluator) line 236 on, locators are expected to be empty or startswith "document." or "func.". Hence, ValueError is raised for 615-619