Farama-Foundation / miniwob-plusplus

MiniWoB++: a web interaction benchmark for reinforcement learning
https://miniwob.farama.org/
MIT License
287 stars 47 forks source link

[Question] Where is the environment chase-circle ? #82

Closed gasse closed 1 year ago

gasse commented 1 year ago

Question

I see there is an HTML environment called chase-circle https://github.com/Farama-Foundation/miniwob-plusplus/blob/master/miniwob/html/miniwob/chase-circle.html

However this environment is absent from the python codebase https://github.com/Farama-Foundation/miniwob-plusplus/blob/master/miniwob/envs/miniwob_envs.py

Is there a reason for that? Has it been forgotten somehow? Or is it not ready yet to be part of the benchmark?

ppasupat commented 1 year ago

As explained in the documentation, chase-circle is one of the excluded tasks. These are time-dependent tasks that require the agent to wait for events to happen before acting. Since the Selenium environment does not have a fixed frame rate, the agent might not get to see some of those events, so we think it's less consistent to benchmark on such tasks.

gasse commented 1 year ago

@ppasupat that makes sense. Thank you for the quick reply!