Farama-Foundation / miniwob-plusplus

MiniWoB++: a web interaction benchmark for reinforcement learning
https://miniwob.farama.org/
MIT License
280 stars 47 forks source link

[Proposal] Add Continuous Integration tests #11

Closed ppasupat closed 1 year ago

ppasupat commented 1 year ago

Proposal

Add Continuous Integration (CI) tests to the codebase.

Motivation

Currently the tests in the Python module require a manual pytest run. It only tests correctness and not the code quality. We want to add automatic tests to ensure both the code quality and correctness.

Pitch

Alternatives

None. This is a required project standard.

Additional context

One potential issue is that the environment is not sandboxed: it uses Selenium and depends on an external Chrome installation. This could cause flaky tests (e.g., in a small number of subtasks, navigating away from the Chrome window could fail the test). We will need to find a way to address this.

Checklist