Hello @Delaunay ! This is a PR to help prevent dashboard tests failures.
Dashboard tests succeed on a local machine but sometimes fail on CI because of request timeouts. Although I don't yet know which exact piece of code is involved, I assume the issue is caused by CI insufficient resource for Orion web API to run correctly. However, the goal here is to test dashboard (ie. frontend UI), not web API (which does have its own Python tests). So I suggest to simply mock the web API for dashboard tests, making sure the mock is not more resource-consuming than the real Orion server.
To do that, this PR provides a fake Orion server, which just serves pre-recorded responses to API calls, without computing anything. It also runs dashboard tests 4 times in CI, and it seems to succeed.
What do you think?
Changes
Use a fake Orion server CI to run dashboard tests. Fake server does not compute anything, it just returns pre-recorded responses to API calls, and is then light and few resource-consuming.
Run dashboard tests 4 times
Checklist
Tests
[ ] I added corresponding tests for bug fixes and new features. If possible, the tests fail without the changes
[ ] All new and existing tests are passing ($ tox -e py38; replace 38 by your Python version if necessary)
Documentation
[ ] I have updated the relevant documentation related to my changes
Description
Hello @Delaunay ! This is a PR to help prevent dashboard tests failures.
Dashboard tests succeed on a local machine but sometimes fail on CI because of request timeouts. Although I don't yet know which exact piece of code is involved, I assume the issue is caused by CI insufficient resource for Orion web API to run correctly. However, the goal here is to test dashboard (ie. frontend UI), not web API (which does have its own Python tests). So I suggest to simply mock the web API for dashboard tests, making sure the mock is not more resource-consuming than the real Orion server.
To do that, this PR provides a fake Orion server, which just serves pre-recorded responses to API calls, without computing anything. It also runs dashboard tests 4 times in CI, and it seems to succeed.
What do you think?
Changes
Checklist
Tests
$ tox -e py38; replace38by your Python version if necessary)Documentation
Quality
$ tox -e lint)