CI: Refactor e2e tests and add smoke tests for production environment

drikusroor commented 3 months ago

This PR adds simple short-lived smoke tests that we can use to do a sanity check after a deploy to production. Furthermore, it refactors the e2e suite as a whole by splitting up the actual e2e tests from the smoke tests so that we can execute them separately if needed. Both the e2e tests and the smoke tests rely on the same base and execution command, but one can filter which one(s) to run using the parameters --smoke and --e2e. Omitting the parameters executes both smoke and e2e tests.

The added smoke tests test the following things for now:

If you go to /server/admin, will it show you the admin login page? This tells us that the backend service is online.
If you go to /collection/thkids, does it show you the dashboard with experiments? This tells us that collections can be properly loaded and that the connection to the backend and its database works as well.
- If so, can you click on an experiment and will it show you the first experiment page? This tells us that experiments can be loaded.

Resolves #1060

This workflow shows that it worked as expected when I temporarily turned off the requirements for the e2e/smoke tests (branch / tag / post deploy / etc.) => https://github.com/Amsterdam-Music-Lab/MUSCLE/actions/runs/9610351545

drikusroor commented 3 months ago

A teensy and somewhat nitpicky request: most of the committed files don't have an empty last line. While my editor isn't bothered by that, some editors are. That's why GitHub also flags this. Please configure your editor to add an empty last line, so future collaborators/maintainers don't curse over our graves. ;-)

This is kind of new to me. If we want empty last lines in every file, shouldn't we have an (opinionated) settings file for VS Code or a code style document at least?

BeritJanssen commented 3 months ago

We have an .editorconfig which should take care of that. That has insert_final_newline = true, as a matter of fact.

drikusroor commented 3 months ago

We have an .editorconfig which should take care of that. That has insert_final_newline = true, as a matter of fact.

Ah I see.. VS Code does need a plugin to be able to work with that. Maybe it would be an idea to add it to the "Recommended plugins" in the .vscode folder or would that be overkill?

Amsterdam-Music-Lab / MUSCLE

CI: Refactor e2e tests and add smoke tests for production environment #1143