Automated tests for avoiding regressions

[renemadsen]

Is your feature request related to a problem? Please describe. Doing the last many years I've been using owlcms. Doing that time there have been versions working better than others in terms of performance/loosing connection to db and the softwares ability to gracefully recover from said problems without restarting the application. Some times we encounter problemes doing a competition, that could have been avoided if there were some automated tests to simulate the entire flow of a competition.

Describe the solution you'd like Start the process of creating any sort of e2e tests, that could be in playwright. Exact step by step description of how to set up the current playwright tests and run them. Tests that would simulate:

• Loong response times
• Verdicts by judges at the near same time as a marshall changes weight for another lifter
• Verdicts by judges just before marshall changes the next weight for the current lifter for the next lift
• Multi platform competition, with all the above happening for both platforms at the same time and scoreboards being updated accordingly

Describe alternatives you've considered Manual testing, which is a dead end and doomed to fail in the end.

Additional context I can assist in starting this process of setting up the github env

[jflamy]

Some observations:

• the testing of verdicts vs marshall changes can be done using the current junit testing framework. All these actions are events sent to the FieldOfPlay class no matter how the decisions are input (whether done by the browser with buttons, independently via phones/tablets, or via MQTT devices).
• multiplatform competitions can be simulated already with the competition simulator. The simulator sends the actual updates to the browsers, and processes decisions through the actual end-to-end flow for MQTT devices (which is a worse case scenario)

So one option would be to set up an unattended playwright environment with the various scoreboards to be monitored, and a test database with multiple platforms. There is already some stress test code that was used to create 140 groups on 3 platforms, which could be adapted.

The simulator could also be improved

• to process weight changes trough as if done by the coach at the marshal table (currently only automatic increments are done)
• to add randomness to the referee decisions amongst themselves, and with respect to marshal changes.

I don't believe there is much value in doing these specific tests through the interface, they are better done through the events.

[jflamy]

Things that could be done through the interface

• the most common (and frustrating) situation is when there is broken athlete data, or a broken migration has broken the athlete data. In such cases, the pages don't load.
• so a simple and useful test would be to load a json, and to open the pages in the typical sequence (registration data, weigh-in, announcer, scoreboards, results). The pages should all load without crashing.
• storing a series of json that caused such crashes in the past, in addition to known correct test cases, would be a very useful addition.