UltraStar-Deluxe / Play

Free and open source singing game with song editor for desktop, mobile, and smart TV
https://ultrastar-play.com
MIT License
381 stars 72 forks source link

Reset test context between Unity Play Mode tests #435

Open achimmihca opened 2 months ago

achimmihca commented 2 months ago

Actual behaviour

When running all Play Mode tests in sequence, then there are test failures. But when executed one after another, then they finish successfully.

This is because Unity does not reset the whole application context between Play Mode tests, which seems to be a known shortcoming of Unity (see forum thread.

This leads to a failing CI pipeline, although the game logic is correct. The test logic is broken.

Expected behaviour

Executing a test alone or in multiple sequence should not make a difference.

Implementation Hints

One has to reset all application state manually in a TearDown method, notably DontDestroyOnLoad GameObjects, and static variables.

achimmihca commented 2 months ago

Maybe there is some better lib to handle tests in Unity. Currently, Responsible is used, but I feel like it is not adding much value.

Besides the shortcoming of Unity, I am also not happy with the current test context in general. Maybe one finds a better solution design that uses existing Dependency Injection to wire test instances instead of production instances.