Add Automated Smoke Testing

[Nolij]

Tested platforms should include:

• [ ] Fabric
  • [ ] Latest Snapshot
  • [ ] Latest Release
  • [ ] 20.1
  • [ ] 18.2
  • [ ] 16.5
  • [ ] 14.4
• [ ] Legacy Fabric
  • [ ] 12.2
  • [ ] 8.9
  • [ ] 7.10
• [ ] Babric b7.3
• [ ] NeoForge
  • [ ] Latest Release
  • [ ] Latest Snapshot (if snapshot builds are usable when this is implemented)
  • [ ] 20.1
  • [ ] 20.4
• [ ] LexForge
  • [ ] 20.5
  • [ ] 20.1
  • [ ] 19.2
  • [ ] 18.2
  • [ ] 16.5
  • [ ] 14.4
  • [ ] 12.2
  • [ ] 8.9
  • [ ] 7.10

Testing should verify that each listed version & platform launches without crashing.

Additionally, the following are nice to haves that may only be applied to the most-used versions:

• Screenshot testing
• Config reload testing

Ideally this should be able to be performed locally as I imagine launch testing alone would take an extremely long time in CI on GitHub-provided runners.

[rhysdh540]

Would it be a good idea to run multiple instances of Minecraft concurrently to improve testing time?

[Nolij]

Would it be a good idea to run multiple instances of Minecraft concurrently to improve testing time?

This would probably be infeasible on GitHub runners, but on a sufficiently powerful PC it should be very doable, hence my desire for it to be able to be run locally. Ideally I'll run all the tests at the same time on my PC.

[embeddedt]

This would probably be infeasible on GitHub runners

You can use a matrix action to run multiple runners at once; up to 20 concurrently running jobs are allowed per project if I recall correctly.