ThePix / QuestJS

A major re-write of Quest that is written in JavaScript and will run in the browser.
MIT License
66 stars 13 forks source link

Tests need improvement #9

Closed KilianKilmister closed 3 years ago

KilianKilmister commented 4 years ago

I adjusted everything again to work with your last commit, and started testing.
I haven't started investigationg the few errors i got, i will do that later. This issue is about the general testing. I wrote the restults [to a file] (https://github.com/KilianKilmister/QuestJS/blob/master/testlog.md).

There are multiple issues i have with them:

  1. Test reporting needs formatting:
    • The reports are very hard to read and the the spacing makes them unnescessarily large.
    • I strongly suggest to use Markdown, as it's very readable in plain text form, too.
    • The results need to be highlighted in some way.
  2. The reports need to be exportable. as a file.
  3. An easy way to automate testing should be implemented.
  4. There should be an option to log to the command line.
  5. Tests need to be chainable and possible to be executed conditionnally.
    • currently the tests will report many false positives caused by an earlyer test.
    • successive test chains should abort or adjust in case of a failure in the chain.
    • this can also automate formatting for test Blocks.
  6. Tests need an indicator that they are actually running so it can't look like they are just timing out.
  7. Testing a game needs to be implemented differently.
    • The current way of implementation makes it impossible for a game to be pluggabel and testable at the same time.
    • The way the test flow is added to the global test object is a flawed substitution for a proper data requesting.
    • i sugest creating a simple script interpreter that would read instructions from a JSON. This would allow pointing to a json object in the game-data, which tells the game-engine to where to find the JSON-test-script and it would then pass the script to the interpreter
  8. test logs should list the line number of the actual test.
  9. test logs should include the most recent part of the call-stack to make finding the actial issue in code easier.
ThePix commented 3 years ago

I am going to close this as I do not think any progress is going to be made.

Killian's approach - broadly Node.js - perfectly valid, but I feel adds a layer of complication for casual users that I want to avoid. As is, Quest 6 can be downed, extracted and opened in a browser, and it works. Adding Node.js significantly increases that initial barrier.

I fully recognise there are issues with unit testing as it is, because I routinely encounter them. "currently the tests will report many false positives caused by an earlyer test." Oh, God yes!