Upgrade to version 2 spec

[ErikSchierboom]

If possible, this test runner should be updated to version 2 of the test runner interface specification.

The version 2 spec differs from version 1 in that the results.json file contains detailed information on individual tests. This information includes things like:

• The test status (passing/failing)
• The error message (if failing)
• The test code that was used to run the test

You are completely free on how you want to extract the test code, either via the AST or via finding the test code in the source code text via string manipulation.

Let me know if there are any questions.

[cdimitroulas]

Hi @ErikSchierboom - I'm interested in looking into this issue.

Having dug into it a little bit and having compared with other test runners like the typescript one, it seems to me that I would need to implement a custom Hspec formatter to output the results.json file in the necessary format to match the test runner interface spec.

A few questions from my side:

• Where would the best place to put this code for the Hspec formatter be? Is this what the pre-compiled/ folder is for?
• Does my general idea for the approach sound correct?

Edit: oh, I think I understand what the pre-compiled folder is for now. It's just to pre-compile certain libraries within the test runner Docker image so that running the tests on an exercise is much faster! So that might not be a good place for the custom Hspec formatter... unless I make the custom formatter a dependency that is also precompiled.

I think what might be challenging here is that the Hspec formatter would actually need to be configured inside the exercise's test suite code :thinking: I need to think about this some more...

[cdimitroulas]

I think if I tackle this I would do a first pass which doesn't include the test_code property as that is a lot more involved due to the need for AST parsing. Would that be acceptable?

[meatball133]

The pre-compiled/ folder is for pre-loading libaries, this is to make compile time shorter while testing exercises

[cdimitroulas]

Looks like it's working nicely