search

enso-org / enso

Hybrid visual and textual functional programming.
https://enso.org
Apache License 2.0
7.31k stars 317 forks source link

Test the testing framework #9018

Open radeusgd opened 4 months ago

radeusgd commented 4 months ago

The #9001 removes the Meta_Test_Suite_Tests project as it was not really working and it was actually testing only a very limited part of the functionality.

One of the problems with it is that we were using the Standard.Test framework to test itself. While this is really cool meta-circularity, it's not really most robust as some kinds of bugs may go unnoticed in that setting (honestly rather unlikely...).

@JaroslavTulach suggested some time ago that it may be more appopriate to be testing the Standard.Test framework using another framework like JUnit. We already have a growing suite of Enso tests in JUnit, so it may make sense to include that as well.

This ticket is to migrate the deleted Meta_Test_Suite_Tests tests and re-add them as JUnit tests. We could also expand on it a bit and improve the test coverage.

JaroslavTulach commented 4 months ago

I do remember suggesting to write JUnit tests to test some behavior hard to test with Standard.Test. However I am not sure I was recommending to rewrite tests of Standard.Test to JUnit. I think testing framework shall be capable to test itself. I believe JUnit tests are also written in JUnit.

radeusgd commented 4 months ago

I do remember suggesting to write JUnit tests to test some behavior hard to test with Standard.Test. However I am not sure I was recommending to rewrite tests of Standard.Test to JUnit. I think testing framework shall be capable to test itself. I believe JUnit tests are also written in JUnit.

Oh, then I must have misunderstood your intent.

Well, in that case we can change the meaning of this ticket to bring-back the Meta_Test_Suite_Tests and get them working again (as they used to be not run on the CI and were broken).