search

exercism / julia

Exercism exercises in Julia.
https://exercism.org/tracks/julia
MIT License
66 stars 69 forks source link

Verbose Unit Testing Update #723

Closed depial closed 4 months ago

depial commented 4 months ago

I've updated all the the runtests.jl files to use verbose unit testing similar to what is mentioned in this issue. However, there are two slight modifications:

  1. I've changed the runtests.jl files in all of the following folders: exercises/concept, exercises/concept.wip, exercises/practice, instead of just those in exercises/practice
  2. I have made the description for the top level testset "test" in order to limit the impact on what is printed in the test name on the website, while still remaining informative for those who use the CLI.

The original description for the top level testset was the exercise name. However, this can be quite long (taking up a lot of space in the website test results), and is also rather irrelevant since the student presumably already knows what exercise they are doing. tests is shorter, standardized and informative on the CLI since it corresponds to a summary of all test outcomes (as shown below).

image

github-actions[bot] commented 4 months ago

This PR touches files which potentially affect the outcome of the tests of an exercise. This will cause all students' solutions to affected exercises to be re-tested.

If this PR does not affect the result of the test (or, for example, adds an edge case that is not worth rerunning all tests for), please add the following to the merge-commit message which will stops student's tests from re-running. Please copy-paste to avoid typos.

[no important files changed]

For more information, refer to the documentation. If you are unsure whether to add the message or not, please ping @exercism/maintainers-admin in a comment. Thank you!

depial commented 4 months ago

Due to test failures, I also modified concept.wip/encounters/runtests.jl by moving the struct definitions outside the local scope of the testset wrapper. All tests are passing now.

depial commented 4 months ago

@ErikSchierboom I've had a look at the test runner and I think the fix to remove the top-level category from the test name on the website is fairly straightforward. I had standardized the top-level testset description for all the exercises in this PR as tests in order to minimize any effects of this not being fixed, but I could change the descriptions if desired. Let me know if there is a preference.

ErikSchierboom commented 4 months ago

@ErikSchierboom I've had a look at the test runner and I think the fix to https://github.com/exercism/julia/pull/680#issuecomment-1886500751 from the test name on the website is fairly straightforward. I had standardized the top-level testset description for all the exercises in this PR as tests in order to minimize any effects of this not being fixed, but I could change the descriptions if desired. Let me know if there is a preference.

I think I'm fine with the standardized description, as the context is indeed known. Before we merge this, could you add a test case with the new runtests.jl structure to https://github.com/exercism/julia-test-runner/tree/main/test/fixtures? The test runner should work well with both the old and the new format.

depial commented 4 months ago

I've tried to add a test case to the test-runner and to fix the test name issue. I opened a PR(closed automatically by github-actions) that has both of these fixes.

depial commented 4 months ago

Thanks for the test runner PR!

Cheers!

depial commented 4 months ago

I don't know why, since all the checks passed here earlier, but CI is now failing for Julia 1.6 - macOS-latest

image

ErikSchierboom commented 4 months ago

That's just the setup workflow, so likely just an intermittent error.

ErikSchierboom commented 4 months ago

It was not, but https://github.com/exercism/julia/pull/733 fixes it.