Closed depial closed 6 months ago
Hello. Thanks for opening a PR on Exercism 🙂
We ask that all changes to Exercism are discussed on our Community Forum before being opened on GitHub. To enforce this, we automatically close all PRs that are submitted. That doesn't mean your PR is rejected but that we want the initial discussion about it to happen on our forum where a wide range of key contributors across the Exercism ecosystem can weigh in.
You can use this link to copy this into a new topic on the forum. If we decide the PR is appropriate, we'll reopen it and continue with it, so please don't delete your local branch.
If you're interested in learning more about this auto-responder, please read this blog post.
Note: If this PR has been pre-approved, please link back to this PR on the forum thread and a maintainer or staff member will reopen it.
This is looking great! There is one issue though where the Dockerfile isn't built correctly. Could you look at https://github.com/exercism/julia-test-runner/actions/runs/8843674200/job/24284240727?pr=111?
I found the first error (line 1046) was because the struct ReportingTestSet
did not have a verbose field, so I've tried to define this field and adjust the other methods to work in concert. Hopefully this resolves the issue.
As an aside: at present, any time verbose = true
in testing, the test name will be suppressed in output on the website (i.e. even if/when verbose = true
is used at lower level testsets). For example:
@testset verbose = true "tests" begin
@testset verbose = true "first real testset" begin
@test 1 ==1
end
end
Would not show first real testset
in the test name on the website. However,
@testset verbose = true "tests" begin
@testset "first real testset" begin
@test 1 ==1
end
end
will show first real testset
in the name on the website. In other words, we will likely have to enforce the use of only the wrapper testset in the new format having verbose = true
.
This side effect seems like a reasonable compromise to still be able to accept both formats, since nobody has used this in other tests before and it could be caught in the review process for new exercises if someone misused verbose = true
. However, let me know if this is unacceptable.
The only other option I can think of is to use the standardized testset name tests
as an additional condition to suppress testset name output (which has its own drawbacks). This would allow the first example above to show first real testset
in the name, but then the standard naming of the wrapper testset would have to be enforced.
Just needed a little tweaking: https://github.com/exercism/julia-test-runner/pull/111/commits/f27c85feca4bb80d10086a8321f4d2ea73f05608
This PR aims to do two things:
verbose_testing/expected_results.json
andverbose_testing/runtests.jl
).I haven't contributed to this part of the project yet, so please review carefully.