julia> using Models, Models.TestUtils, Test
# Normal behaviour: test summary is printed, and nothing is assigned to results1
julia> results1 = @testset "test" begin
@test false
end
test: Test Failed at REPL[2]:2
Expression: false
Stacktrace:
[1] top-level scope at REPL[2]:2
[2] top-level scope at /Users/julia/buildbot/worker/package_macos64/build/usr/share/julia/stdlib/v1.4/Test/src/Test.jl:1113
[3] top-level scope at REPL[2]:2
Test Summary: | Fail Total
test | 1 1
ERROR: Some tests did not pass: 0 passed, 1 failed, 0 errored, 0 broken.
julia> results1.anynonpass
ERROR: UndefVarError: results1 not defined
# Incorrect behaviour: no test summary printed, and returns a DefaultTestSet
# indicating that tests were passed (anynonpass = false)
julia> results2 = @testset "test" begin
test_interface(FakeTemplate{PointEstimate, SingleOutput}())
@test false
end
Models API Interface Test: FakeTemplate: Test Failed at REPL[5]:3
Expression: false
Stacktrace:
[1] top-level scope at REPL[5]:3
[2] top-level scope at /Users/julia/buildbot/worker/package_macos64/build/usr/share/julia/stdlib/v1.4/Test/src/Test.jl:1113
[3] top-level scope at REPL[5]:2
Test.DefaultTestSet("test", Any[Test.DefaultTestSet(#= circular reference @-2 =#)], 0, false)
julia> results2.anynonpass
false
E.g. on Julia 1.4:
Maybe related to https://github.com/JuliaLang/julia/issues/32937?
Seems to be solved by removing
return
from this line