No error message

[EQt]

When I start the test runner on the website on some exercises, e.g. an empty file on complex-numbers, I get

An error occurred while running your tests. This might mean that there was an issue in our infrastructure, or it might mean that you have something in your code that's causing our systems to break.

The problem stays even if I submit code where a local julia runtests.jl reports success.

When I try to run the test in the docker container as described [here][https://exercism.org/docs/building/tooling/docker] with the command below, no output results.json is produced, no output on stdout.

sudo docker container run -v /path/to/job:/mnt/exercism-iteration --network none -m 1GB exercism/julia-test-runner complex-numbers /mnt/exercism-iteration/ /mnt/exercism-iteration/

(hereby /mnt/exercism-iteration is a link to the solution/output directory).

Maybe this is related to #37 ?

[cmcaine]

That's curious. complex-numbers was one of the testcases I used when making the recent changes.

Can you please provide a zip of your solution dir (including tests) and one of us will look into it sometime?

[cmcaine]

If you'd like to debug this yourself, try running something like this julia run.jl complex-numbers ~/exercism/julia/complex-numbers .

And just sprinkle the ExercismTestRunner code with Infiltrator.jl or print debugging or whatever.

[SaschaMann]

@iHiD do you have access to the server logs for this one?

[EQt]

So I start with a fresh download I get via exercism download --exercise=complex-numbers --track=julia. Following your hint I finally get an error message

julia-test-runner$ julia run.jl complex-numbers ~/exercism/julia/complex-numbers .
ERROR: LoadError: Unknown testset.results item: Test Broken
  Skipped: exp(ComplexNumber(0, π)) ≈ ComplexNumber(-1, 0)
Stacktrace:
 [1] error(s::String)
   @ Base ./error.jl:33
 [2] (::ExercismTestReports.var"#walk!#6"{Nothing})(tests::Vector{Dict{String, Union{Nothing, String}}}, prefix::String, testset::ReportingTestSet)
   @ ExercismTestReports ~/src/julia-test-runner/src/tojson.jl:112
 [3] (::ExercismTestReports.var"#walk!#6"{Nothing})(tests::Vector{Dict{String, Union{Nothing, String}}}, prefix::String, testset::ReportingTestSet)
   @ ExercismTestReports ~/src/julia-test-runner/src/tojson.jl:109
 [4] tojson(output::String, ts::ReportingTestSet)
   @ ExercismTestReports ~/src/julia-test-runner/src/tojson.jl:131
 [5] test_runner(exercise_slug::String, solution_dir::String)
   @ ExercismTestReports ~/src/julia-test-runner/src/ExercismTestReports.jl:19
 [6] test_runner(exercise_slug::String, solution_dir::String, output_dir::String)
   @ ExercismTestReports ~/src/julia-test-runner/src/ExercismTestReports.jl:39
 [7] top-level scope
   @ ~/src/julia-test-runner/run.jl:3
in expression starting at /home/elias/src/julia-test-runner/run.jl:3

Still no results.json, though. Interestingly, if I include the parameter --project as in run.sh, no error message appears.

[cmcaine]

Nice. That looks simple enough to fix. Could you also adjust the default case so that it emits a better error, maybe?

Could you add handling for LogTestFailure, too? I think that's the only other one we don't handle:

julia> subtypes(Test.Result)
5-element Vector{Any}:
 Test.Broken
 Test.Error
 Test.Fail
 Test.LogTestFailure
 Test.Pass