Errors should not hide all test case info

[SaschaMann]

When there's an error in the code, e.g. when running an incomplete solution of Gopher's Gorgeous Lasagna, this error will be shown rather than the usual Exercism task test view. This makes it harder to approach the problems task-by-task, as well as making it hard to solve them in a TDD-inspired style.

If there's an error, I'd prefer it to still show which tasks have been completed and which task the error is associated with.

This can sometimes lead to not being able to solve exercises in order. See https://github.com/exercism/go/issues/1755

[SaschaMann]

For example errors like this:

package lasagna

// TODO: define the 'OvenTime' constant
OvenTime := 40

# lasagna [lasagna.test]
./lasagna.go:4:2: syntax error: non-declaration statement outside function body
FAIL    lasagna [build failed]
'/usr/local/go/bin/go test --short --json .' returned exit code 2: exit status 2

---

# lasagna [lasagna.test]
./lasagna.go:8:5: too many arguments to return
    have (int)
    want ()
./lasagna_test.go:39:31: RemainingOvenTime(25) used as value

package lasagna

// TODO: define the 'OvenTime' constant
const OvenTime = 40

// TODO: define the 'RemainingOvenTime()' function
func RemainingOvenTime(t int) {
    return OvenTime - t
}

---

# cars [cars.test]
./cars_assemble.go:6:19: invalid operation: 1 <= speed <= 4 (mismatched types untyped bool and untyped int)

func SuccessRate(speed int) float64 {
    if 1 <= speed <= 4 {
        return 1.0
    } else {
        return float64(speed)
    }
}

They are compiler errors but still associated with a specific task from an Exercism PoV. This association is currently lost.

[junedev]

My thoughts on this as discussed with Sascha on Slack.

• In general we cannot show test failures if there was a compiler error because the tests can't run if the code (or the tests) don't compile.
• I don't think it makes sense to artificially make some code work on the website that wouldn't work for a CLI user or in a real world scenario.
• So imo whats described here is not really a problem of the test runner but rather we need to make sure concept introductions/hints include common compiler errors related to the concept so students learn what they mean/ how to fix them.
• At least all early exercises should include proper stubs with function signatures so people don't get stuck on deciding which types to use etc as long as this is not really part of what the exercise teaches. This is already in work for lasagna.
• Additionally we need to make sure that either basics or some additional very early exercise explains what it means for the test running process that Go is a compiled language, especially for people that did not work with compiled languages before.

The test runner could maybe prefix the compiler errors with "The code did not compile:\n" to make it a bit clearer why there is no actual test output.

@exercism/go More thoughts on this?

[bitfield]

If there's an error, I'd prefer it to still show which tasks have been completed and which task the error is associated with.

I can't think of any way to make this possible. As @junedev says, if a Go file doesn't compile, that's it! Game over. None of the functions in it can be called, even if they would compile correctly in isolation.

Learning Go means learning to deal with compile errors. I agree that, even if it were possible to shield students from errors, it wouldn't be a good idea, and anyway it isn't, so the question doesn't arise. But we can detect them and give some helpful hints, for sure.