search

exercism / elm-test-runner

GNU Affero General Public License v3.0
3 stars 5 forks source link

More independence from elm-test-rs with custom reports #39

Open mpizenberg opened 2 years ago

mpizenberg commented 2 years ago

Due to an issue with the name field in the exercism report of elm-test-rs, we needed to update both the mpizenberg/elm-test-runner elm package with the fix (https://github.com/mpizenberg/elm-test-runner/pull/12) and then the elm-test-rs executable to use that new version of the reporter package.

Related to another issue, I came to the conclusion that it would be possible (I think) to enable custom reporters, configured directly with command line arguments of the elm-test-rs executable. I've detailed the approach a bit more in that issue: https://github.com/mpizenberg/elm-test-runner/issues/11

I'm linking this here, because I think doing this will benefit exercism since essentially, we could be using it like follows:

elm-test-rs --report "custom" --report-package "exercism/elm-reporter@1.0.0" --report-fn "ExercismReporter.implementation"

This would enable any update to the exercism reporter code, as long as it's still compatible with the version of mpizenberg/elm-test-runner used by elm-test-rs.

jiegillet commented 2 years ago

Semi-related question: Currently we have to merge elm-test-rs output to the Exercism extract-test-code output. Would it be possible to do all of those operations in the same place? Can extract-test-code be used via exercism/elm-reporter somehow?

mpizenberg commented 2 years ago

Not currently, because the reporter of the test runner doesn't have access to the associated code.

Actually not only the reporter, there is no part of the test runner that can match exactly an instance of Test with corresponding code. This is because of the dynamic nature of creating tests value because any expression can be used to create test values in a test file. Only top level tests are detected by the test runner and passed to elm-exploration/test that extracts the full detailed list of test values from that Test.