search

mpizenberg / elm-test-rs

Fast and portable executable to run your Elm tests
BSD 3-Clause "New" or "Revised" License
81 stars 13 forks source link

How to combine the elm code of elm-test-rs and of a user successfully? #21

Closed mpizenberg closed 3 years ago

mpizenberg commented 3 years ago

elm-test-rs builds a Runner.elm module combining code from our elm/ directory and from the user project. So the question is how do we combine those two in non-problematic ways?

This question is intricately tied to two other issues:

There are at least two problems that can arise from this code union

  1. Dependency resolution may fail
  2. Compilation may fail

The problem with dependencies (1) arises if we are testing code that depends on the same packages than our code in elm/ but at incompatible versions. This is very likely to happen for applications, that specify exact dependencies in their elm.json but can also easily happen for packages. The compilation problem (2) is most likely an issue with conflicting names for exposed modules.

Incompatible dependencies

One solution to the problem of incompatible dependencies is to "vendor" all our elm code and dependencies. This is for example the approach in elm-test or in elm-review. The main disadvantage of doing so is that a standalone binary is more complex to do and heavier. Another disadvantage is that there is more work to integrate upstream improvements in our vendored code.

Module clashes

That problem also exists in the wild for elm code, not only us. Here is an example related to the Force module of elm-visualization and elm-units. The solution there is to generate new prefixed packages. A tool was made for this by pd-andy.

mpizenberg commented 3 years ago

I have the feeling that the best compromise for us is to keep the code in elm/ as an independent package. Name clashes with other packages are very unlikely since the modules are prefixed with ElmTestRunner. But we should make everything possible to make the dependencies of the code in elm/ the broadest possible. For example not elm/core >= 1.0.5 but 1.0.0 if possible, same for other dependencies. And probably try to replace or vendor billstclair/elm-xml-eeue56 and jorgengranseth/elm-string-format in the package within elm/.

In addition, in the event of an error while solving dependencies of an application, and if we note that solving is possible if we replace exact dependencies by sem-ver compatible ranges, we may inform the user that running test should be possible if they update their dependencies. Maybe a message like this.

Building the testing code failed because your dependencies are incompatible
with the ones of the code running the tests.
The tests should run however if you update your dependencies to the following:

(put a new elm.json here ...)
avh4 commented 3 years ago

I've only really worked on the node js code of elm-test and not on the Elm code, so I'm not sure how possible this is, but just to throw out an idea: maybe the runner code could be compiled separately from the user's code, and then the two resulting js files could be used together in some way to run the tests?

mpizenberg commented 3 years ago

Thanks @avh4 I'll keep that in mind! For the time being our elm code is published as the mpizenberg/elm-test-runner package and now has dependencies satisfied by literally any existing version, so the only issue is for testing elm-test-runner itself at a different version than the one used for the tests. I think we can cope with that.