Enable usage of elm-test like go test

[eeue56]

go test is an awesome feature of Go where the Go distribution ships with a unit testing feature. Running go test on a project will automatically find files with <x>_test.go and run them. It gives a clear output of failed cases and successful files, along with a benchmark for how long it ran for.

Not quite sure how this maps over to elm-test just yet, but the desired goal is to make it so that elm-test can be used with minimal set up.

Some thoughts

@dgtized suggested leiningen's profiles as a possible solution. Ever since he suggested I can't help but feel is the right way forward. The gist of profiles is that you could have a root elm-package.json which would be a subset of a test/elm-package.json, though with a modified sources. This is already the case for every test I've ever seen in Elm. This would avoid having two different elm-package.json files that need updating each time the deps need updating, and improve the ability for us to inject deps. For example, by default we could inject elm-test or elm-check with io into a test project, allowing us to use elm-test just like go test

[rtfeldman]

If we have local file dependencies, do we need a separate concept of profiles?

I figured if the test elm-package.json could depend on the main project by adding a dependency like "NoRedInk": "../" we'd no longer have the "update deps in 2 places" problem, and we could auto-generate this file if it didn't already exist in ./test when you run elm test from the project root.

[eeue56]

As far as I can see that's the same concept as profiles - basically the test profile extends the parent profile with the things we need for testing. Is there something else that makes it different?

[dgtized]

I think it's just simper to declare the dependencies in one place. In addition, if that package file ever was extended to provide more then dependencies (external variables, override test invocation parameters, etc) then the profiles concept is even more powerful. Autogenerating a dependency file seems like more trouble then it's worth.

[avh4]

I think we need to push a change into elm-make that allows dependencies to refer to local directories. We should aim to make testing as integrated with compilation as possible. It may be a bit slower to get changes into elm-make, but I think it is worth doing that vs adding fancy features to elm-test.

[dgtized]

Sure, I wasn't necessarily saying that elm test should be responsible for managing the project files, it might hook in to read specific variables. Pushing it upstream to elm-make seems fine, but the important thing is consolidating the package configuration (particularly dependencies) to be inclusive of tests or alternate build modes (ie targeting browser, node, self-host, etc).

[eeue56]

elm-make --tests would install test deps and compile the stuff in the test folders. Here's a couple of possible elm-package.json structures I see. The first is non-profile based, the second is profile based. In the profile based one, you would do elm-make <target>.

The first one (non-profile) works because we can use it right now without modifying elm-make upstream. We'd use the data to create the elm-package.json in the test directory with the right config.

Non-profile, extend current elm-package.json structure

{
    "version": "1.1.1",
    "summary": "A Elm dict implementation that can store any type",
    "repository": "https://github.com/eeue56/elm-all-dict.git",
    "license": "BSD3",
    "source-directories": [
        "src/"
    ],
    "test-directories": [
        "tests/"
    ],
    "exposed-modules": [
        "AllDict",
        "EveryDict"
    ],
    "dependencies": {
        "elm-lang/core": "3.0.0 <= v < 4.0.0"
    },
    "test-dependencies": {
        "elm-lang/test": "1.0.0 <= v < 2.0.0"
    },
    "elm-version": "0.16.0 <= v < 0.17.0"
}

vs

profile, elm-make upstream replacement

{
    "version": "1.1.1",
    "summary": "A Elm dict implementation that can store any type",
    "repository": "https://github.com/eeue56/elm-all-dict.git",
    "license": "BSD3",
    "elm-version": "0.16.0 <= v < 0.17.0",
    "targets": [ 
        {
            "target": "main",

            "source-directories": [
                "src/"
            ],            
            "exposed-modules": [
                "AllDict",
                "EveryDict"
            ],
            "dependencies": {
                "elm-lang/core": "3.0.0 <= v < 4.0.0"
            },

        },
        { 
            "target" : "test",
            "depends-on" : "main",
            "source-directories": [
                "tests/"
            ],
            "dependencies": {
                "elm-lang/test": "1.0.0 <= v < 2.0.0"
            },
        }
    ]
}

[jhutchins]

It's been nearly a year since this was discussed, is there any preference between the two options that @eeue56 proposed? This is currently my single largest frustration with elm tooling and I would be willing to take a stab at option 1 if there was some support for a method like that. It's very similar to the approach taken by language tools like npm and maven.

[rtfeldman]

@evancz has discussed adding elm-test to the Elm Platform in a future release, and I expect making it easier to get up and running with elm-test (not to mention managing its dependencies without the current elm-package.json duplication) will be a part of that. 🙂

[harrysarson]

As elm.json files can now contain test-dependancies I think this feature is complete! Do reopen if there are more actionable things here that I have missed. (This is a very old issue and I am doing some spring cleaning in this repository.)