elm-explorations / test

Write unit and fuzz tests for Elm code.
https://package.elm-lang.org/packages/elm-explorations/test/latest
BSD 3-Clause "New" or "Revised" License
237 stars 39 forks source link

Fix CI and update to elm 0.19.1 #143

Closed harrysarson closed 4 years ago

harrysarson commented 4 years ago
harrysarson commented 4 years ago

We could just pick the first commit (the second two are more controversial)

harrysarson commented 4 years ago

I do not understand the build logs:

> cat package.json && elm --version && elm make --docs=documentation.json && (cd tests && ./test.sh) && elm-format --validate src tests/src benchmarks

{
  "name": "elm-test-tests",
  "version": "0.0.0",
  "description": "tests for elm-test, so you can elm-test while you elm-test",
  "scripts": {
    "test": "cat package.json && elm --version && elm make --docs=documentation.json && (cd tests && ./test.sh) && elm-format --validate src tests/src benchmarks"
  },
  "author": "Richard Feldman",
  "license": "BSD-3-Clause",
  "devDependencies": {
    "elm": "latest-0.19.0",
    "elm-format": "latest-0.19.0"
  },
  "dependencies": {}
}
0.19.1
... etc

why am I getting elm 0.19.1?

harrysarson commented 4 years ago

Okay, I tracked the issue with failing CI down. The problem is that node_modules are cached and therefore the wrong version of elm is used. The solution is to delete the cache, although someone with commit access to this repository will need to do so (see screenshot of CI from my fork of this repo).

image


Tagging @rtfeldman and @mgold as folk I think have commit access here.

drathier commented 4 years ago

I have commit access. Nuked the caches and rebuilt, it's green now. Looks like you've simplified the build by only running on x64 linux, which isn't ideal wrt floating point rounding, so maybe change that back? I guess arm64 and x64 on mac and linux at least are a good mix. lts/* apparently tests the latest lts only, not sure if we want to test against more node versions.

harrysarson commented 4 years ago

Thanks! Prior to this change we only ran of x86 Linux and x86 macos. Those platforms will have identical floating point behaviour (or so I believe as they both have 64 bit intel processors)

drathier commented 4 years ago

Iirc we had exact matching float tests for expectWithin at first, but it failed in CI. Then I ran the tests locally on Windows and Mac and they differed. Afaik architecture, cpu generation and OS can all affect float calculations, so I think we should test broadly.

harrysarson commented 4 years ago

@drathier you have convinced me that most my changes to .travis.yml were misguided. I have rebased and now only the redundant parts of the ci config have been changed.

This is good to go from my point of view.

drathier commented 4 years ago

All good then :) Thanks for helping out!