./test.sh it served its purpose, but it's hard to maintain: Hard to update, hard to understand which files depend on which commands, why some stuff is there in the first place.
cram tests force to organize the tests around folders and it's explicit commands, and helps to keep tests small which isn't the case for most current snapshots.
With the possibility to have all stages of testing on the same input.
cram tests are fast
They run in watch mode, and only build refmt once.
cram makes it easy to generate snapshots (something we had to manually do) and diff snapshots for us with meaningful error messages
Much easier and less error-prone to add new tests/update present.
A generic test called formatTest/unit/*/syntax.re and formatTest/unit/*/syntax.rei got renamed to refmt_test/general-syntax-re and refmt_test/general-syntax-rei
This PR moves all
formatTests/unit
intorefmt_tests/
. Added the test suite as part of https://github.com/reasonml/reason/pull/2687.It doesn't delete the tests from
formatTest
and it will be done in a further PR.Part of this PR includes a README with the minimum explanation to run tests, you can take a look under refmt_test/README.md
This PR is spiritually the same as https://github.com/reasonml/reason/pull/2616 by me and https://github.com/reasonml/reason/pull/2454 by @sync (btw, the funny fact here is that it was my first attempt to contribute to reasonml/reason and failed because of the complexity around rely/esy/opam/Azure/CircleCI). In this case, I have already ensure the CI works with one single test https://github.com/reasonml/reason/pull/2687 and we can do the migration progressively.
Why
Example of a faiing test in CI
This is an example (on purpose) of tests failing:
Process
The migration path I took:
Some tests are renamed
There were a few tests with same name but different extensions:
formatTest/unit/*/wrappingTest.re
->refmt_test/wrapping-re.t
formatTest/unit_test/*/wrappingTest.rei
->refmt_test/wrapping-rei.t
formatTest/unit/*/whitespace.re
->refmt_test/whitespace-re.t
formatTest/unit_test/*/whitespace.rei
->refmt_test/whitespace-rei.t
A generic test called
formatTest/unit/*/syntax.re
andformatTest/unit/*/syntax.rei
got renamed torefmt_test/general-syntax-re
andrefmt_test/general-syntax-rei