At the moment, the tests of this package are in an poor state. I think that is normal. The package is in early stages of development and hence developer time is spent mainly building the library API and building a sufficient pool of features. Nevertheless, as the API stabilizes, we should shift development effort more into improving tests.
First, we need to write much more tests and improve our test coverage (i.e., the total amount of functions from the source code that are called and tested in the test suite). Second, we should write good quality tests. Here is a guideline for good tests:
Actually unit: test atomic, self-contained functions. Each test must test only one thing, the unit. When a test fails, it should pinpoint the location of the problem. Testing entire processing pipelines (a.k.a. integration tests) should be done only after units are covered, and only if resources/time allow for it!
Known output / Deterministic: tests defined through minimal examples that their result is known analytically are the best tests you can have! If random number generation is necessary, either test valid output range, or use seed for RNG
Robust: Test that the expected outcome is met, not the implementation details. Test that the target functionality is met without utilizing knowledge about the internals. Also, never use internal functions in the test suite.
High coverage: the more functionality of the code is tested, the better
Clean slate: each test file should be runnable by itself, and not rely on previous test files
Fast: use the minimal amount of computations to test what is necessary
Regression: Whenever a bug is fixed, a test is added for this case
Input variety: attempt to cover a wide gambit of input types
One doesn't have to worry about re-writing all tests. In fact, a PR "correcting" a single test file is already very much welcomed!
JanJereczek
commented
5 months ago
At the moment, the tests of this package are in an poor state. I think that is normal. The package is in early stages of development and hence developer time is spent mainly building the library API and building a sufficient pool of features. Nevertheless, as the API stabilizes, we should shift development effort more into improving tests.
First, we need to write much more tests and improve our test coverage (i.e., the total amount of functions from the source code that are called and tested in the test suite). Second, we should write good quality tests. Here is a guideline for good tests:
One doesn't have to worry about re-writing all tests. In fact, a PR "correcting" a single test file is already very much welcomed!