chartgerink
commented
1 week ago I would recommend testing both successes and failures. This means that you test that things work as expected, and that you test your expectations when something fails.
Testing for success
Component renders
Successfully rendering components in various states can build confidence that the components are of sufficient quality. It also allows early signalling of issues, such as when component renders start failing. Successfully rendering components in a range of situations, with realistic variations of the arguments, help identify issues that will drastically affect the platform usage.
Add tests for observed bugs on an ongoing basis
By recreating bugs as closely as possible in a test when they are reported, it is possible to both verify the bug is fixed when the test passes, as well as building confidence the bug is not reintroduced at a later stage. This is a great way how unit testing can lead into regression testing (that is, that the code does not regress in quality).
When done on an ongoing basis, this creates a catalog of tests that helps maintain minimum quality over time.
Queries/mutations tests
Mocking up scenarios of queries and mutations can help streamline development by providing feedback on whether expected data is changing. We know certain output is expected in certain situations, upon which we might build components or other tooling. We know these are critical operations to application functionality.
Testing for the expected success of queries and mutations helps to ensure changes do not create a break in the expected data that is returned from these fundamental operations. Examples include queries for projects, tasks, user information.
Testing for failures
Error handling
Do components handle faulty input in a way that does not create a panic on the page? This can be done by mocking up faulty data that does not have all the arguments needed to render the component. A component is allowed to fail, and these tests ensure they fail in a way we expect and want them to.
Input validation
There are plenty of places throughout the application, both in forms but also in mutations, where there's input being provided. Where that input is known to be subject to certain restrictions (for example, passwords have to be 10 characters), do the relevant components/mutations fail as expected? It is not sufficient to test on the client side as we have to assume that client-side data can be tampered with.
In other words, these tests attempt to submit invalid data on both the client-side and server-side. We expect these submissions to fail, and if they do not, that is a problem.
Queries/mutations tests
Similar to testing for success, testing for expected failures in queries/mutations helps identify situations where users may be able to retrieve or mutate information that they should not be able to. We expect queries to fail for unauthorized users (for example a query for another user's email) and when such a test fails, it highlights we may be exposing data and creating a data breach (see also #151).
This issue helps track the work we're doing to improve testing. This item is owned by @enyeal and active participation is encouraged.
First unit tests for component renders are due before September 5th.
All are involved in helping think about valuable testing strategies to help @enyeal.