Explain necessity of testing

[mleonhard]

Hi Zach, Thank you for writing this book. I think it can help increase the pace of innovation.

Section 3.4 "Testing" is written with the implicit assumption that the reader understands why a startup needs to do some testing. Unfortunately, in my experience, some startup CTOs need to learn this. I have seen some make critical errors like un-staffing testing or migrating to a Service-Oriented Architecture without integration tests. How about enhancing "Section 3.4 Testing" with information on why testing is a necessary part of the software engineering process?

I think it would be good to teach the reader:

1. The function of testing in the software development process is to catch problems earlier in the process, when they are cheaper to fix. Compare the costs to fix a problem while coding, in CI, after code review, in production. Use this to introduce the terms that show the point in the development process where the test runs: "release" test, "continuous integration" test, and "manual" test.
2. Signals that the team is underinvesting in testing work
   • production incidents
   • increasing bug count
   • accumulating tech debt
   • reducing engineering velocity
   • worsening engineer morale
3. A prioritized list of what kinds of tests to set up
   • Manual tests when the product has few features.
   • End-to-end happy-path feature tests that gate releases. These prevent production incidents.
   • Integration tests. A team uses integration tests to make sure that other teams don't break their system. Without such tests, every team must spend energy keeping aware of changes all other teams are making, so they can catch breaking changes early. Integration tests become important once the company has engineering teams with separate standup (status) meetings.
   • Unit tests.
     1. Write good unit tests for any code that has ever had bugs, even if those bugs only occurred during initial coding. Assume that code that handles dates and timezones has bugs until it has full coverage.
     2. For code that has a complicated API, write a single test, with literal input and output data. This test teaches future readers about the shape of the data that the code accepts and returns.
4. Tests are excellent documentation. They never get out of date. They teach readers how the system actually works. Engineers spend most of their time just figuring out how the system works. Therefore investing in testing pays dividends in developer velocity.
5. Define the terminology of testing.
   • a "unit test" (already defined in the book)
   • an "integration test" runs code that is normally deployed separately and makes them talk to each other. A test that mocks all of its dependencies is not an integration test.
   • a "feature test" pretends to be a user and actually uses the feature. For a web app, this test runs the app in a browser and clicks buttons and checks for correct changes to the page and correct changes to the data in the backend.
   • a "test" that runs before deployment
   • a "check" or "monitor" runs on production, after deployment
   • ...
6. Monolith testing is cheap and fast. Service-Oriented Architecture uses many deployment units, so unit tests can cover very little code and most tests must be integration tests, which are far more complicated and slow.
7. Flaky tests harm engineer velocity and morale. Therefore, put someone in charge of getting flaky tests fixed.
8. Test Dev UX is important. When tests are easy to run and run fast, people write more of them and will fix flaky ones faster. Invest in making tests easy to run and fast on engineer laptops.
9. Systems with no tests are "legacy" systems. Only the original authors can safely modify them. If the original authors leave, then the new people must add tests before they can safely add features or fix bugs.
10. Some engineers love to clean up tech debt and will do it eagerly when such changes are low-risk. Test coverage reduces the risk of all code changes, especially code cleanups. Therefore, investing in test tooling can allow your startup to benefit from these instinctive code cleaners.
11. If a system is hard to test, it will be hard to deploy and manage in production. Follow KISS and YAGNI in deployments and reap the benefits in devops and testing.

Thanks again for writing this book. I hope these suggestions can help improve the book.

Sincerely, Michael

[ZachGoldberg]

Love this, and agreed the book does little to motivate testing. I'll put up a PR in the next few days incorporating these ideas, would love your feedback!

[Naki65]

Reading this got me back when I started as Software tester ,I dint progress or working further but now I would be glad to pursue more and learn more .Whats your advice Zach ,do I have to test my knowledge first ?

[john-wells-debug]

Wow, it looks great book and a great writer can you please write content for me on my golf website? If you have any interest and knowledge about golf then please reach out and write to me.

[mleonhard]

Yes, I'm happy to review a PR.

[iamcymentho]

Hello @john-wells-debug ,

I noticed your comment about looking for someone to write about golf for your website. As a software engineer with experience in technical writing, I'm open to exploring different topics, including golf. If you're interested in my writing style and would like to discuss how I can assist you in creating engaging and informative content for your golf website, please feel free to reach out.

You can check out some of my previous articles on my Dev.to profile to get an idea of my writing style and quality.

I'd be happy to explore this opportunity further and contribute to your project.