Add Software Testing Module

[Cmurilochem]

Idea

Learning objectives

• Appreciate the importance of testing software
• Understand the various benefits of testing
• Understand the types of tests and what info they convey
• Get familiar with the idea of continuous integration and its importance
• Refer researchers to useful, more advanced resources

Planned exercises

1. Two multiple choice questions on software testing and why it is important.
2. Matching activity worksheet where the main purpose is to match the type of test with its description/purpose.
3. Describe a scenario where a team of researchers is collaboratively developing a research software but the project is facing several problems with code integration and breaks/bugs with new functionalities. Choose a correct alternative that describes what is CI and its benefits for this project.

Design of the Slides Deck

What is "software testing"?

• Software testing is the process of evaluating and verifying that a software and its components meet specified requirements and work as expected.
• Its primary purpose is to identify defects, errors, flaws, or bugs early in the development process.
• Software testing can be done manually, where a human actively uses the software to observe for defects, or automatically, where test scripts or tools are used to execute tests.
• Untested software can be compared to uncalibrated experimental devices.

Consequences of not testing your (research) software

• https://en.wikipedia.org/wiki/Ariane_flight_V88
  • (1996) This European Space Agency rocket was destroyed just 40 seconds after its launch due to a software bug.
• https://www.science.org/doi/full/10.1126/science.314.5807.1856
  • (2006) A software flaw (a plus sign which should have been a minus) lead to a retraction of 5 scientific articles.
• https://arstechnica.com/information-technology/2019/10/chemists-discover-cross-platform-python-scripts-not-so-cross-platform/
  • (2019) Researchers found that a programming error in a set of Python scripts commonly used for computational analysis of chemistry data returned varying results based on which operating system they were run on—throwing doubt on the results of more than 150 published chemistry studies.
• https://www.wired.com/2010/11/1110mars-climate-observer-report/
  • (1999) The loss of the Mars Climate Orbiter in 1999 is another prominent example of a catastrophe caused by a software error, specifically a unit conversion mistake.
• https://en.wikipedia.org/wiki/Therac-25
  • (1980) A tragic example of software bugs leading to loss of life, the Therac-25 radiation therapy machine had a flaw in its software, which resulted in patients receiving lethal doses of radiation.
• https://en.wikipedia.org/wiki/Year_2000_problem
  • (2000) The Y2K bug was a widespread computer flaw that was expected to cause chaos as the date changed from December 31, 1999, to January 1, 2000.
• https://en.wikipedia.org/wiki/Knight_Capital_Group
  • (2013) Knight Capital, a financial services firm, lost around $440 million within less than an hour due to a software glitch in its trading algorithms.
• https://en.wikipedia.org/wiki/Boeing_737_MAX_groundings
  • (2018-2019) Software issues with the Boeing 737 MAX's Maneuvering Characteristics Augmentation System (MCAS) were implicated in two fatal crashes

What can tests help you do?

• Tests play a crucial role in ensuring reliability, validity, and reproducibility of your software's results
• Tests help users to easily verify that they correctly installed your software, giving examples of how to use it.
• Tests help other developers to contribute to your software and modify things with confidence that nothing is breaking.
• Tests preserving expected functionality
• Manage complexity: If code is easy to test, it’s probably easier to understand and maintain.

Types of tests

• Unit tests: test functions one at a time.
• Integration tests: test how parts work together.
• System tests: test the whole thing running.
• Regression tests: check same results as before.

Continuous Integration (CI)

CI is a practice in software development where team members regularly integrate their individual code changes, often several times a day, into a shared remote repository. After every integration, a working version of the project is build and the software's test suite is run automatically, making it possible to identify defects, errors, flaws, or bugs introduced by changes early. This process helps keep the code up-to-date and reduces the chances of major issues when combining everyone's work.

### Importance of CI CI has a number of key benefits: - Helps bugs to be found early, minimising their damage and making them easier to fix. - Keeps project contributors up to date with each other’s work so they can benefit from new features as soon as possible. - Encourages users to write tests. - Automates running of tests. - Ensures tests are run frequently. ## Useful resources for learning about software testing & CI/CD - https://the-turing-way.netlify.app/reproducible-research/testing - https://coderefinery.github.io/testing/motivation/ - https://www.ibm.com/topics/software-testing - https://the-turing-way.netlify.app/reproducible-research/ci - https://coderefinery.github.io/testing/continuous-integration/

[JaroCamphuijsen]

Hi Carlos. Looks like a good start. I think eventually most of the automation content will go into the CI/CD module (whenever we decide to make that) but for now you can definitely put it in here.

Regarding the learning objectives, I would also add a short chapter about testing types and what they are useful for, researchers could come up with having heard about terms like integration test vs unit test vs regression test etc. They might want a short description of what these different types of tests mean.

If I understand correctly you now specified the design and structure of the main slide deck? It looks good, however it might be a good idea to first think about what assignments/exercises you would like to make such that it will be tested that the learning objectives are actually covered.