Continuous Testing Guide

yunks128 commented 7 months ago

Purpose

The purpose of this pull request (PR) is to introduce and document a comprehensive guide on Continuous Testing. The content covers an overview of continuous testing, a ready-to-use continuous testing plan template, and addresses valuable feedback from the community. The aim is to enhance understanding and implementation of continuous testing in modern software development.

Proposed Changes

[ADD] Added detailed sections on Continuous Testing, including an introduction, phases, and a plan template.
[ADD] Included a workflow diagram and a table for test specifications to assist in visualizing and customizing the testing approach.
[ADD] Integrated an example Continuous Testing Plan for better understanding and application.
[ADD] Incorporated community feedback on usability testing, tool selection, project maturity, priority phases, licensing, and iterative implementation.

Issues

pull-112 testing -frameworks pull requested

Testing

The changes in this PR have been reviewed and validated through the following steps:

Reviewed the content for clarity, completeness, and adherence to the proposed format.
Cross-referenced links to relevant issues to confirm proper linking.
Checked the workflow diagram for coherence and correctness.

See current draft 110-1 branch here: https://github.com/NASA-AMMOS/slim/tree/issue-110-1/docs/guides/software-lifecycle/continuous-testing

riverma commented 4 months ago

Some feedback from the Unity Project:

Consider adding "smoke tests" or "heart beat tests" to test for basic interfaces / functionality preparation for further system tests (thanks @jonathansmolenski)
Can SLIM help with creating a single dashboard for viewing all test results in one place? (thanks @LucaCinquini)
Consider llamacode instead of just llama2 for assisted unit test generation - seems to be more geared towards code. (thanks @ryanhunter-jpl)
Open source / open alternatives to TestRail? (thanks @jonathansmolenski)
U-CS is using Selenium (thanks @galenatjpl)

riverma commented 4 months ago

We may want to consider tying in and recommending The Robot Framework (or equivalent) to help automate System Tests from a variety of testing tools together.

I like how it:

Defines tests as readable code
Has a simple execution engine
Has a plugin-based system for tying together with test tools like Selenium
Can generate styled reports in HTML, PDFs, etc. FYI @LucaCinquini

riverma commented 4 months ago

@yunks128 - as soon as you’re ready, please mark this PR as “Ready for review” by moving it away “Draft” status. Thanks.

riverma commented 3 months ago

Looks like this PR contains a bunch of extra miscellaneous files. @yunks128 opened a fresh PR with just his proposed changes. Closing this PR in place of the new one: https://github.com/NASA-AMMOS/slim/pull/144

NASA-AMMOS / slim