[FEATURE] - Tests Required for Deployment

[trafficone]

Testing

The API, front-end, and database all need to be validated before a deployment can be considered successful. If a deployment is unsuccessful, it should be rolled back.

Test Platform

There is a manual test process, but that should be automated as soon as possible. The following tools have been selected to build the tests.

• PyTest: Build unit & integration tests that can be automatically found and executed
• Playwright: Execute integration tests using a headless browser
• flask_testing: Create easily-testable versions of apps
• FastAPI Testing: Easily test APIs

Tests to Execute

Instead of enumerating each test, make sure the following areas are covered.

API

• Unit test each back-end function
• Unit test each front-end function

  Interface

• Functional tests enumerated in manual test document
• Verify correct rendering of key pages given test data

  Database

• Guarantee each function when interacting with a test DB still works properly

Relation to Deployment

• Pre-deploy and post-deploy will both have tests to run.
• If tests fail, a rollback will be performed.
• Deployment process will need to be such that a rollback is possible and easy.

  Deployment Outline

  Testing Steps

  1. Make backup of current snapshot of code & data.
  2. Copy new code, use linter to verify there are no obvious errors.
  3. Create test DB, run unit tests.
  4. Start test server, run interface and database tests.
  5. Shut down test server and delete test DB. Evaluate test results.
  6. If test passes, begin rollout.
  7. After rollout, run interface tests again on live site to verify everything is working.

Rollout Steps

1. Move new code to server location.
2. Create new DB & load data from old DB.
   1. If there are any migration queries, they should go here.
3. Logroll and backup-roll from previous DB backups.
4. Restart server.

[trafficone]

This is currently implemented in Google Sheets. Automation TBD.

[trafficone]

Rolling all testing into this issue.

16 Unit Testing for NoScrum

To prevent unintended failures on deployment, unit testing should be implemented. API Unit Testing

• Verify each API call completes its expected outcome without side effects
• Verify API calls fail gracefully on incomplete/incorrect input

Interface Unit Testing

• Verify each page renders
• Verify 404, and 401 errors occur where they need to

Non-unit Integration Testing

• Simulate interface interaction & guarantee changes don't break expected interaction touch points.

19 Add CI/CD Deployment Pipeline

Prerequisites:

• (#16) Automated Unit Testing
• Deployment Path (Potentially Migrate to Serverless)
• Rollback Process

[trafficone]

There's a lot in this feature, but I will try to chip away at it in the coming weeks. The highest priority is having tests that can be run automatically using pytest.