CI/CD

[vikingphoenix]

For the purposes of the Eropad Crew, Continuous Integration is simply the services/tools used to automate the DevOps pipeline. Whereas, Continuous Deployment is the process of providing application features and fixes to the User. CI/CD may vary slightly, and for the Academics, here is a short history of CI/CD.

This topic includes: Unit, Penetration and Stress testing.

The workflow is modeled after DevOps, and is basically what the ErgoPad team currently does, and this document will formalize those steps and extend current process to full CI/CD automation.

Continuous Integration

Services/Tools

• github actions - validates overall build
• pytest - validates requirements during github build and test
• k6 (not yet implemented) - penetration testing of the API endpoints, can also be used for stress testing
• AWS monitoring - can also be setup for alerting with advanced logging and some configuration
• Alembic - although not a service, database update automation is currently manual.

Workflow

• Daily pushes to Dev, Weekly pushes to Prod for Production services
• Code reviews at minor version bumps
• Dev reviews at minor version bumps (i.e. weekly push)

Testing/User Acceptance

• Unit Testing - all defined requirements should be included in Github actions (pytest) so that current changes that change past requirements are caught before making it to Production
• Penetration and Stress testing of the API can be handled by K6 (once configured...ugh)

Continuous Deployment

Continuous Deployment extends CI by including daily builds, PR generating a production push that directly effect the user.

Services/Tools

• Vercel - generates review and to-production automation on PR
• Jenkins (CircleCI, AWS CodeDeploy, etc..) - picks up where github actions leaves off and can push all the way to production. (Jenkins is open source and highly capable)

Workflow

• The automation to have a decent CD solution is rather complex and this may forever be a Journey rather than a Destination. As my kid once told me, "it's good to have goals."

Backups/Rollbacks

The database is the only persistent storage in the pipeline, and can be the most tricky to backup depending on code changes.

Database(s)

• Daily DB - stored on S3 using common cleanup interval (*needs automation)
• Rolling back from a daily backup is rather painful if the daily transactions are lost. Fortunately much of the data for Ergopad exists on the BlockChain, lessening the pain. Backup intervals will adjust as needed.
• These backups should be used for testing

Codebase

• Checkpoint Branch on Every Production Push (ideally implemented in CD process) - create a branch that can immediately be checked out to return to previous state.

[vikingphoenix]

• Consider automation with CICD pipeline
• k6 can extract endpoints from fastapi (openapi spec); likely other solutions that may work? Some work better with GitHub?

Notes

• early on, switched to using return JSONRespose, but may consider returning to more traditional raise HTTPException
• FastAPI routes can implement more definition like responses and response_model

[vikingphoenix]

Items that should shake out of this Epic include (incomplete list):

• Jenkins
• K6
• Alembic
• Code Reviews
• Additional Github Actions tests