Improve CICD

[ygrishajev]

Problem Statement

Our current CI pipeline is limited to building a Docker image without pushing it to any registry or automating the deployment process. To streamline our workflow and ensure better control over deployments, we need to extend the CI to handle deployments, use semantic-release for versioning, generate changelogs, and manage Docker images efficiently by promoting from staging to production without a re-build.

Proposed Solution

We propose the following CI/CD workflow:

1. Feature Branch:

   • Run tests and validations exclusively within the feature branch.
   • Only allow rebases that are conflict-free to maintain branch integrity.

2. Merge to Main:

   • When a branch is merged into the main branch, a workflow is triggered.

3. Semantic Release for Versioning and Changelogs:

   • Implement semantic-release to automatically determine the version number based on conventional commits.
   • Automatically generate a changelog based on the commits.
   • The workflow updates the specific app's package.json with the new release version and commits the updated package.json and changelog to the repository.
   • Push these changes back to the origin.

4. Release Candidate:

   • The workflow creates a GitHub release candidate (e.g., app_name/0.0.0-rc.0).
   • A Docker image for the release candidate is built, pushed to the registry, and deployed to the staging environment (specific to the app).

5. Draft Release & Image Promotion:

   • The same workflow creates a draft release with the final version tag (e.g., app_name/0.0.0). If a draft with the same tag already exists, it is updated.
   • Once the draft release is published by a developer, the Docker image from the staging environment is promoted to the production environment without re-building, ensuring consistency and saving resources.

6. Production Deployment:

   • A separate workflow is triggered upon publishing the draft release, promoting the staging Docker image to the production environment.

Tasks

• [ ] Update CI pipeline to automate tests and validations on feature branches.
• [ ] Implement semantic-release for version management and changelog generation.
• [ ] Automate the process of committing and pushing updated package.json and changelog to the repository.
• [ ] Configure the creation and deployment of release candidates to staging.
• [ ] Set up the creation and management of draft releases.
• [ ] Automate the promotion of Docker images from staging to production without re-building.

Benefits

This approach will:

• Automate the deployment process while preserving control over when releases are published.
• Utilize semantic-release for consistent and automated versioning and changelog generation.
• Ensure that all versioning and changelog updates are committed and pushed to the repository automatically.
• Improve efficiency by promoting Docker images from staging to production without the need for a re-build.
• Reduce manual intervention and errors during the deployment process.

Additional Considerations

• Ensure the workflow is compatible with our monorepo structure.
• Validate the process thoroughly on a smaller app before applying it across the entire repository.
• Document the new workflow so that all team members understand the process.

[Epic-SX]

@ygrishajev I am interested in this issue. I want to contribute your team. Is it possible to complete this issue?

[ygrishajev]

@JamesRyo Hey! Thanks for your interest! I think we could use some help here for sure. I'd like to approach this one step by step. Let me think what we could take off with here. I'll post later

[Epic-SX]

Thank you. I am waiting for your post.

[Epic-SX]

Hello I can handle this issue. Could you give me opportunity to handle this issue?

[pcfreak30]

@ygrishajev Just dropping a random feedback, but from my experience, I would use changesets over semantic-release. Unless there is a very good reason you want to force every commit to potentially run a versioning task.

I have used both now and definitely prefer changesets even if the repo seems to not get as much activity.

[ygrishajev]

Hey @pcfreak30 ! Thanks for sharing! Honestly I've never used changesets and haven't even considered it... unfortunately I've already implemented chnagelogs/releases part and it's functional. I would have investigated that option for sure if I came across it before ;)

I'll check it out for the future though for sure!

[pcfreak30]

Hey @pcfreak30 ! Thanks for sharing! Honestly I've never used changesets and haven't even considered it... unfortunately I've already implemented chnagelogs/releases part and it's functional. I would have investigated that option for sure if I came across it before ;)

I'll check it out for the future though for sure!

tbh from my exp, I would go and evaluate if you actually want what I described and do the work to switch if not. I have used semantic and eventually moved and I prefer changesets.