[CHECKLIST] Automate The Relayer Deployment

[shekohex]

Overview

We aim to achieve a ~%100 deployment automation here, from the PR gets merged in the develop branch to the creating a new release and finally deployed on the server(s).

The Automation Flow

Development deployment flow (develop branch)

1. Opening a PR to fix a bug in the develop branch.
2. CI Starts to check the code that it only compiles on Linux targets (no lints, no integration tests, ..etc).
3. Once reviewed, it gets merged automatically , that's it.
4. Creating a new release is automatic, by just creating a new tag. (should take less than 30s to do so)
5. CI notices a new release, it builds a new docker image and publish it to that release tag.
6. After that, the same CI, zips the current deployment config files and adds it to that release.
7. Once Above CI Action finishes, another one starts up to SSH into the Relayer development* boxes to deploy the new changes by running a script we added on the server

Target time for all CI steps from opening a PR: ~10min. Target time for all CI steps from creating a new release tag: ~20min.

Development Relayers are {Relayer4, Relayer5}

Production deployment flow (main branch)

1. We open a PR to merge develop into main branch.
2. CI Starts to Run (lints, unit-tests, integration-tests, ..etc)
3. If any of the checks failed, we open a clean up PR(s) to address these issues and merge it into develop
4. Once the CI is green on main we double-check everything and then we merge it (manually)
5. Create a new stable tag (manually)
6. The same CI from above notices a new stable release got published, it builds a new docker image and publishes it to that release tag.
7. After that, it publishes the current deployment config files to that release.
8. Once done, the other CI notices the new stable release and SSH into the Relayer production* boxes to deploy the new stable relayer by running a script we added on the server.

Production Relayers are {Relayer1, Relayer2, Relayer3}

Target time for all CI steps from opening a PR: ~1hour. Target time for all CI steps from creating a new release tag: ~20min.

Checklist Tasks

• [ ] Update the Github Actions files to be in the following structure
  • lints.yml For Clippy & Rust format
  • build.yml For building the relayer on (Linux, macOS)
  • tests.yml For Running the relayer tests on Linux.
  • image.yml For building and publishing the Docker image + config files
  • publish-rust-docs.yml For publishing the docs.
  • deploy.yml for running the deployment scripts.
  • [ ] Update the Lints to only Run when open a PR that targets the main branch.
  • [ ] Update the Build Step to only check the build on Linux when we open a PR that targets the develop branch.
  • [ ] Update the Docker image publishing to publish by the tag and bundle the config files with the release
  • [ ] Make sure that tests are run only when we open a PR that targets the main branch.
  • [ ] Add the Deployment Actions that will use SSH and run the deploy.sh script on the relayer boxes.
  • [ ] Make use of Ansible scripts when possible.
  • [ ] Make use of https://github.com/dawidd6/action-ansible-playbook github action for deployment
  • [ ] make use of https://github.com/nektos/act for testing all the workflows locally and that it works.

[Nutomic]

I understand that the purpose here is to decrease deployment time by skipping lints and tests for dev builds. However I think thats a very bad idea, because there will inevitably be lint failures or test failures, and we wont catch them during PR review. So instead they need to be fixed when merging to main branch, and by that time it will cost much more time because another person is doing the fixes, and context from the original PR is missing. So there might be some speedup in getting new changes deployed, but at the cost of more effort to fix things later.

I also dont see the need to create release tags for development deployments. It would be enough to create a new CI action for pushes/merges to develop branch, which automatically builds and publishes it. This can use a hardcoded Docker tag like :dev or similar, and include a version name from git describe like v0.4.0-97-g2568a72 (latest tag + commit hash). If we run tests and other checks when merging PRs, then this deployment action doesnt need to run them, and only needs to run the build.

[shekohex]

Closed in favor of #482