search

drone-runners / drone-runner-aws

Drone runner that executes pipelines on single-use virtual machines
Other
5 stars 6 forks source link

feat: [CDE-119]: Add task handling to spawn and cleanup VM for CDE/gitspaces on bare metal #476

Closed vikyathharekal closed 3 months ago

vikyathharekal commented 3 months ago
  1. Extend DLITE_CI_VM_INITIALIZE_TASK and DLITE_CI_VM_CLEANUP_TASK tasks to provision and destroy VM for CDE in bare metal machines
  2. CDE manager will submit these tasks to manager. The task poller here will obtain these tasks from manager and start handling them.
  3. Additional parameters as compared to the CI init task would be the secret used in agent to communicate to cde-manager and gcloud access token to download the agent binary itself.
  4. Added init script for gitspaces which downloads and starts gitspaces agent in addition to what is already being done in linux script (which includes downloading and starting lite engine)
  5. We are asking nomad to give us additional random free ports for gitspace agent and ssh/web. This information is passed in the delegate task params and handled dynamically. These are used in the VM port mapping to expose it outside (8083 -> gitspaces agent, 8088 -> ssh, 8089 -> web). These ports are returned in the response as well. We would eventually consume it in CDE manager to connect to the agent and ssh/web port will be used to remotely access dev container

Overall idea is to leverage the existing code as much as possible and also make sure we don't disturb the existing CI code much so as to not cause any regressions

Commit Checklist

Thank you for creating a pull request! To help us review / merge this can you make sure that your PR adheres as much as possible to the following.

The Basics

If you are adding new functionality, please provide evidence of the new functionality.

dhruv-harness commented 3 months ago

We are asking nomad to give us 2 additional random free ports for gitspace agent and ssh. These are used in the VM port mapping to expose it outside (8083 -> gitspaces agent, 8088 -> ssh). These 2 ports are returned in the response as well. We would eventually consume it in CDE manager to connect to the agent and ssh port will be used to remotely access dev container

@vikyathharekal The number of ports required will be 3. 8083 is for the agent. 8088 for ssh and 8089 for VS Code Web server. Can we change the number of ports required on runtime?