search

mmbell / samurai

Spline Analysis at Mesoscale Utilizing Radar and Aircraft Instrumentation
GNU General Public License v3.0
13 stars 17 forks source link

Talk to Nick Cote about running CI/CD Pipeline on On Prem cloud #104

Closed cenamiller closed 1 month ago

cenamiller commented 2 months ago

Will update with Nick's response, asked for clarification about running github action workflows from public repo, as well as how we would get access to test

cenamiller commented 2 months ago

Response from Nick Cote

Thanks for reaching out. The on-prem cloud resources would be an option. We would host a custom container that connects to GitHub as the self-hosted runner. That container image would contain the software required to run your workflows and it can connect to the Nvidia GPUs we have on our hardware. We add that container image to the on-prem cloud resources through a Helm chart, a collection of YAML files, hosted out of the same registry the runners are for.

As an example I have a Helm chart with specific values for my repo located here : https://github.com/NicholasCote/github-runner/blob/main/gh-runner-helm/values.yaml I connect that to the continuous deployment tool used on the on-prem cloud hardware and it launches the container I specified in the chart with the resources and configuration specified. The container image file I used here is specific to my workflow, building container images and jupyter-book documentation, but contains the self-hosted runner registration script and everything. If you're interested in an example of what that looks like, see here https://github.com/NicholasCote/github-runner/blob/main/Dockerfile. If I want to use a different image, increase resources, etc. I would update the values.yaml file with those changes and it will be sync'd automatically when the changes are pushed up in to GitHub.

I also have branch protection rules in place for certain repositories to enforce a workflow I'd like users to take. GitHub also has Rulesets that can be used to lock down what people can do in the repo and on certain branches. I haven't spent a ton of time looking into the Rulesets yet but am interested because I haven't found a great way to prevent people from opening a PR against main from a fork for example.

Please let me know if this sounds interesting and if you have any questions. I don't mind sitting down for awhile together as a group and getting some initial things in place if that would be helpful to you all as well. Just let me know. Thanks,

cenamiller commented 1 month ago

https://docs.google.com/document/d/1Wmlyno4j7_tZXlsbPmCV6Z885HgwPILgWGgHe-nCJ1A/edit#heading=h.sdx7uwljjxue

We're going to try using them with NCAR/samurai-dev