Question/Discussion long term goal to avoid/reduce duplicate code

sustainable-computing-io / kepler-metal-ci

Testing different CI and Github Action pipelines and publish test results

https://sustainable-computing-io.github.io/kepler-metal-ci/

Apache License 2.0

0 stars 4 forks source link

Question/Discussion long term goal to avoid/reduce duplicate code #45

Open SamYuan1990 opened 2 weeks ago

SamYuan1990 commented 2 weeks ago

We have different repos for CI, hence by this issue I want to reduce duplication. My proposal is: kepler-action repo on behavior of set up k8s cluster for our community CI. hence:

to integration with slef hosted runner (either aws ec2 or Equinix) kepler action should set k8s cluster form zero.
to integration with an existing k8s cluster(either Equinix or aws eks?) kepler action should install prometheus and others for this cluster.

kepler test script, which should starts from a k8s cluster. but we have to make it repo specific as:

helm chat/operator, starts from k8s cluster and install kepler by helm/operator.
kepler, integration test.
kepler model server, to do model training and etc.

Considering that we may have a remote k8s cluster and access it by kubeconfig, hence for testing script, I prefer to use github action and tekton, to avoid environment specific or security issue(for example ssh key leak).

SamYuan1990 commented 1 week ago

security issue for security expose risk https://github.com/sustainable-computing-io/kepler-metal-ci/blob/main/.github/workflows/equinix_k8s_flow.yml#L58 https://github.com/sustainable-computing-io/kepler-metal-ci/blob/main/.github/workflows/equinix_k8s_flow.yml#L64 https://github.com/sustainable-computing-io/kepler-metal-ci/blob/main/.github/workflows/equinix_k8s_flow.yml#L96

need considering for reuse https://github.com/sustainable-computing-io/kepler-metal-ci/blob/main/.github/workflows/equinix_k8s_flow.yml#L75-L90 https://github.com/sustainable-computing-io/kepler-metal-ci/blob/main/.github/workflows/equinix_k8s_flow.yml#L199

considering for community, to support other deployment approaches https://github.com/sustainable-computing-io/kepler-metal-ci/blob/main/.github/workflows/equinix_k8s_flow.yml#L153

seems not good for structure code as dependent mgr https://github.com/sustainable-computing-io/kepler-metal-ci/blob/main/.github/workflows/equinix_k8s_flow.yml#L105 https://github.com/sustainable-computing-io/kepler-metal-ci/blob/main/.github/workflows/equinix_k8s_flow.yml#L132 https://github.com/sustainable-computing-io/kepler-metal-ci/blob/main/.github/workflows/equinix_k8s_flow.yml#L166 https://github.com/sustainable-computing-io/kepler-metal-ci/blob/main/.github/workflows/equinix_k8s_flow.yml#L194

bot usage https://github.com/sustainable-computing-io/kepler-metal-ci/blob/main/.github/workflows/equinix_metal_flow.yml#L88

delete resource in safe https://github.com/sustainable-computing-io/kepler-metal-ci/blob/main/.github/workflows/equinix_metal_flow.yml#L100

use kubevirt instead https://github.com/sustainable-computing-io/kepler-metal-ci/blob/main/.github/workflows/equinix_metal_flow.yml#L63

@rootfs , once you back from vacation we should discuss offline for enhancement parts above.