deislabs / containerd-wasm-shims

containerd shims for running WebAssembly workloads in Kubernetes
Apache License 2.0
311 stars 48 forks source link

A workspace for all the shims? #126

Closed Mossaka closed 1 year ago

Mossaka commented 1 year ago

As we are adding more shims (e.g. #122 ) to the repo, the CI pipeline becomes longer to finish (e.g. also reported in #91). Part of the reasons is that all the shims in this repo defines their own workspaces, so each shim build takes a trunk of time in the CI. The disk space pressure issue described https://github.com/deislabs/containerd-wasm-shims/issues/111#issuecomment-1677141535 is also related to this problem.

I am thinking about

  1. Create a workspace in root directory of the project for all the shims. This will, in theory, reduces the build time. However, the downside of it is that it will create a single shared Cargo.lock file. If I remember correctly, the slight shim and spin shim share different versions of wasmtime. Could we isolate this dependency to achieve this? I don't know.

  2. Parallelize the pipeline to build shims and upload artifacts as described https://github.com/deislabs/containerd-wasm-shims/pull/121#issuecomment-1681040748. Then create a second job to pull the artifacts for the integration tests. This is do-able and I might spend this afternoon on working towards it.

In the meantime, I will do a local test of the lunatic shim, and if everything works, I am fine merging it in. After we fix the infrastructure of running tests on the GH actions, I will do a release all together.

FYI @0xE282B0 @jprendes

Mossaka commented 1 year ago

129 closes this