As we are adding more shims (e.g. #122 ) to the repo, the CI pipeline becomes longer to finish (e.g. also reported in #91). Part of the reasons is that all the shims in this repo defines their own workspaces, so each shim build takes a trunk of time in the CI. The disk space pressure issue described https://github.com/deislabs/containerd-wasm-shims/issues/111#issuecomment-1677141535 is also related to this problem.
I am thinking about
Create a workspace in root directory of the project for all the shims. This will, in theory, reduces the build time. However, the downside of it is that it will create a single shared Cargo.lock file. If I remember correctly, the slight shim and spin shim share different versions of wasmtime. Could we isolate this dependency to achieve this? I don't know.
In the meantime, I will do a local test of the lunatic shim, and if everything works, I am fine merging it in. After we fix the infrastructure of running tests on the GH actions, I will do a release all together.
Mossaka
commented
1 year ago
As we are adding more shims (e.g. #122 ) to the repo, the CI pipeline becomes longer to finish (e.g. also reported in #91). Part of the reasons is that all the shims in this repo defines their own workspaces, so each shim build takes a trunk of time in the CI. The disk space pressure issue described https://github.com/deislabs/containerd-wasm-shims/issues/111#issuecomment-1677141535 is also related to this problem.
I am thinking about
Create a workspace in root directory of the project for all the shims. This will, in theory, reduces the build time. However, the downside of it is that it will create a single shared
Cargo.lockfile. If I remember correctly, the slight shim and spin shim share different versions ofwasmtime. Could we isolate this dependency to achieve this? I don't know.Parallelize the pipeline to build shims and upload artifacts as described https://github.com/deislabs/containerd-wasm-shims/pull/121#issuecomment-1681040748. Then create a second job to pull the artifacts for the integration tests. This is do-able and I might spend this afternoon on working towards it.
In the meantime, I will do a local test of the lunatic shim, and if everything works, I am fine merging it in. After we fix the infrastructure of running tests on the GH actions, I will do a release all together.
FYI @0xE282B0 @jprendes