search

golemcloud / golem

Golem is an open source durable computing platform that makes it easy to build and deploy highly reliable distributed systems.
https://learn.golem.cloud/
Apache License 2.0
515 stars 58 forks source link

Speed up CI by caching compiled WASMs in Github Actions cache #952

Open vigoo opened 1 month ago

vigoo commented 1 month ago

We could probably significantly speed up the CI test tasks by caching the test component's compiled native images in a Github Action cache.

This is not straightforward to do, as we cannot use Golem's built-in compilation cache - it is per component, and many tests are intentionally creating new components even though they use the same source WASM file. So this should be some kind of wasm hash-based file system cache, enabled only for tests, and explicitly saved/restored on CI.

Note: The wasmtime CLI supports something like this (https://docs.wasmtime.dev/cli-cache.html), so maybe there is something available in wasmtime that makes implementing this easy - but even if not, it can be done the same way as the live golem compilation cache works.

vigoo commented 1 month ago

The cache key for this must be separate and only calculated from the test component WASMs, which are changing relatively infrequently.

ologbonowiwi commented 1 month ago

The aim is for multiple pipelines to use the same built component, or from within a given run, should we use the component N times without recompiling it?

Mb about the possibly naive question, but I am currently trying to gain experience around this specific project/codebase