vercel / turborepo

Build system optimized for JavaScript and TypeScript, written in Rust
https://turbo.build/repo/docs
MIT License
26.21k stars 1.81k forks source link

Add guide for using local caching with github actions #1864

Closed jylin closed 11 months ago

jylin commented 2 years ago

Describe the feature you'd like to request

There's a nice guide here on setting up turborepo with github actions using remote caching: https://turborepo.org/docs/ci/github-actions.

It would be nice to have a similar guide but using local caching as github supports uploading local artifacts to be used in subsequent builds (for example using https://github.com/actions/cache). One reason this is nice is because it would support not having to use another service provider.

Describe the solution you'd like

Have some guidance for setting up using turborepo with a local cache while using github actions.

Describe alternatives you've considered

-

la55u commented 2 years ago

+1 the same for Gitlab CI

promer94 commented 2 years ago

From turborepo discord by felixmosh

The problem with using action/cache is the fact that you most specify a restore key which means it will keep only one cache per build. When you are using turborepo, this means that you have a mono-repo, which probably will produce multiple artifacts. Think about a monorepo with A,B,C,D artifacts. The first build will miss all, create a cache to all (the restore key will be somehow generated...) in one cache. Then you changed only A, this build will miss, since, the previous cache contains A, and it should be missed. = no gain. With my action, each turbo artifact is saved separately as GH artifact, In the above scenario, artifacts of B,C,D will restored from cache, and turbo will build only A = HUGE gain

awxalbert commented 2 years ago

The same goes for Gitlab CI. I accidently found that CI's cache works really well with turbo cache, doesn't even need to do extra work other than specify a cache path. It really needs to be mentioned in the doc and could save others' time instead of figuring out how to make remote cache work.

austinwoon commented 1 year ago

From turborepo discord by felixmosh

The problem with using action/cache is the fact that you most specify a restore key which means it will keep only one cache per build. When you are using turborepo, this means that you have a mono-repo, which probably will produce multiple artifacts. Think about a monorepo with A,B,C,D artifacts. The first build will miss all, create a cache to all (the restore key will be somehow generated...) in one cache. Then you changed only A, this build will miss, since, the previous cache contains A, and it should be missed. = no gain. With my action, each turbo artifact is saved separately as GH artifact, In the above scenario, artifacts of B,C,D will restored from cache, and turbo will build only A = HUGE gain

I didnt understand this comment as I was able to set up my turbo cache with the actions-cache action, and have only affected changes be rebuilt. Perhaps he was using an incorrect restore-key?

I have some sample github runs here to showcase expected caching behavior here: https://github.com/austinwoon/turbo-repo-cache-with-github-actions/actions

PR guide here

billyjacoby commented 1 year ago

The same goes for Gitlab CI. I accidently found that CI's cache works really well with turbo cache, doesn't even need to do extra work other than specify a cache path. It really needs to be mentioned in the doc and could save others' time instead of figuring out how to make remote cache work.

I'm having issues getting caches to properly work with Gitlab CI. Judginb by this comment should I also add ./node_modules/.cache/turbo/ to my cache files?

hmnd commented 8 months ago

@awxalbert how did you have your GitLab CI cache configured if you don't mind me asking?