[turborepo] non-monorepo project turbo remote cache on docker build always missed

[zigang93]

What version of Turborepo are you using?

1.6.3

What package manager are you using / does the bug impact?

pnpm

What operating system are you using?

Mac

Describe the Bug

reproduce repo: https://github.com/zigang93/next-with-docker-turbo

run pnpm i before start

1. enable turbo remote cache with vercel npx turbo login and npx turbo link
2. run docker build command with full plain log: PROGRESS_NO_TRUNC=1 docker build --no-cache --progress plain -t next-js .
3. re-run agian, the cache missed.. can notice that hash is always different when each PROGRESS_NO_TRUNC=1 docker build --progress plain -t next-js . command run..

but, I can get remote cache from pnpm build:turbo

Expected Behavior

should be able to remote cache on docker build

To Reproduce

reproduce repo: https://github.com/zigang93/next-with-docker-turbo

run pnpm i before start

1. enable turbo remote cache with vercel npx turbo login and npx turbo link
2. run docker build command with full plain log: PROGRESS_NO_TRUNC=1 docker build --no-cache --progress plain -t next-js .
3. re-run agian, the cache missed.. can notice that hash is always different when each PROGRESS_NO_TRUNC=1 docker build --progress plain -t next-js . command run..

Reproduction Repo

https://github.com/zigang93/next-with-docker-turbo

[anthonyshew]

I believe the issue here is that your Docker container itself isn't logged in. Instead, your host machine is - but your host machine isn't the one doing the building!

Since turbo is executing inside of a container (that isn't logged in), it's expected that the build will miss cache because, from the perspective of the container, there is no cache.

Two options from the documentation:

• Using environment variables, set TURBO_TEAM and TURBO_TOKEN for your container and Docker build stage
• Turbo CLI's --token and --team flags

[zigang93]

I believe the issue here is that your Docker container itself isn't logged in. Instead, your host machine is - but your host machine isn't the one doing the building!

Since turbo is executing inside of a container (that isn't logged in), it's expected that the build will miss cache because, from the perspective of the container, there is no cache.

Two options from the documentation:

• Using environment variables, set TURBO_TEAM and TURBO_TOKEN for your container and Docker build stage
• Turbo CLI's --token and --team flags

Hi, I had tried with your both suggestion, still not working as expected. TURBO_TOKEN is my vercel personal access token that manual generate from vercel settings, I don't see any token in my .turbo config.json so I had to manual generate it.. not sure it is correct way to do? TURBO_TEAM is refer to my .turbo config.json teamid key

I also updated my docker build command with no-cache to avoid any docker build cache

[anthonyshew]

Sounds like you grabbed your TURBO_TOKEN correctly. Your team ID should be coming from here: (If you are on your personal account, the field will say "Your ID")

If that doesn't work, are you seeing errors or just still missing cache?

[zigang93]

Sounds like you grabbed your TURBOTOKEN correctly. Your team ID should be coming from here: (If you are on your personal account, the field will say "Your ID")_

If that doesn't work, are you seeing errors or just still missing cache?

yea, it is same as from my .turbo/config.json

I just found out that if I am using --token flag into my pnpm build:turbo, remote caching will not working properly in my non docker environment..

[zigang93]

I created a token from vercel setting with name "Github Action Token".. as you can see the last active time is never.. which mean TOKEN is not used by remote cache..

I believe that manual generate token is not valid for remote cache... I also not sure why turbo token after login is not show at .turbo/config.json

additional detail:

my pnpm build:turbo script:

I use --remote-only to make sure is only run on remote cache and not local cache

[zigang93]

my current debug step:

package.json with "build:turbo: "turbo build --remote-only --team=\"xxxxx\" --token=\"xxxxx\""

1. run pnpm build:turbo
2. delete .next folder
3. re-run pnpm build:turbo

result: cache missed

---

package.json with "build:turbo: "turbo build --remote-only"

1. run pnpm build:turbo
2. delete .next folder
3. re-run pnpm build:turbo

result: cache hit

[zigang93]

this is how I am creating my token:

[anthonyshew]

One thing to note: I was wrong about TURBO_TEAM needing to be an ID. Instead, it needs to be your team's slug.

However, even with that change (using team slug), I'm still seeing cache misses. I'm definitely not seeing the behavior I'm expecting.

@nathanhammond @tknickman, any ideas?

[zigang93]

@anthonyshew any update??

[matsko]

I'm also experiencing this issue. Turbo in Docker always has a cache miss.

[diXrom]

any news?

[anthonyshew]

Hey, folks! Using latest, it looks like this bug(?) is now fixed for me:

First run:

Second run:

Will reiterate on the key concepts to know when working on Docker + Turbo + non-monorepo and correct some of the mistakes that I made earlier in this thread (sorry, wasn't on core team yet and thought I was being helpful). 😬

• Your Docker builder stage must have access to the Remote Cache. This can be done using environment variables. Edits to the reproduction that I made to get to success:
• The TURBO_TEAM variable is a slug, not an ID. If you're using Vercel, you can find this in your dashboard.
• Make sure you're on the latest version of turbo.

I apologize for this taking as long as it did to get closed out and properly explained. Hope this helps!