search

vercel / turborepo

Build system optimized for JavaScript and TypeScript, written in Rust
https://turbo.build/repo/docs
MIT License
26.21k stars 1.81k forks source link

[turborepo] Failure to remote cache – ‘timed out waiting for cookie’ #2790

Closed steveluscher closed 8 months ago

steveluscher commented 1 year ago

What version of Turborepo are you using?

1.6.3

What package manager are you using / does the bug impact?

pnpm

What operating system are you using?

Linux

Describe the Bug

Outputs don't get saved in my team's remote cache.

Expected Behavior

  1. Run turbo a b c
  2. Delete local cache rm -rf node_modules/.cache/turbo
  3. Run turbo a b c again

Expected remote artifacts to be downloaded and full turbo mode to be achieved.

To Reproduce

  1. Run turbo -vvv --remote-only a b c

Observe ‘timed out waiting for cookie’ failure to cache in logs:

> @solana/web3.js-experimental@0.0.0-development package /home/sol/src/solana/web3.js-experimental
> turbo run -vvv --remote-only build build:typedefs lint verifyTreeShakability:browser verifyTreeShakability:native verifyTreeShakability:node

2022-11-21T19:20:04.707Z [DEBUG] turbo.TurbodClient: starting turbod binary /home/sol/src/solana/web3.js-experimental/node_modules/.pnpm/turbo@1.6.3/node_modules/turbo/bin/turbo
2022-11-21T19:20:04.708Z [DEBUG] turbo: running in daemon mode
2022-11-21T19:20:04.708Z [DEBUG] turbo: global hash env vars: vars=["VERCEL_ANALYTICS_ID"]
2022-11-21T19:20:04.710Z [DEBUG] turbo: global hash: value=1a498b24ba4c453f
2022-11-21T19:20:04.710Z [DEBUG] turbo: local cache folder: path=""
• Running build, build:typedefs, lint, verifyTreeShakability:browser, verifyTreeShakability:native, verifyTreeShakability:node
• Remote caching enabled
2022-11-21T19:20:04.717Z [DEBUG] turbo.: start
2022-11-21T19:20:04.717Z [DEBUG] turbo.: start
2022-11-21T19:20:04.718Z [DEBUG] turbo: task hash env vars for //:build: vars=[]
2022-11-21T19:20:04.718Z [DEBUG] turbo: task hash: value=c8a353a412faadd6
2022-11-21T19:20:04.718Z [DEBUG] turbo: task hash env vars for //:lint: vars=[]
2022-11-21T19:20:04.718Z [DEBUG] turbo: task hash: value=e1eac274878ece20
2022-11-21T19:20:05.219Z [WARN]  turbo.: Failed to check if we can skip restoring outputs for //#lint: rpc error: code = Unknown desc = timed out waiting for cookie. Proceeding to check cache
lint: Failed to check if we can skip restoring outputs for //#lint: rpc error: code = Unknown desc = timed out waiting for cookie. Proceeding to check cache
2022-11-21T19:20:05.219Z [DEBUG] turbo: performing request: method=GET url=https://vercel.com/api/v8/artifacts/e1eac274878ece20?teamId={TEAMID}
2022-11-21T19:20:05.219Z [WARN]  turbo.: Failed to check if we can skip restoring outputs for //#build: rpc error: code = Unknown desc = timed out waiting for cookie. Proceeding to check cache
build: Failed to check if we can skip restoring outputs for //#build: rpc error: code = Unknown desc = timed out waiting for cookie. Proceeding to check cache
2022-11-21T19:20:05.219Z [DEBUG] turbo: performing request: method=GET url=https://vercel.com/api/v8/artifacts/c8a353a412faadd6?teamId={TEAMID}
lint: cache miss, executing e1eac274878ece20
2022-11-21T19:20:05.823Z [DEBUG] turbo: performing request: method=POST url=https://vercel.com/api/v8/artifacts/events?teamId={TEAMID}
lint: 
lint: > @solana/web3.js-experimental@0.0.0-development lint /home/sol/src/solana/web3.js-experimental
lint: > eslint 'src/**'
lint: 
2022-11-21T19:20:06.919Z [DEBUG] turbo.: caching output: outputs="{[.turbo/turbo-lint.log] []}"
build: cache miss, executing c8a353a412faadd6
2022-11-21T19:20:07.251Z [DEBUG] turbo: performing request: method=POST url=https://vercel.com/api/v8/artifacts/events?teamId={TEAMID}
2022-11-21T19:20:07.420Z [WARN]  turbo.: Failed to mark outputs as cached for //#lint: rpc error: code = Unknown desc = timed out waiting for cookie
lint: Failed to mark outputs as cached for //#lint: rpc error: code = Unknown desc = timed out waiting for cookie
2022-11-21T19:20:07.420Z [DEBUG] turbo.: done: status=complete duration=2.201266026s
build: 
build: > @solana/web3.js-experimental@0.0.0-development build /home/sol/src/solana/web3.js-experimental
build: > tsup
build: 
build: [NODE] CLI Building entry: ./src/index.ts
build: [NODE] CLI Using tsconfig: tsconfig.json
build: [NODE] CLI tsup v6.5.0
build: [NODE] CLI Using tsup config: /home/sol/src/solana/web3.js-experimental/tsup.config.ts
build: [NODE] CLI Building entry: ./src/index.ts
build: [NODE] CLI Using tsconfig: tsconfig.json
build: [NODE] CLI tsup v6.5.0
build: [NODE] CLI Using tsup config: /home/sol/src/solana/web3.js-experimental/tsup.config.ts
build: [BROWSER] CLI Building entry: ./src/index.ts
build: [BROWSER] CLI Using tsconfig: tsconfig.json
build: [BROWSER] CLI tsup v6.5.0
build: [BROWSER] CLI Using tsup config: /home/sol/src/solana/web3.js-experimental/tsup.config.ts
build: [BROWSER] CLI Building entry: ./src/index.ts
build: [BROWSER] CLI Using tsconfig: tsconfig.json
build: [BROWSER] CLI tsup v6.5.0
build: [BROWSER] CLI Using tsup config: /home/sol/src/solana/web3.js-experimental/tsup.config.ts
build: [NATIVE] CLI Building entry: ./src/index.ts
build: [NATIVE] CLI Using tsconfig: tsconfig.json
build: [NATIVE] CLI tsup v6.5.0
build: [NATIVE] CLI Using tsup config: /home/sol/src/solana/web3.js-experimental/tsup.config.ts
build: [NATIVE] CLI Building entry: ./src/index.ts
build: [NATIVE] CLI Using tsconfig: tsconfig.json
build: [NATIVE] CLI tsup v6.5.0
build: [NATIVE] CLI Using tsup config: /home/sol/src/solana/web3.js-experimental/tsup.config.ts
build: [NODE] CLI Target: es2020
build: [NODE] CLI Target: es2020
build: [BROWSER] CLI Target: es2020
build: [BROWSER] CLI Target: es2020
build: [NATIVE] CLI Target: es2020
build: [NATIVE] CLI Target: es2020
build: [NODE] CLI Cleaning output folder
build: [NODE] CJS Build start
build: [NODE] ESM Build start
build: [NODE] CLI Cleaning output folder
build: [NODE] CJS Build start
build: [NODE] ESM Build start
build: [NATIVE] CLI Cleaning output folder
build: [NATIVE] ESM Build start
build: [BROWSER] CLI Cleaning output folder
build: [BROWSER] CJS Build start
build: [BROWSER] ESM Build start
build: [BROWSER] IIFE Build start
build: [BROWSER] CLI Cleaning output folder
build: [BROWSER] CJS Build start
build: [BROWSER] ESM Build start
build: [BROWSER] IIFE Build start
build: [NATIVE] CLI Cleaning output folder
build: [NATIVE] ESM Build start
build: [NODE] ESM dist/index.node.esm.js 351.00 B
build: [NODE] ESM ⚡️ Build success in 90ms
build: [BROWSER] IIFE dist/index.iife.production.js 196.00 B
build: [BROWSER] IIFE ⚡️ Build success in 85ms
build: [NODE] CJS dist/index.node.cjs.js 369.00 B
build: [NODE] CJS ⚡️ Build success in 90ms
build: [BROWSER] ESM dist/index.browser.esm.js 362.00 B
build: [BROWSER] ESM ⚡️ Build success in 85ms
build: [BROWSER] CJS dist/index.browser.cjs.js 380.00 B
build: [BROWSER] CJS ⚡️ Build success in 85ms
build: [NATIVE] ESM dist/index.native.esm.js 366.00 B
build: [NATIVE] ESM ⚡️ Build success in 89ms
build: [BROWSER] IIFE dist/index.iife.development.js     443.00 B
build: [BROWSER] IIFE dist/index.iife.development.js.map 948.00 B
build: [BROWSER] IIFE ⚡️ Build success in 93ms
build: [BROWSER] ESM dist/index.browser.esm.js     439.00 B
build: [BROWSER] ESM dist/index.browser.esm.js.map 1.18 KB
build: [BROWSER] ESM ⚡️ Build success in 94ms
build: [NATIVE] ESM dist/index.native.esm.js     442.00 B
build: [NATIVE] ESM dist/index.native.esm.js.map 1.18 KB
build: [NATIVE] ESM ⚡️ Build success in 95ms
build: [NODE] ESM dist/index.node.esm.js     425.00 B
build: [NODE] ESM dist/index.node.esm.js.map 1.17 KB
build: [NODE] ESM ⚡️ Build success in 99ms
build: [BROWSER] CJS dist/index.browser.cjs.js     457.00 B
build: [BROWSER] CJS dist/index.browser.cjs.js.map 1.18 KB
build: [BROWSER] CJS ⚡️ Build success in 94ms
build: [NODE] CJS dist/index.node.cjs.js     443.00 B
build: [NODE] CJS dist/index.node.cjs.js.map 1.17 KB
build: [NODE] CJS ⚡️ Build success in 99ms
2022-11-21T19:20:07.745Z [DEBUG] turbo.: caching output: outputs="{[.turbo/turbo-build.log dist/**] []}"
2022-11-21T19:20:08.247Z [WARN]  turbo.: Failed to mark outputs as cached for //#build: rpc error: code = Unknown desc = timed out waiting for cookie
build: Failed to mark outputs as cached for //#build: rpc error: code = Unknown desc = timed out waiting for cookie
2022-11-21T19:20:08.247Z [DEBUG] turbo.: done: status=complete duration=3.027893249s
2022-11-21T19:20:08.247Z [DEBUG] turbo.: start
2022-11-21T19:20:08.247Z [DEBUG] turbo.: start
2022-11-21T19:20:08.247Z [DEBUG] turbo.: start
2022-11-21T19:20:08.247Z [DEBUG] turbo: task hash env vars for //:verifyTreeShakability:native: vars=[]
2022-11-21T19:20:08.247Z [DEBUG] turbo.: start
2022-11-21T19:20:08.247Z [DEBUG] turbo: task hash env vars for //:verifyTreeShakability:browser: vars=[]
2022-11-21T19:20:08.247Z [DEBUG] turbo: task hash: value=6b56544a323649e4
2022-11-21T19:20:08.247Z [DEBUG] turbo: task hash env vars for //:build:typedefs: vars=[]
2022-11-21T19:20:08.247Z [DEBUG] turbo: task hash: value=a8b840bcfd3065a9
2022-11-21T19:20:08.247Z [DEBUG] turbo: task hash: value=247324d9a4b361ac
2022-11-21T19:20:08.247Z [DEBUG] turbo: task hash env vars for //:verifyTreeShakability:node: vars=[]
2022-11-21T19:20:08.247Z [DEBUG] turbo: task hash: value=e482892a4e66085b
2022-11-21T19:20:08.748Z [WARN]  turbo.: Failed to check if we can skip restoring outputs for //#verifyTreeShakability:node: rpc error: code = Unknown desc = timed out waiting for cookie. Proceeding to check cache
verifyTreeShakability:node: Failed to check if we can skip restoring outputs for //#verifyTreeShakability:node: rpc error: code = Unknown desc = timed out waiting for cookie. Proceeding to check cache
2022-11-21T19:20:08.748Z [DEBUG] turbo: performing request: method=GET url=https://vercel.com/api/v8/artifacts/e482892a4e66085b?teamId={TEAMID}
2022-11-21T19:20:08.748Z [WARN]  turbo.: Failed to check if we can skip restoring outputs for //#verifyTreeShakability:browser: rpc error: code = Unknown desc = timed out waiting for cookie. Proceeding to check cache
verifyTreeShakability:browser: Failed to check if we can skip restoring outputs for //#verifyTreeShakability:browser: rpc error: code = Unknown desc = timed out waiting for cookie. Proceeding to check cache
2022-11-21T19:20:08.748Z [DEBUG] turbo: performing request: method=GET url=https://vercel.com/api/v8/artifacts/6b56544a323649e4?teamId={TEAMID}
2022-11-21T19:20:08.748Z [WARN]  turbo.: Failed to check if we can skip restoring outputs for //#build:typedefs: rpc error: code = Unknown desc = timed out waiting for cookie. Proceeding to check cache
build:typedefs: Failed to check if we can skip restoring outputs for //#build:typedefs: rpc error: code = Unknown desc = timed out waiting for cookie. Proceeding to check cache
2022-11-21T19:20:08.748Z [DEBUG] turbo: performing request: method=GET url=https://vercel.com/api/v8/artifacts/a8b840bcfd3065a9?teamId={TEAMID}
2022-11-21T19:20:08.748Z [WARN]  turbo.: Failed to check if we can skip restoring outputs for //#verifyTreeShakability:native: rpc error: code = Unknown desc = timed out waiting for cookie. Proceeding to check cache
verifyTreeShakability:native: Failed to check if we can skip restoring outputs for //#verifyTreeShakability:native: rpc error: code = Unknown desc = timed out waiting for cookie. Proceeding to check cache
2022-11-21T19:20:08.748Z [DEBUG] turbo: performing request: method=GET url=https://vercel.com/api/v8/artifacts/247324d9a4b361ac?teamId={TEAMID}
verifyTreeShakability:native: cache miss, executing 247324d9a4b361ac
verifyTreeShakability:browser: cache miss, executing 6b56544a323649e4
build:typedefs: cache miss, executing a8b840bcfd3065a9
verifyTreeShakability:node: cache miss, executing e482892a4e66085b
2022-11-21T19:20:09.275Z [DEBUG] turbo: performing request: method=POST url=https://vercel.com/api/v8/artifacts/events?teamId={TEAMID}
verifyTreeShakability:browser: 
verifyTreeShakability:browser: > @solana/web3.js-experimental@0.0.0-development verifyTreeShakability:browser /home/sol/src/solana/web3.js-experimental
verifyTreeShakability:browser: > agadoo dist/index.browser.esm.js
verifyTreeShakability:browser: 
build:typedefs: 
build:typedefs: > @solana/web3.js-experimental@0.0.0-development build:typedefs /home/sol/src/solana/web3.js-experimental
build:typedefs: > tsc -p ./tsconfig.declarations.json
build:typedefs: 
verifyTreeShakability:node: 
verifyTreeShakability:node: > @solana/web3.js-experimental@0.0.0-development verifyTreeShakability:node /home/sol/src/solana/web3.js-experimental
verifyTreeShakability:node: > agadoo dist/index.native.esm.js
verifyTreeShakability:node: 
verifyTreeShakability:native: 
verifyTreeShakability:native: > @solana/web3.js-experimental@0.0.0-development verifyTreeShakability:native /home/sol/src/solana/web3.js-experimental
verifyTreeShakability:native: > agadoo dist/index.node.esm.js
verifyTreeShakability:native: 
verifyTreeShakability:browser: 
verifyTreeShakability:browser: 
verifyTreeShakability:browser: Success! dist/index.browser.esm.js is fully tree-shakeable
verifyTreeShakability:node: 
verifyTreeShakability:node: 
verifyTreeShakability:node: Success! dist/index.native.esm.js is fully tree-shakeable
verifyTreeShakability:native: 
verifyTreeShakability:native: 
verifyTreeShakability:native: Success! dist/index.node.esm.js is fully tree-shakeable
2022-11-21T19:20:09.553Z [DEBUG] turbo.: caching output: outputs="{[.turbo/turbo-verifyTreeShakability:browser.log] []}"
2022-11-21T19:20:09.560Z [DEBUG] turbo.: caching output: outputs="{[.turbo/turbo-verifyTreeShakability:node.log] []}"
2022-11-21T19:20:09.561Z [DEBUG] turbo.: caching output: outputs="{[.turbo/turbo-verifyTreeShakability:native.log] []}"
2022-11-21T19:20:10.054Z [WARN]  turbo.: Failed to mark outputs as cached for //#verifyTreeShakability:browser: rpc error: code = Unknown desc = timed out waiting for cookie
verifyTreeShakability:browser: Failed to mark outputs as cached for //#verifyTreeShakability:browser: rpc error: code = Unknown desc = timed out waiting for cookie
2022-11-21T19:20:10.054Z [DEBUG] turbo.: done: status=complete duration=1.306132708s
2022-11-21T19:20:10.061Z [WARN]  turbo.: Failed to mark outputs as cached for //#verifyTreeShakability:node: rpc error: code = Unknown desc = timed out waiting for cookie
verifyTreeShakability:node: Failed to mark outputs as cached for //#verifyTreeShakability:node: rpc error: code = Unknown desc = timed out waiting for cookie
2022-11-21T19:20:10.061Z [DEBUG] turbo.: done: status=complete duration=1.313211685s
2022-11-21T19:20:10.062Z [WARN]  turbo.: Failed to mark outputs as cached for //#verifyTreeShakability:native: rpc error: code = Unknown desc = timed out waiting for cookie
verifyTreeShakability:native: Failed to mark outputs as cached for //#verifyTreeShakability:native: rpc error: code = Unknown desc = timed out waiting for cookie
2022-11-21T19:20:10.062Z [DEBUG] turbo.: done: status=complete duration=1.314136719s
2022-11-21T19:20:10.880Z [DEBUG] turbo.: caching output: outputs="{[.turbo/turbo-build:typedefs.log dist/**] []}"
2022-11-21T19:20:11.381Z [WARN]  turbo.: Failed to mark outputs as cached for //#build:typedefs: rpc error: code = Unknown desc = timed out waiting for cookie
build:typedefs: Failed to mark outputs as cached for //#build:typedefs: rpc error: code = Unknown desc = timed out waiting for cookie
2022-11-21T19:20:11.381Z [DEBUG] turbo.: done: status=complete duration=2.632834966s

 Tasks:    6 successful, 6 total
Cached:    0 cached, 6 total
  Time:    6.674s

Reproduction Repo

https://github.com/solana-labs/solana

tknickman commented 1 year ago

Hey @steveluscher, thanks for reporting this. Can you tell me more about your OS (version, arch etc.)?

Also, please try passing --no-daemon to turbo build

steveluscher commented 1 year ago

Sure thing, @tknickman!

$ uname -a
Linux HOSTNAME 5.4.0-81-generic #91-Ubuntu SMP Thu Jul 15 19:09:17 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

Adding --no-daemon had no effect. :(

HQ92 commented 1 year ago

I also have this issue and am receiving the same error message along with:

"Failed to check if we can skip restoring outputs for cms#build: rpc error: code = Unknown desc = timed out waiting for cookie."

"Failed to mark outputs as cached for frontend#build: rpc error: code = Unknown desc = timed out waiting for cookie"

This is making one of the major benefits of TurboRepo invalid.

I also tried --no-daemon and it made no difference

coldino commented 1 year ago

I also see this issue - just sprung up out of nowhere in the middle of a session. It had been working for hours before.

I'm on Windows:

bash ❯ uname -a
MINGW64_NT-10.0-19045 devbox 3.3.6-341.x86_64 2022-09-05 20:28 UTC x86_64 Msys

bash ❯ pnpm exec turbo --version
1.7.0
bencun commented 1 year ago

Turbo seems to be working just fine but I get the same errors, or rather - messages.

Turbo 1.8.8 Windows 11

isimmons commented 1 year ago

Windows 10 here. Same messages but it works for running tests and tsc. Using --no-daemon made the error messages go away.

I have turbo installed globally and locally in my project

I first started running dev tasks for nextjs and vite/react and got a error about not being able to connect to the process. The message told me to run turbo daemon clean so I had to install turbo globally and then because it was open/locked by the local turbo, I had to manually delete the turbo .pid file.

Everything runs a lot faster with --no-daemon . So I guess it's all a timing issue with my OS?

anthonyshew commented 8 months ago

Hey, folks! I'm seeing that this issue has been inactive for quite some time. If you're still seeing this, please give us a comment here and we can re-open it. Thank you!