vercel / turborepo

Build system optimized for JavaScript and TypeScript, written in Rust
https://turbo.build/repo/docs
MIT License
26.45k stars 1.84k forks source link

Turborepo hanging in CI #8281

Open jakubmazanec opened 5 months ago

jakubmazanec commented 5 months ago

Verify canary release

Link to code that reproduces this issue

https://github.com/jakubmazanec/apps/pull/21

What package manager are you using / does the bug impact?

npm

What operating system are you using?

Windows

Which canary version will you have in your reproduction?

2.0.0-canary.2

Describe the Bug

After updating from 1.10.3 to 1.13.3, Turborepo freezes on a test task during GitHub workflow run (which uses Linux; https://github.com/jakubmazanec/apps/actions/runs/9357742231/job/25758162977). I was unable to figure out why, nor any further details, since it is working locally on my Windows laptop. Also, I couldn't actually check the canary version, since it has breaking changes and I cannot modify my config.

Expected Behavior

It works, like it is working on a version 1.10.3.

To Reproduce

In other issues I see mentions that Turborepo currently randomly freezes on Linux, so maybe it's related?

Additional context

No response

NicholasLYang commented 5 months ago

Hey, thanks for the issue. Could you try running in verbose mode (pass -vv to the turbo command) and send over the logs?

jakubmazanec commented 5 months ago

I'm sorry, I cannot modify the workflows again, but I have logs from last week ago; after that, I just used older version of Turborepo, after reading issues like #7382 or #5773, because the logs shows only that Turborepo waits for task, but doesn't say why. The task itself was codegen for EdgeDB, but then I replaced it with other commands (don't remember what these were), and sometimes it worked, sometimes didn't (but consistently for each specific variation of the task). I'm guessing the issue will be really weird and I'm not sure me debugging it will help. Anyway, here are the logs:

2024-05-25T18:03:01.1048899Z ##[group]Run npm run test -- --filter=dram -vv
2024-05-25T18:03:01.1049360Z npm run test -- --filter=dram -vv
2024-05-25T18:03:01.1107107Z shell: /usr/bin/bash -e {0}
2024-05-25T18:03:01.1107405Z env:
2024-05-25T18:03:01.1107927Z   NPM_CONFIG_USERCONFIG: /home/runner/work/_temp/.npmrc
2024-05-25T18:03:01.1108365Z   NODE_AUTH_TOKEN: XXXXX-XXXXX-XXXXX-XXXXX
2024-05-25T18:03:01.1108732Z   EDGEDB_INSTANCE: jakubmazanec/whisky
2024-05-25T18:03:01.1109069Z   EDGEDB_BRANCH: whisky
2024-05-25T18:03:01.1111924Z   EDGEDB_SECRET_KEY: ***
2024-05-25T18:03:01.1112209Z ##[endgroup]
2024-05-25T18:03:01.2430682Z 
2024-05-25T18:03:01.2430902Z > test
2024-05-25T18:03:01.2432109Z > turbo run test --no-daemon --filter=dram -vv
2024-05-25T18:03:01.2432713Z 
2024-05-25T18:03:01.2756917Z 2024-05-25T18:03:01.275+0000 [DEBUG] turborepo_lib::shim: Global turbo version: 1.13.3
2024-05-25T18:03:01.2759678Z 2024-05-25T18:03:01.275+0000 [DEBUG] turborepo_repository::package_json: loading package.json from /home/runner/work/apps/apps/package.json
2024-05-25T18:03:01.2796203Z 2024-05-25T18:03:01.279+0000 [DEBUG] turborepo_repository::package_json: loading package.json from /home/runner/work/apps/package.json
2024-05-25T18:03:01.2799253Z 2024-05-25T18:03:01.279+0000 [DEBUG] turborepo_repository::package_json: loading package.json from /home/runner/work/package.json
2024-05-25T18:03:01.2801681Z 2024-05-25T18:03:01.279+0000 [DEBUG] turborepo_repository::package_json: loading package.json from /home/runner/package.json
2024-05-25T18:03:01.2802944Z 2024-05-25T18:03:01.279+0000 [DEBUG] turborepo_repository::package_json: loading package.json from /home/package.json
2024-05-25T18:03:01.2804111Z 2024-05-25T18:03:01.279+0000 [DEBUG] turborepo_repository::package_json: loading package.json from /package.json
2024-05-25T18:03:01.2805199Z 2024-05-25T18:03:01.279+0000 [DEBUG] turborepo_lib::shim: Repository Root: /home/runner/work/apps/apps
2024-05-25T18:03:01.2806633Z 2024-05-25T18:03:01.279+0000 [DEBUG] turborepo_repository::package_json: loading package.json from /home/runner/work/apps/apps/node_modules/turbo-linux-64/package.json
2024-05-25T18:03:01.2808192Z 2024-05-25T18:03:01.279+0000 [DEBUG] turborepo_lib::shim: Local turbo path: /home/runner/work/apps/apps/node_modules/turbo-linux-64/bin/turbo
2024-05-25T18:03:01.2809313Z 2024-05-25T18:03:01.279+0000 [DEBUG] turborepo_lib::shim: Local turbo version: 1.13.3
2024-05-25T18:03:01.2810256Z 2024-05-25T18:03:01.279+0000 [DEBUG] turborepo_lib::shim: Currently running turbo is local turbo.
2024-05-25T18:03:01.2840455Z 2024-05-25T18:03:01.283+0000 [DEBUG] turborepo_telemetry::config: Telemetry config path: /home/runner/.config/turborepo/telemetry.json
2024-05-25T18:03:01.2843186Z 2024-05-25T18:03:01.284+0000 [DEBUG] turborepo_repository::package_json: loading package.json from /home/runner/work/apps/apps/package.json
2024-05-25T18:03:01.2859560Z 2024-05-25T18:03:01.285+0000 [DEBUG] turborepo_repository::package_json: loading package.json from /home/runner/work/apps/apps/package.json
2024-05-25T18:03:01.2861172Z 2024-05-25T18:03:01.285+0000 [DEBUG] turborepo_telemetry::config: Telemetry config path: /home/runner/.config/turborepo/telemetry.json
2024-05-25T18:03:01.2869649Z 2024-05-25T18:03:01.285+0000 [DEBUG] turborepo_lib::run::builder: skipping turbod since --no-daemon was passed
2024-05-25T18:03:01.2871901Z 2024-05-25T18:03:01.285+0000 [DEBUG] turborepo_telemetry: Starting telemetry event queue flush (num_events=10)
2024-05-25T18:03:01.2873798Z 2024-05-25T18:03:01.286+0000 [DEBUG] turborepo_telemetry: Done telemetry event queue flush
2024-05-25T18:03:01.2876213Z 2024-05-25T18:03:01.286+0000 [DEBUG] turborepo_repository::discovery: discovering packages using caching strategy
2024-05-25T18:03:01.2878384Z 2024-05-25T18:03:01.286+0000 [DEBUG] turborepo_repository::discovery: discovering packages using primary strategy
2024-05-25T18:03:01.2879766Z 2024-05-25T18:03:01.286+0000 [DEBUG] turborepo_repository::discovery: discovering packages using local strategy
2024-05-25T18:03:01.2948529Z 2024-05-25T18:03:01.294+0000 [DEBUG] turborepo_repository::package_json: loading package.json from /home/runner/work/apps/apps/apps/somewhere/package.json
2024-05-25T18:03:01.2950283Z 2024-05-25T18:03:01.294+0000 [DEBUG] turborepo_repository::package_json: loading package.json from /home/runner/work/apps/apps/apps/dram/package.json
2024-05-25T18:03:01.2951844Z 2024-05-25T18:03:01.294+0000 [DEBUG] turborepo_repository::discovery: discovering packages using caching strategy
2024-05-25T18:03:01.2953011Z 2024-05-25T18:03:01.294+0000 [DEBUG] turborepo_repository::discovery: discovering packages using caching strategy
2024-05-25T18:03:01.3022893Z 2024-05-25T18:03:01.301+0000 [DEBUG] log: No cached session for DnsName(DnsName(DnsName("telemetry.vercel.com")))
2024-05-25T18:03:01.3024938Z 2024-05-25T18:03:01.302+0000 [DEBUG] log: Not resuming any session
2024-05-25T18:03:01.3133347Z 2024-05-25T18:03:01.312+0000 [DEBUG] log: Using ciphersuite TLS13_AES_128_GCM_SHA256
2024-05-25T18:03:01.3135119Z 2024-05-25T18:03:01.312+0000 [DEBUG] log: Not resuming
2024-05-25T18:03:01.3136891Z 2024-05-25T18:03:01.312+0000 [DEBUG] log: TLS1.3 encrypted extensions: [Protocols([6832])]
2024-05-25T18:03:01.3138801Z 2024-05-25T18:03:01.312+0000 [DEBUG] log: ALPN protocol is Some(b"h2")
2024-05-25T18:03:01.3140223Z 2024-05-25T18:03:01.313+0000 [DEBUG] log: Ticket saved
2024-05-25T18:03:01.3332391Z 2024-05-25T18:03:01.333+0000 [DEBUG] turborepo_repository::discovery: discovering packages using caching strategy
2024-05-25T18:03:01.3335158Z 2024-05-25T18:03:01.333+0000 [DEBUG] turborepo_lib::run::task_access: TASK ACCESS TRACE: unable to restore config from cache
2024-05-25T18:03:01.3354334Z • Packages in scope: dram
2024-05-25T18:03:01.3355402Z • Running test in 1 packages
2024-05-25T18:03:01.3356210Z • Remote caching disabled
2024-05-25T18:03:01.3358310Z 2024-05-25T18:03:01.335+0000 [DEBUG] turborepo_telemetry::config: Telemetry config path: /home/runner/.config/turborepo/telemetry.json
2024-05-25T18:03:01.3361367Z 2024-05-25T18:03:01.335+0000 [DEBUG] turborepo_telemetry::config: Telemetry config path: /home/runner/.config/turborepo/telemetry.json
2024-05-25T18:03:01.3364068Z 2024-05-25T18:03:01.335+0000 [DEBUG] turborepo_telemetry::config: Telemetry config path: /home/runner/.config/turborepo/telemetry.json
2024-05-25T18:03:01.3366485Z 2024-05-25T18:03:01.335+0000 [DEBUG] turborepo_telemetry::config: Telemetry config path: /home/runner/.config/turborepo/telemetry.json
2024-05-25T18:03:01.3368918Z 2024-05-25T18:03:01.335+0000 [DEBUG] turborepo_telemetry::config: Telemetry config path: /home/runner/.config/turborepo/telemetry.json
2024-05-25T18:03:01.3371341Z 2024-05-25T18:03:01.336+0000 [DEBUG] turborepo_telemetry::config: Telemetry config path: /home/runner/.config/turborepo/telemetry.json
2024-05-25T18:03:01.3373771Z 2024-05-25T18:03:01.336+0000 [DEBUG] turborepo_telemetry::config: Telemetry config path: /home/runner/.config/turborepo/telemetry.json
2024-05-25T18:03:01.3376182Z 2024-05-25T18:03:01.336+0000 [DEBUG] turborepo_telemetry::config: Telemetry config path: /home/runner/.config/turborepo/telemetry.json
2024-05-25T18:03:01.3378652Z 2024-05-25T18:03:01.336+0000 [DEBUG] turborepo_telemetry::config: Telemetry config path: /home/runner/.config/turborepo/telemetry.json
2024-05-25T18:03:01.3381154Z 2024-05-25T18:03:01.336+0000 [DEBUG] turborepo_telemetry::config: Telemetry config path: /home/runner/.config/turborepo/telemetry.json
2024-05-25T18:03:01.3383450Z 2024-05-25T18:03:01.337+0000 [DEBUG] turborepo_telemetry: Starting telemetry event queue flush (num_events=10)
2024-05-25T18:03:01.3385352Z 2024-05-25T18:03:01.337+0000 [DEBUG] turborepo_telemetry: Done telemetry event queue flush
2024-05-25T18:03:01.3454596Z 2024-05-25T18:03:01.345+0000 [DEBUG] turborepo_lib::run::global_hash: global hash env vars ["EDGEDB_BRANCH", "EDGEDB_INSTANCE", "EDGEDB_SECRET_KEY"]
2024-05-25T18:03:01.3574143Z 2024-05-25T18:03:01.357+0000 [DEBUG] turborepo_lib::run::global_hash: external deps hash: 45b40dcd82af3717
2024-05-25T18:03:01.3576102Z 2024-05-25T18:03:01.357+0000 [DEBUG] turborepo_lib::run: running visitor
2024-05-25T18:03:01.3578119Z 2024-05-25T18:03:01.357+0000 [DEBUG] turborepo_telemetry::config: Telemetry config path: /home/runner/.config/turborepo/telemetry.json
2024-05-25T18:03:01.3579829Z 2024-05-25T18:03:01.357+0000 [DEBUG] turborepo_telemetry::config: Telemetry config path: /home/runner/.config/turborepo/telemetry.json
2024-05-25T18:03:01.3581201Z 2024-05-25T18:03:01.357+0000 [DEBUG] turborepo_telemetry::config: Telemetry config path: /home/runner/.config/turborepo/telemetry.json
2024-05-25T18:03:01.3582564Z 2024-05-25T18:03:01.357+0000 [DEBUG] turborepo_telemetry::config: Telemetry config path: /home/runner/.config/turborepo/telemetry.json
2024-05-25T18:03:01.3583717Z 2024-05-25T18:03:01.357+0000 [DEBUG] turborepo_lib::task_hash: auto detected framework for dram
2024-05-25T18:03:01.3584739Z 2024-05-25T18:03:01.357+0000 [DEBUG] turborepo_lib::task_hash: framework: vite, env_prefix: ["VITE_*"]
2024-05-25T18:03:01.3590338Z 2024-05-25T18:03:01.358+0000 [DEBUG] turborepo_lib::task_hash: task hash env vars for dram:codegen
2024-05-25T18:03:01.3590981Z  vars: []
2024-05-25T18:03:01.3591809Z 2024-05-25T18:03:01.358+0000 [DEBUG] turborepo_lib::task_graph::visitor: task dram#codegen hash is 09839c4ba504b8c4
2024-05-25T18:03:01.3593125Z 2024-05-25T18:03:01.358+0000 [DEBUG] turborepo_telemetry::config: Telemetry config path: /home/runner/.config/turborepo/telemetry.json
2024-05-25T18:03:01.3594503Z 2024-05-25T18:03:01.359+0000 [DEBUG] turborepo_telemetry::config: Telemetry config path: /home/runner/.config/turborepo/telemetry.json
2024-05-25T18:03:01.3604557Z 2024-05-25T18:03:01.360+0000 [DEBUG] turborepo_lib::process::child: waiting for task
2024-05-25T18:03:02.3585941Z 2024-05-25T18:03:02.358+0000 [DEBUG] turborepo_telemetry: Starting telemetry event queue flush (num_events=6)
2024-05-25T18:03:02.3587410Z 2024-05-25T18:03:02.358+0000 [DEBUG] turborepo_telemetry: Done telemetry event queue flush
2024-05-25T18:06:01.3169465Z 2024-05-25T18:06:01.316+0000 [DEBUG] log: Sending warning alert CloseNotify
2024-05-25T18:12:35.0089034Z ##[error]The operation was canceled.
allypalanzi commented 5 months ago

I'm getting some timeouts on Github actions as well... here are my verbose logs if helpful:

Run pnpm test
 WARN  Issue while reading "/home/runner/work/glitch-frontend/glitch-frontend/.npmrc". Failed to replace env in config: ${GH_TOKEN}
 WARN  Issue while reading "/home/runner/work/glitch-frontend/glitch-frontend/.npmrc". Failed to replace env in config: ${GH_TOKEN}
> glitch-frontend@0.0.0 test /home/runner/work/glitch-frontend/glitch-frontend
> turbo run test -vv
2024-06-05T14:42:26.177+0000 [DEBUG] turborepo_lib::shim: Global turbo version: 2.0.1
2024-06-05T14:42:26.177+0000 [DEBUG] turborepo_repository::package_json: loading package.json from /home/runner/work/glitch-frontend/glitch-frontend/package.json
2024-06-05T14:42:26.1[8](https://github.com/glitchdotcom/glitch-frontend/actions/runs/9386035498/job/25845628523?pr=443#step:8:9)3+0000 [DEBUG] turborepo_repository::package_json: loading package.json from /home/runner/work/glitch-frontend/package.json
2024-06-05T14:42:26.183+0000 [DEBUG] turborepo_repository::package_json: loading package.json from /home/runner/work/package.json
2024-06-05T14:42:26.183+0000 [DEBUG] turborepo_repository::package_json: loading package.json from /home/runner/package.json
2024-06-05T14:42:26.183+0000 [DEBUG] turborepo_repository::package_json: loading package.json from /home/package.json
2024-06-05T14:42:26.184+0000 [DEBUG] turborepo_repository::package_json: loading package.json from /package.json
2024-06-05T14:42:26.184+0000 [DEBUG] turborepo_lib::shim: Repository Root: /home/runner/work/glitch-frontend/glitch-frontend
2024-06-05T14:42:26.184+0000 [DEBUG] turborepo_lib::shim: No local turbo binary found at: /home/runner/work/glitch-frontend/glitch-frontend/node_modules/turbo-linux-64/bin/turbo
2024-06-05T14:42:26.184+0000 [DEBUG] turborepo_lib::shim: No local turbo binary found at: /home/runner/work/glitch-frontend/glitch-frontend/node_modules/turbo/node_modules/turbo-linux-64/bin/turbo
2024-06-05T14:42:26.184+0000 [DEBUG] turborepo_repository::package_json: loading package.json from /home/runner/work/glitch-frontend/glitch-frontend/node_modules/.pnpm/turbo@2.0.1/node_modules/turbo-linux-64/package.json
2024-06-05T14:42:26.184+0000 [DEBUG] turborepo_lib::shim: Local turbo path: /home/runner/work/glitch-frontend/glitch-frontend/node_modules/.pnpm/turbo-linux-64@2.0.1/node_modules/turbo-linux-64/bin/turbo
2024-06-05T14:42:26.184+0000 [DEBUG] turborepo_lib::shim: Local turbo version: 2.0.1
2024-06-05T14:42:26.184+0000 [DEBUG] turborepo_lib::shim: Currently running turbo is local turbo.
2024-06-05T14:42:26.18[9](https://github.com/glitchdotcom/glitch-frontend/actions/runs/9386035498/job/25845628523?pr=443#step:8:10)+0000 [DEBUG] turborepo_telemetry::config: Telemetry config path: /home/runner/.config/turborepo/telemetry.json
Attention:
Turborepo now collects completely anonymous telemetry regarding usage.
This information is used to shape the Turborepo roadmap and prioritize features.
You can learn more, including how to opt-out if you'd not like to participate in this anonymous program, by visiting the following URL:
https://turbo.build/repo/docs/telemetry
2024-06-05T14:42:26.192+0000 [DEBUG] turborepo_repository::package_json: loading package.json from /home/runner/work/glitch-frontend/glitch-frontend/package.json
2024-06-05T14:42:26.193+0000 [DEBUG] turborepo_telemetry::config: Telemetry config path: /home/runner/.config/turborepo/telemetry.json
2024-06-05T14:42:26.193+0000 [DEBUG] turborepo_lib::run::builder: skipping turbod since we appear to be in a non-interactive context
2024-06-05T14:42:26.193+0000 [DEBUG] turborepo_repository::discovery: discovering packages using caching strategy
2024-06-05T14:42:26.193+0000 [DEBUG] turborepo_repository::discovery: discovering packages using primary strategy
2024-06-05T14:42:26.193+0000 [DEBUG] turborepo_telemetry: Starting telemetry event queue flush (num_events=[10](https://github.com/glitchdotcom/glitch-frontend/actions/runs/9386035498/job/25845628523?pr=443#step:8:11))
2024-06-05T14:42:26.193+0000 [DEBUG] turborepo_repository::discovery: discovering packages using local strategy
2024-06-05T14:42:26.193+0000 [DEBUG] turborepo_telemetry: Done telemetry event queue flush
2024-06-05T14:42:26.198+0000 [DEBUG] globwalk: processing includes: ["apps/*/package.json", "packages/*/package.json"]
2024-06-05T14:42:26.198+0000 [DEBUG] globwalk: processing excludes: ["**/node_modules/**", "**/bower_components/**"]
2024-06-05T14:42:26.198+0000 [DEBUG] globwalk: processed includes: ["/home/runner/work/glitch-frontend/glitch-frontend/apps/*/package.json", "/home/runner/work/glitch-frontend/glitch-frontend/packages/*/package.json"]
2024-06-05T14:42:26.198+0000 [DEBUG] globwalk: processed excludes: ["/home/runner/work/glitch-frontend/glitch-frontend/**/node_modules/**", "/home/runner/work/glitch-frontend/glitch-frontend/**/bower_components/**"]
2024-06-05T14:42:26.199+0000 [DEBUG] log: No cached session for DnsName(DnsName(DnsName("telemetry.vercel.com")))
2024-06-05T14:42:26.199+0000 [DEBUG] log: Not resuming any session
2024-06-05T14:42:26.206+0000 [DEBUG] turborepo_repository::package_json: loading package.json from /home/runner/work/glitch-frontend/glitch-frontend/packages/openapi/package.json
2024-06-05T14:42:26.206+0000 [DEBUG] turborepo_repository::package_json: loading package.json from /home/runner/work/glitch-frontend/glitch-frontend/packages/design-tokens/package.json
2024-06-05T14:42:26.206+0000 [DEBUG] turborepo_repository::package_json: loading package.json from /home/runner/work/glitch-frontend/glitch-frontend/apps/design-docs/package.json
2024-06-05T14:42:26.206+0000 [DEBUG] turborepo_repository::package_json: loading package.json from /home/runner/work/glitch-frontend/glitch-frontend/apps/blog/package.json
2024-06-05T14:42:26.206+0000 [DEBUG] turborepo_repository::package_json: loading package.json from /home/runner/work/glitch-frontend/glitch-frontend/apps/dotcom/package.json
2024-06-05T14:42:26.206+0000 [DEBUG] turborepo_repository::package_json: loading package.json from /home/runner/work/glitch-frontend/glitch-frontend/packages/tsconfig/package.json
2024-06-05T14:42:26.206+0000 [DEBUG] turborepo_repository::package_json: loading package.json from /home/runner/work/glitch-frontend/glitch-frontend/packages/browserslist-config-glitch/package.json
2024-06-05T14:42:26.207+0000 [DEBUG] turborepo_repository::discovery: discovering packages using caching strategy
2024-06-05T14:42:26.207+0000 [DEBUG] turborepo_repository::discovery: discovering packages using caching strategy
2024-06-05T14:42:26.2[11](https://github.com/glitchdotcom/glitch-frontend/actions/runs/9386035498/job/25845628523?pr=443#step:8:12)+0000 [DEBUG] log: Using ciphersuite TLS13_AES_[12](https://github.com/glitchdotcom/glitch-frontend/actions/runs/9386035498/job/25845628523?pr=443#step:8:13)8_GCM_SHA256
2024-06-05T14:42:26.211+0000 [DEBUG] log: Not resuming
2024-06-05T14:42:26.211+0000 [DEBUG] log: TLS1.3 encrypted extensions: [Protocols([6832])]
2024-06-05T14:42:26.211+0000 [DEBUG] log: ALPN protocol is Some(b"h2")
2024-06-05T14:42:26.212+0000 [DEBUG] log: Ticket saved
2024-06-05T14:42:26.272+0000 [DEBUG] turborepo_repository::discovery: discovering packages using caching strategy
• Packages in scope: @glitchdotcom/browserslist-config-glitch, blog, design-docs, design-tokens, dotcom, openapi, tsconfig
• Running test in 7 packages
2024-06-05T14:42:26.272+0000 [DEBUG] turborepo_lib::run::task_access: TASK ACCESS TRACE: unable to restore config from cache
2024-06-05T14:42:26.273+0000 [DEBUG] turborepo_telemetry::config: Telemetry config path: /home/runner/.config/turborepo/telemetry.json
2024-06-05T14:42:26.276+0000 [DEBUG] turborepo_telemetry::config: Telemetry config path: /home/runner/.config/turborepo/telemetry.json
2024-06-05T14:42:26.276+0000 [DEBUG] turborepo_telemetry::config: Telemetry config path: /home/runner/.config/turborepo/telemetry.json
• Remote caching disabled
2024-06-05T14:42:26.276+0000 [DEBUG] turborepo_telemetry::config: Telemetry config path: /home/runner/.config/turborepo/telemetry.json
2024-06-05T14:42:26.293+0000 [DEBUG] turborepo_telemetry::config: Telemetry config path: /home/runner/.config/turborepo/telemetry.json
2024-06-05T14:42:26.293+0000 [DEBUG] turborepo_telemetry::config: Telemetry config path: /home/runner/.config/turborepo/telemetry.json
2024-06-05T14:42:26.306+0000 [DEBUG] turborepo_telemetry: Starting telemetry event queue flush (num_events=10)
2024-06-05T14:42:26.306+0000 [DEBUG] turborepo_telemetry: Done telemetry event queue flush
2024-06-05T14:42:26.306+0000 [DEBUG] turborepo_telemetry::config: Telemetry config path: /home/runner/.config/turborepo/telemetry.json
2024-06-05T14:42:26.307+0000 [DEBUG] turborepo_telemetry::config: Telemetry config path: /home/runner/.config/turborepo/telemetry.json
2024-06-05T14:42:26.323+0000 [DEBUG] turborepo_telemetry::config: Telemetry config path: /home/runner/.config/turborepo/telemetry.json
2024-06-05T14:42:26.323+0000 [DEBUG] turborepo_telemetry::config: Telemetry config path: /home/runner/.config/turborepo/telemetry.json
2024-06-05T14:42:26.331+0000 [DEBUG] turborepo_telemetry::config: Telemetry config path: /home/runner/.config/turborepo/telemetry.json
2024-06-05T14:42:26.331+0000 [DEBUG] turborepo_telemetry::config: Telemetry config path: /home/runner/.config/turborepo/telemetry.json
2024-06-05T14:42:26.344+0000 [DEBUG] turborepo_telemetry::config: Telemetry config path: /home/runner/.config/turborepo/telemetry.json
2024-06-05T14:42:26.344+0000 [DEBUG] turborepo_telemetry::config: Telemetry config path: /home/runner/.config/turborepo/telemetry.json
2024-06-05T14:42:26.368+0000 [DEBUG] turborepo_telemetry::config: Telemetry config path: /home/runner/.config/turborepo/telemetry.json
2024-06-05T14:42:26.368+0000 [DEBUG] turborepo_telemetry::config: Telemetry config path: /home/runner/.config/turborepo/telemetry.json
2024-06-05T14:42:26.381+0000 [DEBUG] turborepo_telemetry::config: Telemetry config path: /home/runner/.config/turborepo/telemetry.json
2024-06-05T14:42:26.381+0000 [DEBUG] turborepo_telemetry::config: Telemetry config path: /home/runner/.config/turborepo/telemetry.json
2024-06-05T14:42:26.382+0000 [DEBUG] turborepo_telemetry: Starting telemetry event queue flush (num_events=10)
2024-06-05T14:42:26.382+0000 [DEBUG] turborepo_telemetry: Done telemetry event queue flush
2024-06-05T14:42:26.390+0000 [DEBUG] turborepo_telemetry::config: Telemetry config path: /home/runner/.config/turborepo/telemetry.json
2024-06-05T14:42:26.390+0000 [DEBUG] turborepo_telemetry::config: Telemetry config path: /home/runner/.config/turborepo/telemetry.json
2024-06-05T14:42:26.657+0000 [DEBUG] turborepo_lib::run::global_hash: global hash env vars []
2024-06-05T14:42:26.657+0000 [DEBUG] turborepo_lib::run::global_hash: external deps hash: a2a0fbb583989ef2
2024-06-05T14:42:26.657+0000 [DEBUG] turborepo_lib::run: running visitor
2024-06-05T14:42:26.657+0000 [DEBUG] turborepo_telemetry::config: Telemetry config path: /home/runner/.config/turborepo/telemetry.json
2024-06-05T14:42:26.657+0000 [DEBUG] turborepo_telemetry::config: Telemetry config path: /home/runner/.config/turborepo/telemetry.json
2024-06-05T14:42:26.657+0000 [DEBUG] turborepo_lib::task_hash: task hash env vars for design-tokens:build
 vars: []
2024-06-05T14:42:26.657+0000 [DEBUG] turborepo_lib::task_graph::visitor: task design-tokens#build hash is eac1[13](https://github.com/glitchdotcom/glitch-frontend/actions/runs/9386035498/job/25845628523?pr=443#step:8:14)dab88acbea
2024-06-05T14:42:26.658+0000 [DEBUG] turborepo_telemetry::config: Telemetry config path: /home/runner/.config/turborepo/telemetry.json
2024-06-05T14:42:26.658+0000 [DEBUG] turborepo_telemetry::config: Telemetry config path: /home/runner/.config/turborepo/telemetry.json
2024-06-05T[14](https://github.com/glitchdotcom/glitch-frontend/actions/runs/9386035498/job/25845628523?pr=443#step:8:15):42:26.658+0000 [DEBUG] turborepo_lib::task_hash: task hash env vars for design-tokens:test
 vars: []
2024-06-05T14:42:26.658+0000 [DEBUG] turborepo_lib::task_graph::visitor: task design-tokens#test hash is 81f2db4f93b917e9
2024-06-05T14:42:26.658+0000 [DEBUG] log: Engine visitor dropped callback sender without sending result
2024-06-05T14:42:26.658+0000 [DEBUG] turborepo_telemetry::config: Telemetry config path: /home/runner/.config/turborepo/telemetry.json
2024-06-05T14:42:26.658+0000 [DEBUG] turborepo_telemetry::config: Telemetry config path: /home/runner/.config/turborepo/telemetry.json
2024-06-05T14:42:26.658+0000 [DEBUG] turborepo_lib::task_hash: task hash env vars for tsconfig:test
 vars: []
2024-06-05T14:42:26.658+0000 [DEBUG] turborepo_lib::task_graph::visitor: task tsconfig#test hash is cd0ab2c9a79dfc99
2024-06-05T14:42:26.659+0000 [DEBUG] turborepo_telemetry::config: Telemetry config path: /home/runner/.config/turborepo/telemetry.json
2024-06-05T14:42:26.659+0000 [DEBUG] turborepo_telemetry::config: Telemetry config path: /home/runner/.config/turborepo/telemetry.json
2024-06-05T14:42:26.659+0000 [DEBUG] turborepo_lib::task_hash: task hash env vars for @glitchdotcom/browserslist-config-glitch:test
 vars: []
2024-06-05T14:42:26.659+0000 [DEBUG] turborepo_lib::task_graph::visitor: task @glitchdotcom/browserslist-config-glitch#test hash is 4da3071a3fdfb6e0
2024-06-05T14:42:26.659+0000 [DEBUG] log: Engine visitor dropped callback sender without sending result
2024-06-05T14:42:26.659+0000 [DEBUG] log: Engine visitor dropped callback sender without sending result
2024-06-05T14:42:26.659+0000 [DEBUG] log: Engine visitor dropped callback sender without sending result
2024-06-05T14:42:26.659+0000 [DEBUG] turborepo_telemetry::config: Telemetry config path: /home/runner/.config/turborepo/telemetry.json
2024-06-05T14:42:26.659+0000 [DEBUG] turborepo_telemetry::config: Telemetry config path: /home/runner/.config/turborepo/telemetry.json
2024-06-05T14:42:26.659+0000 [DEBUG] turborepo_lib::task_hash: task hash env vars for @glitchdotcom/browserslist-config-glitch:build
 vars: []
2024-06-05T14:42:26.659+0000 [DEBUG] turborepo_lib::task_graph::visitor: task @glitchdotcom/browserslist-config-glitch#build hash is 73d83b6b7f82b63a
2024-06-05T14:42:26.659+0000 [DEBUG] turborepo_telemetry: Starting telemetry event queue flush (num_events=10)
2024-06-05T14:42:26.659+0000 [DEBUG] turborepo_telemetry: Done telemetry event queue flush
2024-06-05T14:42:26.660+0000 [DEBUG] turborepo_telemetry::config: Telemetry config path: /home/runner/.config/turborepo/telemetry.json
2024-06-05T14:42:26.660+0000 [DEBUG] turborepo_telemetry::config: Telemetry config path: /home/runner/.config/turborepo/telemetry.json
2024-06-05T14:42:26.660+0000 [DEBUG] log: Engine visitor dropped callback sender without sending result
2024-06-05T14:42:26.660+0000 [DEBUG] turborepo_lib::task_hash: task hash env vars for tsconfig:build
 vars: []
2024-06-05T14:42:26.660+0000 [DEBUG] turborepo_lib::task_graph::visitor: task tsconfig#build hash is 76835719dfb3a5df
2024-06-05T14:42:26.660+0000 [DEBUG] turborepo_telemetry::config: Telemetry config path: /home/runner/.config/turborepo/telemetry.json
2024-06-05T14:42:26.660+0000 [DEBUG] log: Engine visitor dropped callback sender without sending result
2024-06-05T14:42:26.660+0000 [DEBUG] turborepo_telemetry::config: Telemetry config path: /home/runner/.config/turborepo/telemetry.json
2024-06-05T14:42:26.660+0000 [DEBUG] turborepo_lib::task_hash: task hash env vars for openapi:test
 vars: []
2024-06-05T14:42:26.660+0000 [DEBUG] turborepo_lib::task_graph::visitor: task openapi#test hash is 07cef2b7f30df350
2024-06-05T14:42:26.661+0000 [DEBUG] log: Engine visitor dropped callback sender without sending result
2024-06-05T14:42:26.661+0000 [DEBUG] turborepo_telemetry::config: Telemetry config path: /home/runner/.config/turborepo/telemetry.json
2024-06-05T14:42:26.661+0000 [DEBUG] turborepo_telemetry::config: Telemetry config path: /home/runner/.config/turborepo/telemetry.json
2024-06-05T14:42:26.661+0000 [DEBUG] turborepo_lib::task_hash: auto detected framework for design-docs
2024-06-05T14:42:26.661+0000 [DEBUG] turborepo_lib::task_hash: framework: vite, env_prefix: ["VITE_*"]
2024-06-05T14:42:26.663+0000 [DEBUG] turborepo_lib::task_hash: task hash env vars for design-docs:test
 vars: []
2024-06-05T14:42:26.664+0000 [DEBUG] turborepo_lib::task_graph::visitor: task design-docs#test hash is 5ea[17](https://github.com/glitchdotcom/glitch-frontend/actions/runs/9386035498/job/25845628523?pr=443#step:8:18)e2afdee5a37
2024-06-05T14:42:26.664+0000 [DEBUG] log: Engine visitor dropped callback sender without sending result
2024-06-05T14:42:26.664+0000 [DEBUG] turborepo_telemetry::config: Telemetry config path: /home/runner/.config/turborepo/telemetry.json
2024-06-05T14:42:26.664+0000 [DEBUG] turborepo_telemetry::config: Telemetry config path: /home/runner/.config/turborepo/telemetry.json
[20](https://github.com/glitchdotcom/glitch-frontend/actions/runs/9386035498/job/25845628523?pr=443#step:8:21)24-06-05T14:42:26.664+0000 [DEBUG] turborepo_lib::task_hash: task hash env vars for blog:test
 vars: []
2024-06-05T14:42:26.664+0000 [DEBUG] turborepo_lib::task_graph::visitor: task blog#test hash is 261bdde80298afb6
2024-06-05T14:42:26.665+0000 [DEBUG] log: Engine visitor dropped callback sender without sending result
2024-06-05T14:42:26.665+0000 [DEBUG] turborepo_telemetry::config: Telemetry config path: /home/runner/.config/turborepo/telemetry.json
2024-06-05T14:42:26.665+0000 [DEBUG] turborepo_telemetry::config: Telemetry config path: /home/runner/.config/turborepo/telemetry.json
2024-06-05T14:42:26.665+0000 [DEBUG] turborepo_lib::task_hash: auto detected framework for dotcom
2024-06-05T14:42:26.665+0000 [DEBUG] turborepo_lib::task_hash: framework: vite, env_prefix: ["VITE_*"]
2024-06-05T14:42:26.667+0000 [DEBUG] turborepo_lib::task_hash: task hash env vars for dotcom:test
 vars: []
2024-06-05T14:42:26.667+0000 [DEBUG] turborepo_lib::task_graph::visitor: task dotcom#test hash is 3b7149d39ce1094c
2024-06-05T14:42:26.667+0000 [DEBUG] turborepo_telemetry::config: Telemetry config path: /home/runner/.config/turborepo/telemetry.json
2024-06-05T14:42:26.668+0000 [DEBUG] turborepo_lib::process::child: waiting for task
2024-06-05T14:42:27.665+0000 [DEBUG] turborepo_telemetry: Starting telemetry event queue flush (num_events=7)
2024-06-05T14:42:27.666+0000 [DEBUG] turborepo_telemetry: Done telemetry event queue flush
2024-06-05T14:45:26.[21](https://github.com/glitchdotcom/glitch-frontend/actions/runs/9386035498/job/25845628523?pr=443#step:8:22)3+0000 [DEBUG] log: Sending warning alert CloseNotify
tomcru commented 5 months ago

Same issue over here inside GitHub Action, just stuck waiting for task.

Logs are on 2.0.1 - also tried ^2.0.0-canary.4. Command works fine on local and 1.13.4.

Run npm run api/test:integration
> api/test:integration
> turbo test:integration --filter=@repo/api -vv
2024-06-05T19:47:10.522+0000 [DEBUG] turborepo_lib::shim: Global turbo version: 2.0.1
2024-06-05T19:47:10.523+0000 [DEBUG] turborepo_repository::package_json: loading package.json from /home/runner/work/mono/mono/package.json
2024-06-05T19:47:10.526+0000 [DEBUG] turborepo_repository::package_json: loading package.json from /home/runner/work/mono/package.json
2024-06-05T19:47:10.526+0000 [DEBUG] turborepo_repository::package_json: loading package.json from /home/runner/work/package.json
2024-06-05T19:47:10.526+0000 [DEBUG] turborepo_repository::package_json: loading package.json from /home/runner/package.json
2024-06-05T19:47:10.526+0000 [DEBUG] turborepo_repository::package_json: loading package.json from /home/package.json
2024-06-05T19:47:10.526+0000 [DEBUG] turborepo_repository::package_json: loading package.json from /package.json
2024-06-05T19:47:10.526+0000 [DEBUG] turborepo_lib::shim: Repository Root: /home/runner/work/mono/mono
2024-06-05T19:47:10.526+0000 [DEBUG] turborepo_lib::shim: No local turbo binary found at: /home/runner/work/mono/mono/node_modules/turbo-linux-64/bin/turbo
2024-06-05T19:47:10.526+0000 [DEBUG] turborepo_lib::shim: No local turbo binary found at: /home/runner/work/mono/mono/node_modules/turbo/node_modules/turbo-linux-64/bin/turbo
2024-06-05T19:47:10.527+0000 [DEBUG] turborepo_lib::shim: Running command as global turbo
2024-06-05T19:47:10.532+0000 [DEBUG] turborepo_telemetry::config: Telemetry config path: /home/runner/.config/turborepo/telemetry.json
Attention:
Turborepo now collects completely anonymous telemetry regarding usage.
This information is used to shape the Turborepo roadmap and prioritize features.
You can learn more, including how to opt-out if you'd not like to participate in this anonymous program, by visiting the following URL:
https://turbo.build/repo/docs/telemetry
2024-06-05T19:47:10.538+0000 [DEBUG] turborepo_repository::package_json: loading package.json from /home/runner/work/mono/mono/package.json
2024-06-05T19:47:10.538+0000 [DEBUG] turborepo_telemetry::config: Telemetry config path: /home/runner/.config/turborepo/telemetry.json
2024-06-05T19:47:10.538+0000 [DEBUG] turborepo_lib::run::builder: skipping turbod since we appear to be in a non-interactive context
2024-06-05T19:47:10.538+0000 [DEBUG] turborepo_repository::discovery: discovering packages using caching strategy
2024-06-05T19:47:10.538+0000 [DEBUG] turborepo_repository::discovery: discovering packages using primary strategy
2024-06-05T19:47:10.538+0000 [DEBUG] turborepo_repository::discovery: discovering packages using local strategy
2024-06-05T19:47:10.540+0000 [DEBUG] globwalk: processing includes: ["apps/*/package.json", "packages/*/package.json"]
2024-06-05T19:47:10.540+0000 [DEBUG] globwalk: processing excludes: ["**/node_modules/**"]
2024-06-05T19:47:10.540+0000 [DEBUG] globwalk: processed includes: ["/home/runner/work/mono/mono/apps/*/package.json", "/home/runner/work/mono/mono/packages/*/package.json"]
2024-06-05T19:47:10.540+0000 [DEBUG] globwalk: processed excludes: ["/home/runner/work/mono/mono/**/node_modules/**"]
2024-06-05T19:47:10.541+0000 [DEBUG] turborepo_telemetry: Starting telemetry event queue flush (num_events=10)
2024-06-05T19:47:10.541+0000 [DEBUG] turborepo_telemetry: Done telemetry event queue flush
2024-06-05T19:47:10.551+0000 [DEBUG] log: No cached session for DnsName(DnsName(DnsName("telemetry.vercel.com")))
2024-06-05T19:47:10.551+0000 [DEBUG] turborepo_repository::package_json: loading package.json from /home/runner/work/mono/mono/apps/cron/package.json
2024-06-05T19:47:10.551+0000 [DEBUG] log: Not resuming any session
2024-06-05T19:47:10.551+0000 [DEBUG] turborepo_repository::package_json: loading package.json from /home/runner/work/mono/mono/apps/api/package.json
2024-06-05T19:47:10.551+0000 [DEBUG] turborepo_repository::package_json: loading package.json from /home/runner/work/mono/mono/packages/eslint-config/package.json
2024-06-05T19:47:10.551+0000 [DEBUG] turborepo_repository::package_json: loading package.json from /home/runner/work/mono/mono/apps/web/package.json
2024-06-05T19:47:10.551+0000 [DEBUG] turborepo_repository::package_json: loading package.json from /home/runner/work/mono/mono/packages/ui/package.json
2024-06-05T19:47:10.551+0000 [DEBUG] turborepo_repository::package_json: loading package.json from /home/runner/work/mono/mono/apps/igdb/package.json
2024-06-05T19:47:10.551+0000 [DEBUG] turborepo_repository::package_json: loading package.json from /home/runner/work/mono/mono/packages/tailwind-config/package.json
2024-06-05T19:47:10.551+0000 [DEBUG] turborepo_repository::package_json: loading package.json from /home/runner/work/mono/mono/packages/prettier-config/package.json
2024-06-05T19:47:10.551+0000 [DEBUG] turborepo_repository::package_json: loading package.json from /home/runner/work/mono/mono/packages/typescript-config/package.json
2024-06-05T19:47:10.551+0000 [DEBUG] turborepo_repository::package_json: loading package.json from /home/runner/work/mono/mono/apps/lambda/package.json
2024-06-05T19:47:10.552+0000 [DEBUG] turborepo_repository::discovery: discovering packages using caching strategy
2024-06-05T19:47:10.552+0000 [DEBUG] turborepo_repository::discovery: discovering packages using caching strategy
2024-06-05T19:47:10.558+0000 [DEBUG] log: Using ciphersuite TLS13_AES_128_GCM_SHA256
2024-06-05T19:47:10.558+0000 [DEBUG] log: Not resuming
2024-06-05T19:47:10.558+0000 [DEBUG] log: TLS1.3 encrypted extensions: [Protocols([6832])]
2024-06-05T19:47:10.558+0000 [DEBUG] log: ALPN protocol is Some(b"h2")
2024-06-05T19:47:10.558+0000 [DEBUG] log: Ticket saved
2024-06-05T19:47:10.586+0000 [DEBUG] turborepo_repository::discovery: discovering packages using caching strategy
2024-06-05T19:47:10.586+0000 [DEBUG] turborepo_lib::run::task_access: TASK ACCESS TRACE: unable to restore config from cache
• Packages in scope: @repo/api
2024-06-05T19:47:10.588+0000 [DEBUG] turborepo_telemetry::config: Telemetry config path: /home/runner/.config/turborepo/telemetry.json
• Running test:integration in 1 packages
• Remote caching disabled
2024-06-05T19:47:10.589+0000 [DEBUG] turborepo_telemetry::config: Telemetry config path: /home/runner/.config/turborepo/telemetry.json
2024-06-05T19:47:10.589+0000 [DEBUG] turborepo_telemetry::config: Telemetry config path: /home/runner/.config/turborepo/telemetry.json
2024-06-05T19:47:10.589+0000 [DEBUG] turborepo_telemetry::config: Telemetry config path: /home/runner/.config/turborepo/telemetry.json
2024-06-05T19:47:10.601+0000 [DEBUG] turborepo_lib::run::global_hash: global hash env vars ["BETTERSTACK_TOKEN", "CLOUDFLARE_ID", "CLOUDFLARE_KEY", "CLOUDFLARE_PUBLIC_KEY", "DATABASE_URL", "DISCORD_CLIENT_ID", "DISCORD_CLIENT_SECRET", "JWT_SECRET", "LOOPS_API_KEY", "NODE_ENV", "STEAM_API_KEY"]
2024-06-05T19:47:10.603+0000 [DEBUG] globwalk: processing includes: ["**/.env.*local"]
2024-06-05T19:47:10.603+0000 [DEBUG] globwalk: processing excludes: ["**/node_modules/**"]
2024-06-05T19:47:10.603+0000 [DEBUG] globwalk: processed includes: ["/home/runner/work/mono/mono/**/.env.*local"]
2024-06-05T19:47:10.603+0000 [DEBUG] globwalk: processed excludes: ["/home/runner/work/mono/mono/**/node_modules/**"]
2024-06-05T19:47:10.610+0000 [DEBUG] turborepo_lib::run::global_hash: external deps hash: cbb1da543c251cbb
2024-06-05T19:47:10.610+0000 [DEBUG] turborepo_lib::run: running visitor
2024-06-05T19:47:10.610+0000 [DEBUG] turborepo_telemetry::config: Telemetry config path: /home/runner/.config/turborepo/telemetry.json
2024-06-05T19:47:10.610+0000 [DEBUG] turborepo_telemetry::config: Telemetry config path: /home/runner/.config/turborepo/telemetry.json
2024-06-05T19:47:10.610+0000 [DEBUG] turborepo_telemetry::config: Telemetry config path: /home/runner/.config/turborepo/telemetry.json
2024-06-05T19:47:10.610+0000 [DEBUG] turborepo_telemetry::config: Telemetry config path: /home/runner/.config/turborepo/telemetry.json
2024-06-05T19:47:10.612+0000 [DEBUG] turborepo_lib::task_hash: task hash env vars for @repo/api:test:integration
 vars: []
2024-06-05T19:47:10.612+0000 [DEBUG] turborepo_lib::task_graph::visitor: task @repo/api#test:integration hash is f0a1ac4b790ba9f0
2024-06-05T19:47:10.612+0000 [DEBUG] turborepo_telemetry::config: Telemetry config path: /home/runner/.config/turborepo/telemetry.json
2024-06-05T19:47:10.612+0000 [DEBUG] turborepo_telemetry::config: Telemetry config path: /home/runner/.config/turborepo/telemetry.json
2024-06-05T19:47:10.614+0000 [DEBUG] turborepo_lib::process::child: waiting for task
2024-06-05T19:47:11.612+0000 [DEBUG] turborepo_telemetry: Starting telemetry event queue flush (num_events=8)
2024-06-05T19:47:11.612+0000 [DEBUG] turborepo_telemetry: Done telemetry event queue flush
trm217 commented 5 months ago

Same issue here with v2.0.2. I have a test task in a nextjs app that uses vitest. In CI the github action just stops and prints no output after pnpm run test is being executed.

ryanbruns commented 5 months ago

Very similar issue here when trying to run a .sh command on the other end of our scripting with 2.0.3.

Shortened down the logs to after the script was invoked

@react2go/fulfillmentvu2:test:script: 
@react2go/fulfillmentvu2:test:script: > @react2go/my-app@0.1.0 test:script /builds/ware2go/engineering/frontend/react2go/apps/my-app
@react2go/fulfillmentvu2:test:script: > ./cicd/prepare-build.sh
@react2go/fulfillmentvu2:test:script: 
@react2go/fulfillmentvu2:test:script: Preparing App...
@react2go/fulfillmentvu2:test:script: Script is going
@react2go/fulfillmentvu2:test:script: Script is going and is in the else block
@react2go/fulfillmentvu2:test:script: Start: Setting up App in non-production mode.
@react2go/fulfillmentvu2:test:script: 
2024-06-07T15:07:05.012+0000 [DEBUG] turborepo_telemetry: Starting telemetry event queue flush (num_events=4)
2024-06-07T15:07:05.012+0000 [DEBUG] turborepo_telemetry: Done telemetry event queue flush
2024-06-07T15:07:05.013+0000 [TRACE] log: signal: Want
2024-06-07T15:07:05.013+0000 [TRACE] log: signal: Want
2024-06-07T15:07:05.117+0000 [TRACE] tokio_util::codec::framed_impl: attempting to decode a frame
2024-06-07T15:07:05.117+0000 [TRACE] tokio_util::codec::framed_impl: frame decoded from buffer
2024-06-07T15:07:05.117+0000 [TRACE] tokio_util::codec::framed_impl: attempting to decode a frame
2024-06-07T15:07:05.117+0000 [TRACE] tokio_util::codec::framed_impl: frame decoded from buffer
2024-06-07T15:07:05.117+0000 [TRACE] tokio_util::codec::framed_impl: attempting to decode a frame
2024-06-07T15:10:03.671+0000 [TRACE] log: signal: Closed
2024-06-07T15:10:03.672+0000 [DEBUG] log: Sending warning alert CloseNotify
2024-06-07T15:10:03.672+0000 [TRACE] log: deregistering event source from poller

We're using pnpm and on a self hosted Gitlab CI Runner and we're running on k8s for all of this. The same script works fine on the v1 version available right now.

estebane97 commented 5 months ago

I believe this is related to the Environment variables being set as strict mode by default on the latest version.

https://turbo.build/repo/docs/crafting-your-repository/using-environment-variables#strict-mode

Strict Mode filters the environment variables available to a task's runtime to only those that are specified in the globalEnv and env keys in turbo.json.

in Vitest if not watch option is set, it will default to the environment variable CI:

https://vitest.dev/config/#watch

Default: !process.env.CI

If that variable is not available to the test task, then it will enable watch mode...

adding the env variable to the test task worked for me:

"test": { "env": ["CI"], ... }

anthonyshew commented 5 months ago

Folks, we've shipped a few changes in 2.0.4-canary.0 that we believe should address some (all?) of the issues being seen here.

Please do try it out and report back so we know we squashed it. If not, would appreciate that feedback as well. 🙏

allypalanzi commented 5 months ago

that fixed it for me! thank you :)

jakubmazanec commented 5 months ago

@anthonyshew Sadly, Turborepo still hangs: https://github.com/jakubmazanec/apps/actions/runs/9453810487/job/26039943627?pr=21

anthonyshew commented 5 months ago

@jakubmazanec If you re-run it with --env-mode=loose, does it pass?

jakubmazanec commented 5 months ago

@anthonyshew No, it still hangs: https://github.com/jakubmazanec/apps/actions/runs/9455590634/job/26045708340?pr=21

trm217 commented 5 months ago

Doesn't seems to have worked. I will retry once I have credits again for running GitHub Actions (this bug cost me 1'000 min of compute 🫠)

tomcru commented 5 months ago

2.0.4-canary.0 Fixed this for me, thank you 👍

clintonwoo commented 5 months ago

My build was hanging but turns out it was because storybook telemetry was timing out in CI. I used env variables to disable and it worked. The env vars to disable telemetry don't seem to work for turborepo though since it still prints out many logs with -vvv saying it's doing telemetry

jakubmazanec commented 5 months ago

@anthonyshew Any progress?

anthonyshew commented 5 months ago

Does setting the TURBO_UI system variable to false help for those of you still having issues?

sethidden commented 5 months ago

Does setting the TURBO_UI system variable to false help for those of you still having issues?

Setting TURBO_UI=false infront of turbo run in my package.json script did not resolve the issue on turbo 2.0.5 - it's still freezing.

EDIT: turbo 2.0.7-canary.0 same thing

jakubmazanec commented 5 months ago

TURBO_UI=0 has no effect, still hangs.

jakubmazanec commented 5 months ago

Also, please remember that - at least for me - the problem started with version 1.13.0. In my original comment I used v1.10.3, and now I have also tested versions 1.11.1 and 1.12.5, both working fine.

kasper573 commented 4 months ago

2.0.4-canary.0 seemed to have fix this error for me as well! Thank you :)

(Was using turbo 2.0.3 before)

gnohj commented 4 months ago

@jakubmazanec noticing the same thing.. we are trying to upgrade to 2.0.6 from 1.13.3 and it hangs on both versions.. rolled back to 1.12.5 and it works

kpervin commented 4 months ago

Attempting to use 2.0.7 and we're getting hang-ups, upgraded from 1.10.16. Interestingly when running --vvv and having TURBO_TELEMETRY_DISABLED=1 OR running turbo telemetry disable still prints logs regarding telemetry as mentioned here. The Error: The operation was canceled. was due to timeout.

Logs:

Run yarn build --cache-dir=.turbo -vvv
  yarn build --cache-dir=.turbo -vvv
  shell: /usr/bin/bash -e {0}
  env:
    ENV_FILE_BASE64: ***
    TURBO_TELEMETRY_DISABLED: 1
2024-07-18T13:41:55.901+0000 [DEBUG] turborepo_lib::shim: Global turbo version: 2.0.7
2024-07-18T13:41:55.901+0000 [DEBUG] turborepo_repository::package_json: loading package.json from /home/runner/work/repository/repository/package.json
2024-07-18T13:41:55.907+0000 [DEBUG] turborepo_repository::package_json: loading package.json from /home/runner/work/repository/package.json
2024-07-18T13:41:55.907+0000 [DEBUG] turborepo_repository::package_json: loading package.json from /home/runner/work/package.json
2024-07-18T13:41:55.907+0000 [DEBUG] turborepo_repository::package_json: loading package.json from /home/runner/package.json
2024-07-18T13:41:55.907+0000 [DEBUG] turborepo_repository::package_json: loading package.json from /home/package.json
2024-07-18T13:41:55.907+0000 [DEBUG] turborepo_repository::package_json: loading package.json from /package.json
2024-07-18T13:41:55.907+0000 [DEBUG] turborepo_lib::shim: Repository Root: /home/runner/work/repository/repository
2024-07-18T13:41:55.907+0000 [DEBUG] turborepo_repository::package_json: loading package.json from /home/runner/work/repository/repository/node_modules/turbo-linux-64/package.json
2024-07-18T13:41:55.907+0000 [DEBUG] turborepo_lib::shim::local_turbo_state: Local turbo path: /home/runner/work/repository/repository/node_modules/turbo-linux-64/bin/turbo
2024-07-18T13:41:55.907+0000 [DEBUG] turborepo_lib::shim::local_turbo_state: Local turbo version: 2.0.7
2024-07-18T13:41:55.907+0000 [DEBUG] turborepo_lib::shim: Currently running turbo is local turbo.
2024-07-18T13:41:55.908+0000 [TRACE] log: registering event source with poller: token=Token(1), interests=READABLE
2024-07-18T13:41:55.914+0000 [DEBUG] turborepo_telemetry::config: Telemetry config path: /home/runner/.config/turborepo/telemetry.json
2024-07-18T13:41:55.917+0000 [TRACE] turborepo_lib::run::builder: performing run on "linux-64"
2024-07-18T13:41:55.917+0000 [DEBUG] turborepo_repository::package_json: loading package.json from /home/runner/work/repository/repository/package.json
2024-07-18T13:41:55.918+0000 [DEBUG] turborepo_telemetry::config: Telemetry config path: /home/runner/.config/turborepo/telemetry.json
2024-07-18T13:41:55.918+0000 [DEBUG] turborepo_telemetry: Starting telemetry event queue flush (num_events=10)
2024-07-18T13:41:55.918+0000 [DEBUG] turborepo_telemetry: Done telemetry event queue flush
2024-07-18T13:41:55.918+0000 [DEBUG] turborepo_lib::run::builder: skipping turbod since we appear to be in a non-interactive context
2024-07-18T13:41:55.918+0000 [DEBUG] turborepo_repository::discovery: discovering packages using caching strategy
2024-07-18T13:41:55.918+0000 [DEBUG] turborepo_repository::discovery: discovering packages using primary strategy
2024-07-18T13:41:55.918+0000 [DEBUG] turborepo_repository::discovery: discovering packages using local strategy
2024-07-18T13:41:55.922+0000 [DEBUG] globwalk: processing includes: ["apps/*/package.json", "packages/*/package.json"]
2024-07-18T13:41:55.922+0000 [DEBUG] globwalk: processing excludes: ["**/node_modules", "**/.git", "**/.yarn"]
2024-07-18T13:41:55.922+0000 [DEBUG] globwalk: processed includes: ["/home/runner/work/repository/repository/apps/*/package.json", "/home/runner/work/repository/repository/packages/*/package.json"]
2024-07-18T13:41:55.922+0000 [DEBUG] globwalk: processed excludes: ["/home/runner/work/repository/repository/**/node_modules/**", "/home/runner/work/repository/repository/**/node_modules", "/home/runner/work/repository/repository/**/.git/**", "/home/runner/work/repository/repository/**/.git", "/home/runner/work/repository/repository/**/.yarn/**", "/home/runner/work/repository/repository/**/.yarn"]
2024-07-18T13:41:55.934+0000 [DEBUG] turborepo_repository::package_json: loading package.json from /home/runner/work/repository/repository/packages/env-validation/package.json
2024-07-18T13:41:55.934+0000 [DEBUG] turborepo_repository::package_json: loading package.json from /home/runner/work/repository/repository/apps/web/package.json
2024-07-18T13:41:55.935+0000 [DEBUG] turborepo_repository::package_json: loading package.json from /home/runner/work/repository/repository/packages/openapi/package.json
2024-07-18T13:41:55.935+0000 [DEBUG] turborepo_repository::package_json: loading package.json from /home/runner/work/repository/repository/apps/server/package.json
2024-07-18T13:41:55.935+0000 [DEBUG] turborepo_repository::discovery: discovering packages using caching strategy
2024-07-18T13:41:55.936+0000 [DEBUG] turborepo_repository::discovery: discovering packages using caching strategy
2024-07-18T13:41:56.033+0000 [DEBUG] turborepo_repository::discovery: discovering packages using caching strategy
2024-07-18T13:41:56.033+0000 [DEBUG] turborepo_lib::run::task_access: TASK ACCESS TRACE: unable to restore config from cache
• Packages in scope: @repo/env-validation, openapi, server, web
• Running build in 4 packages
• Remote caching disabled
2024-07-18T13:41:56.036+0000 [TRACE] turborepo_lib::task_hash: scm running in git mode
2024-07-18T13:41:56.036+0000 [DEBUG] turborepo_telemetry::config: Telemetry config path: /home/runner/.config/turborepo/telemetry.json
2024-07-18T13:41:56.036+0000 [DEBUG] turborepo_telemetry::config: Telemetry config path: /home/runner/.config/turborepo/telemetry.json
2024-07-18T13:41:56.037+0000 [DEBUG] turborepo_telemetry::config: Telemetry config path: /home/runner/.config/turborepo/telemetry.json
2024-07-18T13:41:56.037+0000 [DEBUG] turborepo_telemetry::config: Telemetry config path: /home/runner/.config/turborepo/telemetry.json
2024-07-18T13:41:56.038+0000 [DEBUG] globwalk: processing includes: ["packages/env-validation/src/**/*.ts", "packages/env-validation/package.json", "packages/env-validation/turbo.json"]
2024-07-18T13:41:56.038+0000 [DEBUG] globwalk: processing excludes: []
2024-07-18T13:41:56.038+0000 [DEBUG] globwalk: processed includes: ["/home/runner/work/repository/repository/packages/env-validation/src/**/*.ts", "/home/runner/work/repository/repository/packages/env-validation/package.json", "/home/runner/work/repository/repository/packages/env-validation/turbo.json"]
2024-07-18T13:41:56.038+0000 [DEBUG] globwalk: processed excludes: []
2024-07-18T13:41:56.044+0000 [DEBUG] turborepo_telemetry::config: Telemetry config path: /home/runner/.config/turborepo/telemetry.json
2024-07-18T13:41:56.044+0000 [DEBUG] turborepo_telemetry: Starting telemetry event queue flush (num_events=10)
2024-07-18T13:41:56.044+0000 [DEBUG] turborepo_telemetry: Done telemetry event queue flush
2024-07-18T13:41:56.044+0000 [DEBUG] turborepo_telemetry::config: Telemetry config path: /home/runner/.config/turborepo/telemetry.json
2024-07-18T13:41:56.044+0000 [DEBUG] globwalk: processing includes: ["apps/web/lib/**/*.ts", "apps/web/lib/**/*.graphql", "apps/web/content/**/*.md", "apps/web/pages/**/*.tsx", "apps/web/src/**/*.tsx", "apps/web/src/**/*.ts", "apps/web/styles/**/*.tsx", "apps/web/styles/**/*.ts", "apps/web/styles/**/*.css", "apps/web/package.json", "apps/web/turbo.json"]
2024-07-18T13:41:56.044+0000 [DEBUG] globwalk: processing excludes: []
2024-07-18T13:41:56.044+0000 [DEBUG] globwalk: processed includes: ["/home/runner/work/repository/repository/apps/web/lib/**/*.ts", "/home/runner/work/repository/repository/apps/web/lib/**/*.graphql", "/home/runner/work/repository/repository/apps/web/content/**/*.md", "/home/runner/work/repository/repository/apps/web/pages/**/*.tsx", "/home/runner/work/repository/repository/apps/web/src/**/*.tsx", "/home/runner/work/repository/repository/apps/web/src/**/*.ts", "/home/runner/work/repository/repository/apps/web/styles/**/*.tsx", "/home/runner/work/repository/repository/apps/web/styles/**/*.ts", "/home/runner/work/repository/repository/apps/web/styles/**/*.css", "/home/runner/work/repository/repository/apps/web/package.json", "/home/runner/work/repository/repository/apps/web/turbo.json"]
2024-07-18T13:41:56.044+0000 [DEBUG] globwalk: processed excludes: []
2024-07-18T13:41:56.053+0000 [DEBUG] turborepo_telemetry::config: Telemetry config path: /home/runner/.config/turborepo/telemetry.json
2024-07-18T13:41:56.054+0000 [DEBUG] turborepo_telemetry::config: Telemetry config path: /home/runner/.config/turborepo/telemetry.json
2024-07-18T13:41:56.054+0000 [DEBUG] globwalk: processing includes: ["apps/server/src/**/*.ts", "apps/server/package.json", "apps/server/turbo.json"]
2024-07-18T13:41:56.054+0000 [DEBUG] globwalk: processing excludes: []
2024-07-18T13:41:56.054+0000 [DEBUG] globwalk: processed includes: ["/home/runner/work/repository/repository/apps/server/src/**/*.ts", "/home/runner/work/repository/repository/apps/server/package.json", "/home/runner/work/repository/repository/apps/server/turbo.json"]
2024-07-18T13:41:56.054+0000 [DEBUG] globwalk: processed excludes: []
2024-07-18T13:41:56.133+0000 [DEBUG] turborepo_lib::run::global_hash: global hash env vars []
2024-07-18T13:41:56.136+0000 [DEBUG] globwalk: processing includes: [".env", "tsconfig*.json"]
2024-07-18T13:41:56.136+0000 [DEBUG] globwalk: processing excludes: ["**/node_modules", "**/.git", "**/.yarn"]
2024-07-18T13:41:56.136+0000 [DEBUG] globwalk: '/home/runner/work/repository/repository/.env' doesn''t have metadata
2024-07-18T13:41:56.136+0000 [DEBUG] globwalk: processed includes: ["/home/runner/work/repository/repository/.env", "/home/runner/work/repository/repository/tsconfig*.json"]
2024-07-18T13:41:56.136+0000 [DEBUG] globwalk: processed excludes: ["/home/runner/work/repository/repository/**/node_modules/**", "/home/runner/work/repository/repository/**/node_modules", "/home/runner/work/repository/repository/**/.git/**", "/home/runner/work/repository/repository/**/.git", "/home/runner/work/repository/repository/**/.yarn/**", "/home/runner/work/repository/repository/**/.yarn"]
2024-07-18T13:41:56.144+0000 [DEBUG] turborepo_lib::run::global_hash: external deps hash: dbe93f51c7ecfb1c
2024-07-18T13:41:56.144+0000 [DEBUG] turborepo_lib::run: running visitor
2024-07-18T13:41:56.144+0000 [DEBUG] turborepo_telemetry::config: Telemetry config path: /home/runner/.config/turborepo/telemetry.json
2024-07-18T13:41:56.144+0000 [DEBUG] turborepo_telemetry::config: Telemetry config path: /home/runner/.config/turborepo/telemetry.json
2024-07-18T13:41:56.145+0000 [DEBUG] turborepo_lib::task_hash: task hash env vars for openapi:build
 vars: []
2024-07-18T13:41:56.145+0000 [DEBUG] turborepo_lib::task_graph::visitor: task openapi#build hash is 80f0c30545c976c5
2024-07-18T13:41:56.146+0000 [DEBUG] turborepo_telemetry::config: Telemetry config path: /home/runner/.config/turborepo/telemetry.json
2024-07-18T13:41:56.147+0000 [DEBUG] turborepo_telemetry::config: Telemetry config path: /home/runner/.config/turborepo/telemetry.json
2024-07-18T13:41:56.147+0000 [DEBUG] turborepo_telemetry::config: Telemetry config path: /home/runner/.config/turborepo/telemetry.json
2024-07-18T13:41:56.147+0000 [DEBUG] turborepo_lib::task_hash: task hash env vars for @repo/env-validation:build
 vars: []
2024-07-18T13:41:56.147+0000 [DEBUG] turborepo_lib::task_graph::visitor: task @repo/env-validation#build hash is d858b992cbe95fde
2024-07-18T13:41:56.149+0000 [DEBUG] turborepo_telemetry::config: Telemetry config path: /home/runner/.config/turborepo/telemetry.json
2024-07-18T13:41:56.150+0000 [TRACE] log: registering event source with poller: token=Token(139813316862592), interests=READABLE | WRITABLE
2024-07-18T13:41:56.150+0000 [TRACE] log: registering event source with poller: token=Token(139813316863360), interests=READABLE | WRITABLE
2024-07-18T13:41:56.150+0000 [TRACE] log: registering event source with poller: token=Token(139813316863744), interests=READABLE | WRITABLE
2024-07-18T13:41:56.150+0000 [DEBUG] turborepo_lib::process::child: waiting for task
2024-07-18T13:41:56.152+0000 [TRACE] log: registering event source with poller: token=Token(139813316866176), interests=READABLE | WRITABLE
2024-07-18T13:41:56.152+0000 [TRACE] log: registering event source with poller: token=Token(139813316868608), interests=READABLE | WRITABLE
2024-07-18T13:41:56.152+0000 [TRACE] log: registering event source with poller: token=Token(139813324222720), interests=READABLE | WRITABLE
2024-07-18T13:41:56.152+0000 [DEBUG] turborepo_lib::process::child: waiting for task
2024-07-18T13:41:57.147+0000 [DEBUG] turborepo_telemetry: Starting telemetry event queue flush (num_events=6)
2024-07-18T13:41:57.147+0000 [DEBUG] turborepo_telemetry: Done telemetry event queue flush
2024-07-18T13:42:00.883+0000 [DEBUG] turborepo_lib::process::child: child process exited normally
2024-07-18T13:42:00.883+0000 [DEBUG] turborepo_lib::process::child: child process stopped
2024-07-18T13:42:00.883+0000 [TRACE] log: deregistering event source from poller
2024-07-18T13:42:00.883+0000 [TRACE] log: deregistering event source from poller
2024-07-18T13:42:00.883+0000 [DEBUG] turborepo_lib::run::cache: caching outputs: outputs: TaskOutputs { inclusions: ["packages/env-validation/.turbo/turbo-build.log", "packages/env-validation/dist/**"], exclusions: [] }
2024-07-18T13:42:00.883+0000 [DEBUG] globwalk: processing includes: ["packages/env-validation/.turbo/turbo-build.log", "packages/env-validation/dist/**"]
2024-07-18T13:42:00.883+0000 [DEBUG] globwalk: processing excludes: []
2024-07-18T13:42:00.884+0000 [DEBUG] globwalk: processed includes: ["/home/runner/work/repository/repository/packages/env-validation/.turbo/turbo-build.log", "/home/runner/work/repository/repository/packages/env-validation/dist/**"]
2024-07-18T13:42:00.884+0000 [DEBUG] globwalk: processed excludes: []
@repo/env-validation:build
2024-07-18T13:42:00.888+0000 [DEBUG] turborepo_telemetry::config: Telemetry config path: /home/runner/.config/turborepo/telemetry.json
2024-07-18T13:42:00.888+0000 [DEBUG] turborepo_telemetry::config: Telemetry config path: /home/runner/.config/turborepo/telemetry.json
2024-07-18T13:42:00.888+0000 [DEBUG] turborepo_lib::task_hash: auto detected framework for web
2024-07-18T13:42:00.888+0000 [DEBUG] turborepo_lib::task_hash: framework: nextjs, env_prefix: ["NEXT_PUBLIC_*"]
2024-07-18T13:42:00.890+0000 [DEBUG] turborepo_lib::task_hash: task hash env vars for web:build
 vars: []
2024-07-18T13:42:00.890+0000 [DEBUG] turborepo_lib::task_graph::visitor: task web#build hash is 9bd84a3d62ed98ee
2024-07-18T13:42:00.891+0000 [DEBUG] turborepo_telemetry::config: Telemetry config path: /home/runner/.config/turborepo/telemetry.json
2024-07-18T13:42:00.892+0000 [TRACE] log: registering event source with poller: token=Token(139813324222720), interests=READABLE | WRITABLE
2024-07-18T13:42:00.892+0000 [TRACE] log: registering event source with poller: token=Token(139813316010240), interests=READABLE | WRITABLE
2024-07-18T13:42:00.892+0000 [TRACE] log: registering event source with poller: token=Token(139813316010624), interests=READABLE | WRITABLE
2024-07-18T13:42:00.893+0000 [DEBUG] turborepo_lib::process::child: waiting for task
2024-07-18T13:42:01.891+0000 [DEBUG] turborepo_telemetry: Starting telemetry event queue flush (num_events=2)
2024-07-18T13:42:01.891+0000 [DEBUG] turborepo_telemetry: Done telemetry event queue flush
2024-07-18T13:43:13.758+0000 [DEBUG] turborepo_lib::process::child: child process exited normally
2024-07-18T13:43:13.758+0000 [DEBUG] turborepo_lib::process::child: child process stopped
2024-07-18T13:43:13.758+0000 [TRACE] log: deregistering event source from poller
2024-07-18T13:43:13.758+0000 [TRACE] log: deregistering event source from poller
2024-07-18T13:43:13.758+0000 [DEBUG] turborepo_lib::run::cache: caching outputs: outputs: TaskOutputs { inclusions: ["apps/web/.next/**", "apps/web/.turbo/turbo-build.log"], exclusions: ["apps/web/.next/cache/**"] }
2024-07-18T13:43:13.758+0000 [DEBUG] globwalk: processing includes: ["apps/web/.next/**", "apps/web/.turbo/turbo-build.log"]
2024-07-18T13:43:13.758+0000 [DEBUG] globwalk: processing excludes: ["apps/web/.next/cache/**"]
2024-07-18T13:43:13.758+0000 [DEBUG] globwalk: processed includes: ["/home/runner/work/repository/repository/apps/web/.next/**", "/home/runner/work/repository/repository/apps/web/.turbo/turbo-build.log"]
2024-07-18T13:43:13.758+0000 [DEBUG] globwalk: processed excludes: ["/home/runner/work/repository/repository/apps/web/.next/cache/**"]
web:build
  cache miss, executing 9bd84a3d62ed98ee
  ⚠ No build cache found. Please configure build caching for faster rebuilds. Read more: https://nextjs.org/docs/messages/no-cache
  Attention: Next.js now collects completely anonymous telemetry regarding usage.
  This information is used to shape Next.js' roadmap and prioritize features.
  You can learn more, including how to opt-out if you'd not like to participate in this anonymous program, by visiting the following URL:
  https://nextjs.org/telemetry

  Browserslist: caniuse-lite is outdated. Please run:
    npx browserslist@latest --update-db
    Why you should do it regularly: https://github.com/browserslist/browserslist#browsers-data-updating
     Linting and checking validity of types ...

  ./src/components/pages/OrderHistory/OrderHistoryTable/OrderHistoryTable.tsx
  250:27  Warning: React Hook useCallback received a function whose dependencies are unknown. Pass an inline function instead.  react-hooks/exhaustive-deps

  info  - Need to disable some ESLint rules? Learn more here: https://nextjs.org/docs/basic-features/eslint#disabling-rules
     ▲ Next.js 14.0.3
     - Environments: .env.local

     Creating an optimized production build ...
     Using tsconfig file: ./tsconfig.build.json
   ✓ Compiled successfully
     Collecting page data ...
     Generating static pages (0/13) ...

     Generating static pages (3/13) 

     Generating static pages (6/13) 

     Generating static pages (9/13) 

   ✓ Generating static pages (13/13) 
     Finalizing page optimization ...
     Collecting build traces ...

  Route (pages)                                    Size     First Load JS
  <removing as unimportant>

  ƒ Middleware                                     65.4 kB

  ○  (Static)   prerendered as static content
  ●  (SSG)      prerendered as static HTML (uses getStaticProps)
  λ  (Dynamic)  server-rendered on demand using Node.js

Error: The operation was canceled.

Failing job:

  build:
    name: Build
    if: ${{ !contains(github.event.head_commit.message, 'skip ci') }}
    runs-on: ubuntu-22.04
    timeout-minutes: 10
    steps:
      - name: Checkout
        uses: actions/checkout@v4

      - name: Use Node.js
        uses: actions/setup-node@v4
        with:
          node-version-file: '.nvmrc'

      - name: Install Packages
        runs: yarn i

      - name: Environment Variables
        uses: ./.github/actions/write-env-to-apps

      - name: Cache turbo build setup
        uses: actions/cache@v4
        with:
          path: .turbo
          key: ${{ runner.os }}-turbo-${{ github.sha }}
          restore-keys: |
            ${{ runner.os }}-turbo-

      - name: Build
        run: dotenv -- turbo run build --cache-dir=.turbo --env-mode=loose
petergrau commented 4 months ago

2.0.6 actually worked for us for about a week but now it hangs again. Anyone has a workaround of this or a proper fix?

chris-olszewski commented 4 months ago

If those encountering this issue could try running with -vvv on 2.0.10-canary.0 and provide the resulting logs that would be very helpful. I added many trace level logs around the area where the programs appear to be hanging.

jakubmazanec commented 4 months ago
2024-07-23T22:57:42.6095494Z ##[group]Run npm run test -- --filter=dram -vvv --env-mode=loose
2024-07-23T22:57:42.6096290Z npm run test -- --filter=dram -vvv --env-mode=loose
2024-07-23T22:57:42.6135936Z shell: /usr/bin/bash -e {0}
2024-07-23T22:57:42.6136254Z env:
2024-07-23T22:57:42.6136470Z   TURBO_UI: 0
2024-07-23T22:57:42.6136734Z   EDGEDB_INSTANCE: jakubmazanec/whisky
2024-07-23T22:57:42.6137083Z   EDGEDB_BRANCH: turborepo
2024-07-23T22:57:42.6139904Z   EDGEDB_SECRET_KEY: ***
2024-07-23T22:57:42.6140201Z ##[endgroup]
2024-07-23T22:57:42.7202938Z 
2024-07-23T22:57:42.7203427Z > test
2024-07-23T22:57:42.7204671Z > turbo run test --no-daemon --filter=dram -vvv --env-mode=loose
2024-07-23T22:57:42.7205365Z 
2024-07-23T22:57:42.7530413Z 2024-07-23T22:57:42.752+0000 [DEBUG] turborepo_lib::shim: Global turbo version: 2.0.10-canary.0
2024-07-23T22:57:42.7532399Z 2024-07-23T22:57:42.752+0000 [DEBUG] turborepo_repository::package_json: loading package.json from /home/runner/work/apps/apps/package.json
2024-07-23T22:57:42.7566554Z 2024-07-23T22:57:42.756+0000 [DEBUG] turborepo_repository::package_json: loading package.json from /home/runner/work/apps/package.json
2024-07-23T22:57:42.7568970Z 2024-07-23T22:57:42.756+0000 [DEBUG] turborepo_repository::package_json: loading package.json from /home/runner/work/package.json
2024-07-23T22:57:42.7571221Z 2024-07-23T22:57:42.756+0000 [DEBUG] turborepo_repository::package_json: loading package.json from /home/runner/package.json
2024-07-23T22:57:42.7573377Z 2024-07-23T22:57:42.756+0000 [DEBUG] turborepo_repository::package_json: loading package.json from /home/package.json
2024-07-23T22:57:42.7575223Z 2024-07-23T22:57:42.756+0000 [DEBUG] turborepo_repository::package_json: loading package.json from /package.json
2024-07-23T22:57:42.7576330Z 2024-07-23T22:57:42.756+0000 [DEBUG] turborepo_lib::shim: Repository Root: /home/runner/work/apps/apps
2024-07-23T22:57:42.7577751Z 2024-07-23T22:57:42.756+0000 [DEBUG] turborepo_repository::package_json: loading package.json from /home/runner/work/apps/apps/node_modules/turbo-linux-64/package.json
2024-07-23T22:57:42.7579414Z 2024-07-23T22:57:42.756+0000 [DEBUG] turborepo_lib::shim::local_turbo_state: Local turbo path: /home/runner/work/apps/apps/node_modules/turbo-linux-64/bin/turbo
2024-07-23T22:57:42.7580749Z 2024-07-23T22:57:42.756+0000 [DEBUG] turborepo_lib::shim::local_turbo_state: Local turbo version: 2.0.10-canary.0
2024-07-23T22:57:42.7581803Z 2024-07-23T22:57:42.756+0000 [DEBUG] turborepo_lib::shim: Currently running turbo is local turbo.
2024-07-23T22:57:42.7582869Z 2024-07-23T22:57:42.757+0000 [TRACE] log: registering event source with poller: token=Token(1), interests=READABLE
2024-07-23T22:57:42.7607742Z 2024-07-23T22:57:42.760+0000 [DEBUG] turborepo_telemetry::config: Telemetry config path: /home/runner/.config/turborepo/telemetry.json
2024-07-23T22:57:42.7656614Z 2024-07-23T22:57:42.765+0000 [DEBUG] turborepo_lib::process: spawning children with pty: false
2024-07-23T22:57:42.7659010Z 2024-07-23T22:57:42.765+0000 [TRACE] turborepo_lib::run::builder: performing run on "linux-64"
2024-07-23T22:57:42.7661880Z 2024-07-23T22:57:42.765+0000 [DEBUG] turborepo_repository::package_json: loading package.json from /home/runner/work/apps/apps/package.json
2024-07-23T22:57:42.7664831Z 2024-07-23T22:57:42.765+0000 [DEBUG] turborepo_telemetry::config: Telemetry config path: /home/runner/.config/turborepo/telemetry.json
2024-07-23T22:57:42.7667116Z 2024-07-23T22:57:42.765+0000 [DEBUG] turborepo_lib::run::builder: skipping turbod since --no-daemon was passed
2024-07-23T22:57:42.7668911Z 2024-07-23T22:57:42.766+0000 [DEBUG] turborepo_repository::discovery: discovering packages using caching strategy
2024-07-23T22:57:42.7671017Z 2024-07-23T22:57:42.766+0000 [DEBUG] turborepo_repository::discovery: discovering packages using primary strategy
2024-07-23T22:57:42.7673020Z 2024-07-23T22:57:42.766+0000 [DEBUG] turborepo_repository::discovery: discovering packages using local strategy
2024-07-23T22:57:42.7675450Z 2024-07-23T22:57:42.766+0000 [DEBUG] turborepo_telemetry: Starting telemetry event queue flush (num_events=10)
2024-07-23T22:57:42.7677118Z 2024-07-23T22:57:42.766+0000 [DEBUG] turborepo_telemetry: Done telemetry event queue flush
2024-07-23T22:57:42.7694144Z 2024-07-23T22:57:42.769+0000 [DEBUG] globwalk: processing includes: ["apps/*/package.json", "packages/*/package.json"]
2024-07-23T22:57:42.7695441Z 2024-07-23T22:57:42.769+0000 [DEBUG] globwalk: processing excludes: ["**/node_modules/**"]
2024-07-23T22:57:42.7697767Z 2024-07-23T22:57:42.769+0000 [DEBUG] globwalk: processed includes: ["/home/runner/work/apps/apps/apps/*/package.json", "/home/runner/work/apps/apps/packages/*/package.json"]
2024-07-23T22:57:42.7699877Z 2024-07-23T22:57:42.769+0000 [DEBUG] globwalk: processed excludes: ["/home/runner/work/apps/apps/**/node_modules/**"]
2024-07-23T22:57:42.7708301Z 2024-07-23T22:57:42.770+0000 [TRACE] log: registering event source with poller: token=Token(140667599760640), interests=READABLE | WRITABLE
2024-07-23T22:57:42.7732599Z 2024-07-23T22:57:42.773+0000 [DEBUG] log: No cached session for DnsName(DnsName(DnsName("telemetry.vercel.com")))
2024-07-23T22:57:42.7736566Z 2024-07-23T22:57:42.773+0000 [DEBUG] log: Not resuming any session
2024-07-23T22:57:42.7737521Z 2024-07-23T22:57:42.773+0000 [TRACE] log: Sending ClientHello Message {
2024-07-23T22:57:42.7738330Z     version: TLSv1_0,
2024-07-23T22:57:42.7738850Z     payload: Handshake {
2024-07-23T22:57:42.7739458Z         parsed: HandshakeMessagePayload {
2024-07-23T22:57:42.7740123Z             typ: ClientHello,
2024-07-23T22:57:42.7740666Z             payload: ClientHello(
2024-07-23T22:57:42.7741233Z                 ClientHelloPayload {
2024-07-23T22:57:42.7741850Z                     client_version: TLSv1_2,
2024-07-23T22:57:42.7742857Z                     random: a0b1505dd42711491ee0a3ebd061584b30e86a7f1a195f08b6e3ef00ee7c4f3a,
2024-07-23T22:57:42.7744208Z                     session_id: 2b1ebe10701dd5d9e5771f1c536dd47cc8fa6ee11cb6197e1c73856403a5f83c,
2024-07-23T22:57:42.7745213Z                     cipher_suites: [
2024-07-23T22:57:42.7745891Z                         TLS13_AES_256_GCM_SHA384,
2024-07-23T22:57:42.7746607Z                         TLS13_AES_128_GCM_SHA256,
2024-07-23T22:57:42.7747347Z                         TLS13_CHACHA20_POLY1305_SHA256,
2024-07-23T22:57:42.7748134Z                         TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,
2024-07-23T22:57:42.7748944Z                         TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,
2024-07-23T22:57:42.7749794Z                         TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256,
2024-07-23T22:57:42.7750627Z                         TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,
2024-07-23T22:57:42.7751427Z                         TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,
2024-07-23T22:57:42.7752277Z                         TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256,
2024-07-23T22:57:42.7753130Z                         TLS_EMPTY_RENEGOTIATION_INFO_SCSV,
2024-07-23T22:57:42.7753648Z                     ],
2024-07-23T22:57:42.7753969Z                     compression_methods: [
2024-07-23T22:57:42.7754449Z                         Null,
2024-07-23T22:57:42.7754740Z                     ],
2024-07-23T22:57:42.7755016Z                     extensions: [
2024-07-23T22:57:42.7755367Z                         SupportedVersions(
2024-07-23T22:57:42.7755695Z                             [
2024-07-23T22:57:42.7756002Z                                 TLSv1_3,
2024-07-23T22:57:42.7756349Z                                 TLSv1_2,
2024-07-23T22:57:42.7756667Z                             ],
2024-07-23T22:57:42.7756943Z                         ),
2024-07-23T22:57:42.7757420Z                         ECPointFormats(
2024-07-23T22:57:42.7757744Z                             [
2024-07-23T22:57:42.7758109Z                                 Uncompressed,
2024-07-23T22:57:42.7758453Z                             ],
2024-07-23T22:57:42.7758723Z                         ),
2024-07-23T22:57:42.7759009Z                         NamedGroups(
2024-07-23T22:57:42.7759570Z                             [
2024-07-23T22:57:42.7759892Z                                 X25519,
2024-07-23T22:57:42.7760767Z                                 secp256r1,
2024-07-23T22:57:42.7761147Z                                 secp384r1,
2024-07-23T22:57:42.7761486Z                             ],
2024-07-23T22:57:42.7761766Z                         ),
2024-07-23T22:57:42.7762086Z                         SignatureAlgorithms(
2024-07-23T22:57:42.7762434Z                             [
2024-07-23T22:57:42.7762775Z                                 ECDSA_NISTP384_SHA384,
2024-07-23T22:57:42.7763189Z                                 ECDSA_NISTP256_SHA256,
2024-07-23T22:57:42.7763565Z                                 ED25519,
2024-07-23T22:57:42.7763926Z                                 RSA_PSS_SHA512,
2024-07-23T22:57:42.7764307Z                                 RSA_PSS_SHA384,
2024-07-23T22:57:42.7764680Z                                 RSA_PSS_SHA256,
2024-07-23T22:57:42.7765061Z                                 RSA_PKCS1_SHA512,
2024-07-23T22:57:42.7765450Z                                 RSA_PKCS1_SHA384,
2024-07-23T22:57:42.7765825Z                                 RSA_PKCS1_SHA256,
2024-07-23T22:57:42.7766171Z                             ],
2024-07-23T22:57:42.7766449Z                         ),
2024-07-23T22:57:42.7766785Z                         ExtendedMasterSecretRequest,
2024-07-23T22:57:42.7767197Z                         CertificateStatusRequest(
2024-07-23T22:57:42.7767567Z                             OCSP(
2024-07-23T22:57:42.7767950Z                                 OCSPCertificateStatusRequest {
2024-07-23T22:57:42.7768393Z                                     responder_ids: [],
2024-07-23T22:57:42.7768789Z                                     extensions: ,
2024-07-23T22:57:42.7769143Z                                 },
2024-07-23T22:57:42.7769449Z                             ),
2024-07-23T22:57:42.7769724Z                         ),
2024-07-23T22:57:42.7770007Z                         ServerName(
2024-07-23T22:57:42.7770313Z                             [
2024-07-23T22:57:42.7770636Z                                 ServerName {
2024-07-23T22:57:42.7771036Z                                     typ: HostName,
2024-07-23T22:57:42.7771464Z                                     payload: HostName(
2024-07-23T22:57:42.7771854Z                                         (
2024-07-23T22:57:42.7772275Z                                             74656c656d657472792e76657263656c2e636f6d,
2024-07-23T22:57:42.7772724Z                                             DnsName(
2024-07-23T22:57:42.7773164Z                                                 "telemetry.vercel.com",
2024-07-23T22:57:42.7773580Z                                             ),
2024-07-23T22:57:42.7773940Z                                         ),
2024-07-23T22:57:42.7774275Z                                     ),
2024-07-23T22:57:42.7774599Z                                 },
2024-07-23T22:57:42.7774910Z                             ],
2024-07-23T22:57:42.7775191Z                         ),
2024-07-23T22:57:42.7775549Z                         SignedCertificateTimestampRequest,
2024-07-23T22:57:42.7775954Z                         KeyShare(
2024-07-23T22:57:42.7776245Z                             [
2024-07-23T22:57:42.7776579Z                                 KeyShareEntry {
2024-07-23T22:57:42.7776966Z                                     group: X25519,
2024-07-23T22:57:42.7777541Z                                     payload: 25a0637cd29cae2f5a2b37906e9eb677785023e467809ccf3bff133404d8b33e,
2024-07-23T22:57:42.7778063Z                                 },
2024-07-23T22:57:42.7778371Z                             ],
2024-07-23T22:57:42.7778649Z                         ),
2024-07-23T22:57:42.7778956Z                         PresharedKeyModes(
2024-07-23T22:57:42.7779290Z                             [
2024-07-23T22:57:42.7779604Z                                 PSK_DHE_KE,
2024-07-23T22:57:42.7779952Z                             ],
2024-07-23T22:57:42.7780226Z                         ),
2024-07-23T22:57:42.7780659Z                         Protocols(
2024-07-23T22:57:42.7781086Z                             [
2024-07-23T22:57:42.7781393Z                                 6832,
2024-07-23T22:57:42.7781754Z                                 687474702f312e31,
2024-07-23T22:57:42.7782105Z                             ],
2024-07-23T22:57:42.7782384Z                         ),
2024-07-23T22:57:42.7782677Z                         SessionTicket(
2024-07-23T22:57:42.7783012Z                             Request,
2024-07-23T22:57:42.7783308Z                         ),
2024-07-23T22:57:42.7783566Z                     ],
2024-07-23T22:57:42.7783819Z                 },
2024-07-23T22:57:42.7784052Z             ),
2024-07-23T22:57:42.7784276Z         },
2024-07-23T22:57:42.7787306Z         encoded: 010001040303a0b1505dd42711491ee0a3ebd061584b30e86a7f1a195f08b6e3ef00ee7c4f3a202b1ebe10701dd5d9e5771f1c536dd47cc8fa6ee11cb6197e1c73856403a5f83c0014130213011303c02cc02bcca9c030c02fcca800ff010000a7002b00050403040303000b00020100000a00080006001d00170018000d001400120503040308070806080508040601050104010017000000050005010000000000000019001700001474656c656d657472792e76657263656c2e636f6d00120000003300260024001d002025a0637cd29cae2f5a2b37906e9eb677785023e467809ccf3bff133404d8b33e002d000201010010000e000c02683208687474702f312e3100230000,
2024-07-23T22:57:42.7790212Z     },
2024-07-23T22:57:42.7790431Z }
2024-07-23T22:57:42.7791485Z 2024-07-23T22:57:42.775+0000 [DEBUG] turborepo_repository::package_json: loading package.json from /home/runner/work/apps/apps/apps/dram/package.json
2024-07-23T22:57:42.7793410Z 2024-07-23T22:57:42.775+0000 [DEBUG] turborepo_repository::package_json: loading package.json from /home/runner/work/apps/apps/apps/somewhere/package.json
2024-07-23T22:57:42.7794781Z 2024-07-23T22:57:42.776+0000 [DEBUG] turborepo_repository::discovery: discovering packages using caching strategy
2024-07-23T22:57:42.7796827Z 2024-07-23T22:57:42.776+0000 [DEBUG] turborepo_repository::discovery: discovering packages using caching strategy
2024-07-23T22:57:42.7798533Z 2024-07-23T22:57:42.779+0000 [TRACE] log: We got ServerHello ServerHelloPayload {
2024-07-23T22:57:42.7799525Z     legacy_version: TLSv1_2,
2024-07-23T22:57:42.7800683Z     random: 28259505089fa7462de3aab6d99215f20c176f387ea5709ca3844b081e842572,
2024-07-23T22:57:42.7801931Z     session_id: 2b1ebe10701dd5d9e5771f1c536dd47cc8fa6ee11cb6197e1c73856403a5f83c,
2024-07-23T22:57:42.7802939Z     cipher_suite: TLS13_AES_128_GCM_SHA256,
2024-07-23T22:57:42.7803563Z     compression_method: Null,
2024-07-23T22:57:42.7804070Z     extensions: [
2024-07-23T22:57:42.7804517Z         SupportedVersions(
2024-07-23T22:57:42.7805021Z             TLSv1_3,
2024-07-23T22:57:42.7805451Z         ),
2024-07-23T22:57:42.7805832Z         KeyShare(
2024-07-23T22:57:42.7806257Z             KeyShareEntry {
2024-07-23T22:57:42.7806790Z                 group: X25519,
2024-07-23T22:57:42.7807709Z                 payload: b3109a12098e020ec804a3a46451ce62809ac07f06e8b8316f002620012a9e03,
2024-07-23T22:57:42.7808589Z             },
2024-07-23T22:57:42.7808970Z         ),
2024-07-23T22:57:42.7809324Z     ],
2024-07-23T22:57:42.7809672Z }
2024-07-23T22:57:42.7810689Z 2024-07-23T22:57:42.779+0000 [DEBUG] log: Using ciphersuite TLS13_AES_128_GCM_SHA256
2024-07-23T22:57:42.7811873Z 2024-07-23T22:57:42.779+0000 [DEBUG] log: Not resuming
2024-07-23T22:57:42.7812931Z 2024-07-23T22:57:42.779+0000 [TRACE] log: EarlyData rejected
2024-07-23T22:57:42.7813986Z 2024-07-23T22:57:42.779+0000 [TRACE] log: Dropping CCS
2024-07-23T22:57:42.7815355Z 2024-07-23T22:57:42.779+0000 [DEBUG] log: TLS1.3 encrypted extensions: [Protocols([6832])]
2024-07-23T22:57:42.7816831Z 2024-07-23T22:57:42.779+0000 [DEBUG] log: ALPN protocol is Some(b"h2")
2024-07-23T22:57:42.7896717Z 2024-07-23T22:57:42.779+0000 [TRACE] log: Server cert is [Certificate(b"0\x82\x04\xf30\x82\x03\xdb\xa0\x03\x02\x01\x02\x02\x12\x04e\x13y\x0b\xc4:a\x10\x91.\x1d\xee\xbb\x98\x9d\xaa\x8a0\r\x06\t*\x86H\x86\xf7\r\x01\x01\x0b\x05\0031\x0b0\t\x06\x03U\x04\x06\x13\x02US1\x160\x14\x06\x03U\x04\n\x13\rLet's Encrypt1\x0c0\n\x06\x03U\x04\x03\x13\x03R110\x1e\x17\r240703074859Z\x17\r241001074858Z0\x171\x150\x13\x06\x03U\x04\x03\x0c\x0c*.vercel.com0\x82\x01\"0\r\x06\t*\x86H\x86\xf7\r\x01\x01\x01\x05\0\x03\x82\x01\x0f\00\x82\x01\n\x02\x82\x01\x01\0\xedO\x0e\\-\xeeX\x05\x9d\xb3;\x93H9\x0fa\xa0m\\0$\xb3\xb1\xe9\xde\xe1|\xb00()\xa7H\x8e7\xe0\x89l\xe6^T\xd5\xd8$\x1d\xd0\xb3^\x84\x84\xfa\x9aE \xf7\xf6Y\x94\xa2 \xf8\xdbG\x9bNO\xb7\xb0B\xb6\x83\x0c\xac\xa9\xa2\xbc\x1f\xd7\x85|4\xd2n\xda'\x8e\xd3\x80\x88\xc9\xd0\xd8\xcct\x9b\x96\x1b\xec&\xf9\x90\xeb\x07/\x85>\xd1\xa8\x92\xf4\"\xdewN|*\xdc\x89\xa0\xd8K\x9d\x0e_+dGg~JT\xe3\x997\xee$\xd3\xfc\xaa\xe1\xb4\xb6\x01\xa6\"U?\xd7/h{*&\xa2\x8b#R:|\x1e\x108\xf7\xb7\x15\xe1\xb6\x11T\x07\xb2_\xb4\xa7\xd6\x98k\xc7H\xfa\t\xfb\x19O\xfa,\x1b\xb9\xf5\x0e\xc5\xaej\xc9\t\xa8\xaf8C\xa3f\xd4x\xb5O\x91|\xda\xa4\xad\xab\xd1\xd9K\x96\xd3\xbc\xe7\xe7\xe5\xfc@x\xa5}\xbd<\x82o\xb4%\x80\x06\xafb\xa6\xc6A\xf2\xd2\xa6F\xd6'\xb9?6\x1b\x17\xc4\x08\xed}q?\x95\x02\x03\x01\0\x01\xa3\x82\x02\x1b0\x82\x02\x170\x0e\x06\x03U\x1d\x0f\x01\x01\xff\x04\x04\x03\x02\x05\xa00\x1d\x06\x03U\x1d%\x04\x160\x14\x06\x08+\x06\x01\x05\x05\x07\x03\x01\x06\x08+\x06\x01\x05\x05\x07\x03\x020\x0c\x06\x03U\x1d\x13\x01\x01\xff\x04\x020\00\x1d\x06\x03U\x1d\x0e\x04\x16\x04\x14\x06\x13T\xb3qowY\x8a\x89\xfa-\x1eC\x93\xa3?\xc6Y\xec0\x1f\x06\x03U\x1d#\x04\x180\x16\x80\x14\xc5\xcfF\xa4\xea\xf4\xc3\xc0zl\x95\xc4-\xb0^\x92/&\xe3\xb90W\x06\x08+\x06\x01\x05\x05\x07\x01\x01\x04K0I0\"\x06\x08+\x06\x01\x05\x05\x070\x01\x86\x16http://r11.o.lencr.org0#\x06\x08+\x06\x01\x05\x05\x070\x02\x86\x17http://r11.i.lencr.org/0#\x06\x03U\x1d\x11\x04\x1c0\x1a\x82\x0c*.vercel.com\x82\nvercel.com0\x13\x06\x03U\x1d \x04\x0c0\n0\x08\x06\x06g\x81\x0c\x01\x02\x010\x82\x01\x03\x06\n+\x06\x01\x04\x01\xd6y\x02\x04\x02\x04\x81\xf4\x04\x81\xf1\0\xef\0v\0?\x17KO\xd7\"GX\x94\x1de\x1c\x84\xbe\r\x12\xed\x907\x7f\x1f\x85j\xeb\xc1\xbf(\x85\xec\xf8dn\0\0\x01\x90w\xc8bF\0\0\x04\x03\0G0E\x02 |\x83\n\xc2/\xbank\xce\x08\xc07\xf3Y\xf7\x12\xe2\xcb\xb4\x03Y\x14\x07\x18\x93\x8d\x90\x87u\xecO\xa8\x02!\0\xe7&^\x87\x83z\xd8\xc8\x11\xdaz\xa6\xda\xe7)\x89\xf6\x1d\xcd\xf5\xe1B\xcf\x10\\\xbd5\xb2\xab\x8d\x8aA\0u\0\x19\x98\x10q\t\xf0\xd6R.0\x80\xd2\x9e?d\xbb\x83n(\xcc\xf9\x0fR\x8e\xee\xdf\xceJ?\x16\xb4\xca\0\0\x01\x90w\xc8b\xed\0\0\x04\x03\0F0D\x02 +\xbf\xd0\x1f\x82qC\t\x10q\x1d\x8a\xcd\xec\xd3\xcd\r\xbd9D|qn\xc1\x1e\xd2\xcf&\xe8\xd4\x97\xfa\x02 {w\xcdg\xe4\x9a\xef\xf7*W\x1e)\x95#I\xf1\xd8h\x96\xb7\xda\xe8\x91d\x12\x1f\xdb\xd9AP)T0\r\x06\t*\x86H\x86\xf7\r\x01\x01\x0b\x05\0\x03\x82\x01\x01\0\x88Y=lj\x97\x7fa\xfb\x87\xa7\x80\x0eb\x0c\xd3\xb0\xf2\xed\xe4)L\xe8\xad\xf7\xcc5\xc0\xb4DU\xef^\x83\x05\xc0\xed\x01\xab\xa1}\xd5\xc9\x19\xc2a\xf3\xa1O&\xeb\xae\x8a\xacW\xb3R\xebMx\xb1\x84.\x8c\xe12Vm!\xc5:\xb1\xfa\xb2\xf7\xd7el\xa5/S2\xcb\xe7*\xa8;\xf7\xe3 \xdf,\n\xf8Z>\x99O\xa8\xd8\x7f\x95\x89\x0e\x8b~h:U\xa9o\x12pG\xcf\x12D9O\xfe\xfbg\xb1\xd4e\xbb\x1d\xea\xee\x13\xf1Un{\xbe\xe4\x15\xd2G\xe47\xadP\xb3\xfe\xe9L3ym\x92y8\xcb\x12\xb9\xb6A)@<0\xd5\x01\x9c\xc9\x1b\x867\x16\xb3\x95]\x0e\x13\x0b\xd6\x84\xa0]\no\x85\x18M\xed\xdf\x06\x1fM\x0b\xefj\xea\xf7e\xa0\n\xed\xd6\x0e\xbe\xc5\xc6\x17\x92\xc7z\xc5*\xf3\xd2\xb6);\xa2\xb2*\xcb\x10n\xa1\x07\xd0\x18\xd0v\x9cC\x9b\xc4\xd3\x1a\xd9S'\xd4\xf9q\xe4\xb8\xb85.l`\xb4hYi&\xe2B\xfdU\xb7"), Certificate(b"0\x82\x05\x060\x82\x02\xee\xa0\x03\x02\x01\x02\x02\x11\0\x8a}>\x13\xd6/0\xef#\x86\xbd)\x07k4\xf80\r\x06\t*\x86H\x86\xf7\r\x01\x01\x0b\x05\00O1\x0b0\t\x06\x03U\x04\x06\x13\x02US1)0'\x06\x03U\x04\n\x13 Internet Security Research Group1\x150\x13\x06\x03U\x04\x03\x13\x0cISRG Root X10\x1e\x17\r240313000000Z\x17\r270312235959Z031\x0b0\t\x06\x03U\x04\x06\x13\x02US1\x160\x14\x06\x03U\x04\n\x13\rLet's Encrypt1\x0c0\n\x06\x03U\x04\x03\x13\x03R110\x82\x01\"0\r\x06\t*\x86H\x86\xf7\r\x01\x01\x01\x05\0\x03\x82\x01\x0f\00\x82\x01\n\x02\x82\x01\x01\0\xba\x87\xbc\\\x1b\09\xcb\xca\n\xcd\xd4g\x10\xf9\x01<\xa5N\xa5a\xcb&\xcaR\xfb\x15\x01\xb7\xb9(\xf5(\x1e\xed'\xb3$\x189g\t\x0c\x08\xec\xe0:\xb0;w\x0e\xbd\xf3\xe59TA\x0cN\xaeA\xd6\x99t\xdeQ\xdb\xef{\xffX\xbd\xa8\xb7\x13\xf6\xde1\xd5\xf2r\xc9rj\x0b\x83t\x95\x9cF\0d\x14\x99\xf3\xb1\xd9\"\xd9\xcd\xa8\x92\xaa\x1c&z?\xfe\xefX\x05{\x08\x95\x81\xdbq\x0f\x8e\xfb\xe31\t\xbb\t\xbePM_\x8f\x91v=Z\x9d\x9e\x83\xf2\xe9\xc4f\xb3\xe1\x06fCH\x18\x80e\xa07\x18\x9a\x9b\x842\x97\xb1\xb2\xbd\xc4\xf8\x15\0\x9d'\x88\xfb\xe2c\x17\x96l\x9b'gK\xc4\xdb(^i\xc2y\xf0I\\\xe0$P\xe1\xc4\xbc\xa1\x05\xac{@m\0\xb4\xc2A?\xa7X\xb8/\xc5\\\x9b\xa5\xbb\t\x9e\xf1\xfe\xeb\xb0\x859\xfd\xa8\n\xefE\xc4x\xebe*\xc2\xcf_<\xde\xe3\\M\x1b\xf7\x0b'+\xaa\x0bBwSOyj\x1d\x87\xd9\x02\x03\x01\0\x01\xa3\x81\xf80\x81\xf50\x0e\x06\x03U\x1d\x0f\x01\x01\xff\x04\x04\x03\x02\x01\x860\x1d\x06\x03U\x1d%\x04\x160\x14\x06\x08+\x06\x01\x05\x05\x07\x03\x02\x06\x08+\x06\x01\x05\x05\x07\x03\x010\x12\x06\x03U\x1d\x13\x01\x01\xff\x04\x080\x06\x01\x01\xff\x02\x01\00\x1d\x06\x03U\x1d\x0e\x04\x16\x04\x14\xc5\xcfF\xa4\xea\xf4\xc3\xc0zl\x95\xc4-\xb0^\x92/&\xe3\xb90\x1f\x06\x03U\x1d#\x04\x180\x16\x80\x14y\xb4Y\xe6{\xb6\xe5\xe4\x01s\x80\x08\x88\xc8\x1aX\xf6\xe9\x9bn02\x06\x08+\x06\x01\x05\x05\x07\x01\x01\x04&0$0\"\x06\x08+\x06\x01\x05\x05\x070\x02\x86\x16http://x1.i.lencr.org/0\x13\x06\x03U\x1d \x04\x0c0\n0\x08\x06\x06g\x81\x0c\x01\x02\x010'\x06\x03U\x1d\x1f\x04 0\x1e0\x1c\xa0\x1a\xa0\x18\x86\x16http://x1.c.lencr.org/0\r\x06\t*\x86H\x86\xf7\r\x01\x01\x0b\x05\0\x03\x82\x02\x01\0N\xe2\x89]\n\x03\x1c\x908\xd0\xf5\x1f\xf9q\\\xf8\xc3\x8f\xb27\x88zo\xb0%\x1f\xed\xbe\xb7\xd8\x86\x06\x8e\xe9\t\x84\xcdr\xbf\x81\xf3\xfc\xca\xcfSH\xed\xbd\xf6iB\xd4\xa5\x11>5\xc8\x13\xb2\x92\x1d\x05_\xea.\xd4\xd8\xf8I\xc3\xad\xf5\x99\x96\x9c\xef&\xd8\xe1\xb4$\x0bH M\xfc\xd3T\xb4\xa9\xc6!\xc8\xe16\x1b\xffwd)\x17\xb9\xf0K\xef]\xea\xcdy\xd0\xbf\x90\xbf\xbe#\xb2\x90\xdaJ\xa9H1t\xa9D\x0b\xe1\xe2\xf6-\x83q\xa4u{\xd2\x94\xc1\x05\x19F\x1c\xb9\x8f\xf3\xc4tH%*\r\xe5\xf5\xdbC\xe2\xdb\x93\x9b\xb9\x19\xb4\x1f/\xdfj\x0e\x8f1\xd3c\x0f\xbb)\xdc\xddf,?\xb0\x1bgQ\xf8A<\xe4M\xb9\xac\xb8\xa4\x9cfc\xf5\xab\x85#\x1d\xccS\xb6\xabq\xae\xdc\xc5\x01q\xda6\xee\n\x18*2\xfd\t1|\x8f\xf6s\xe7\x9c\x9c\xb5J\x15jw\x82Z\xcf\xda\x8dE\xfe\x1f*d\x050>s\xc2\xc6\x0c\xb9\xd6;cJ\xabF\x03\xfe\x99\xc0F@'`c\xdfP:\x07G\xd8\x15J\x9f\xeaG\x1f\x99Z\x08b\x0c\xb6l3\x08M\xd78\xedH-.\x05h\xae\x80]\xefL\xdc\xd8 A_h\xf1\xbbZ\xcd\xe3\x0e\xb0\x0c1\x87\x9bC\xdeIC\xe1\xc8\x04?\xd1<\x1b\x87E0i\xa8\xa9r\x0ey\x12\x1c1\xd8>#W\xdd\xa7O\xa0\xf0\x1c\x81\xd1w\x1fo\xd6\xd2\xb9\xa8\xb3\x03\x16\x819K\x9fU\xae\xd2j\xe4\xb3\xbf\xea\xa5\xd5\x9fK\xa3\xc9\xd6;r\xf3J\xf6T\xab\x0c\xfc8\xf7`\x80\xdfn5\xcau\xa1T\xe4/\xbcn\x17\xc9\x1a\xa57\xb5\xa2\x9a\xba\xec\xf4\xc0uFOw\xa8\xe8YV\x91f-n\xde)\x81\xd6\xa6\x97\x05^dE\xbe,\xce\xeadBD\xb0\xc3O\xad\xf0\xb4\xdc\x03\xca\x99\x9b\t\x82\x95\x82\rc\x8af\xf9\x19r\xf8\xd5\xb9\x89\x10\xe2\x89\x98\t5\xf9\xa2\x1c\xbe\x92s#t\xe9\x9d\x1f\xd7;J\x9a\x84X\x10\xc2\xf3\xa7\xe25\xec~;E\xce0FRk\xc0\xc0")]
2024-07-23T22:57:42.7970935Z 2024-07-23T22:57:42.779+0000 [TRACE] log: Unvalidated OCSP response: [48, 130, 1, 244, 10, 1, 0, 160, 130, 1, 237, 48, 130, 1, 233, 6, 9, 43, 6, 1, 5, 5, 7, 48, 1, 1, 4, 130, 1, 218, 48, 130, 1, 214, 48, 129, 191, 161, 53, 48, 51, 49, 11, 48, 9, 6, 3, 85, 4, 6, 19, 2, 85, 83, 49, 22, 48, 20, 6, 3, 85, 4, 10, 19, 13, 76, 101, 116, 39, 115, 32, 69, 110, 99, 114, 121, 112, 116, 49, 12, 48, 10, 6, 3, 85, 4, 3, 19, 3, 82, 49, 49, 24, 15, 50, 48, 50, 52, 48, 55, 50, 51, 48, 57, 52, 55, 48, 48, 90, 48, 117, 48, 115, 48, 75, 48, 9, 6, 5, 43, 14, 3, 2, 26, 5, 0, 4, 20, 26, 82, 185, 180, 89, 228, 195, 51, 152, 33, 126, 134, 109, 75, 189, 138, 59, 214, 103, 206, 4, 20, 8, 185, 17, 59, 165, 208, 133, 24, 180, 234, 15, 160, 173, 159, 134, 30, 142, 252, 56, 39, 2, 18, 4, 101, 19, 121, 11, 196, 58, 97, 16, 145, 46, 29, 238, 187, 152, 157, 170, 138, 128, 0, 24, 15, 50, 48, 50, 52, 48, 55, 50, 51, 48, 57, 52, 55, 48, 48, 90, 160, 17, 24, 15, 50, 48, 50, 52, 48, 55, 51, 48, 48, 57, 52, 54, 53, 56, 90, 48, 13, 6, 9, 42, 134, 72, 134, 247, 13, 1, 1, 11, 5, 0, 3, 130, 1, 1, 0, 65, 186, 230, 108, 104, 132, 163, 24, 121, 5, 2, 158, 12, 187, 50, 28, 77, 17, 51, 179, 93, 105, 51, 82, 82, 206, 62, 0, 222, 211, 84, 174, 59, 220, 237, 18, 190, 35, 36, 35, 70, 86, 86, 190, 68, 128, 65, 221, 229, 118, 64, 33, 239, 226, 161, 113, 56, 252, 233, 76, 20, 186, 12, 5, 210, 197, 87, 96, 170, 78, 57, 218, 130, 190, 19, 42, 227, 39, 108, 69, 239, 51, 2, 103, 0, 92, 237, 80, 139, 42, 155, 13, 44, 44, 190, 143, 1, 94, 91, 207, 227, 61, 188, 1, 211, 28, 167, 168, 246, 201, 90, 121, 183, 209, 108, 87, 227, 88, 82, 246, 73, 131, 157, 88, 179, 204, 203, 220, 68, 90, 81, 134, 13, 222, 98, 249, 101, 130, 64, 184, 192, 62, 170, 192, 250, 204, 57, 93, 21, 226, 17, 36, 161, 105, 158, 144, 241, 249, 53, 147, 51, 52, 124, 38, 201, 61, 34, 124, 240, 31, 231, 213, 110, 30, 142, 161, 175, 13, 65, 22, 241, 127, 174, 82, 33, 164, 99, 99, 181, 58, 237, 134, 137, 30, 81, 106, 129, 34, 181, 98, 189, 215, 35, 236, 106, 215, 86, 206, 241, 11, 201, 181, 90, 172, 250, 235, 71, 187, 231, 242, 215, 158, 45, 53, 32, 215, 15, 86, 225, 24, 189, 166, 67, 115, 124, 242, 43, 182, 241, 152, 169, 104, 222, 197, 151, 176, 243, 89, 22, 215, 173, 32, 109, 177, 200, 185]
2024-07-23T22:57:42.7982466Z 2024-07-23T22:57:42.780+0000 [DEBUG] log: Ticket saved
2024-07-23T22:57:42.7983468Z 2024-07-23T22:57:42.780+0000 [TRACE] log: signal: Want
2024-07-23T22:57:42.7984665Z 2024-07-23T22:57:42.780+0000 [TRACE] log: signal found waiting giver, notifying
2024-07-23T22:57:42.7985977Z 2024-07-23T22:57:42.780+0000 [TRACE] log: poll_want: taker wants!
2024-07-23T22:57:42.7987048Z 2024-07-23T22:57:42.780+0000 [TRACE] log: signal: Want
2024-07-23T22:57:42.7987989Z 2024-07-23T22:57:42.780+0000 [TRACE] log: signal: Want
2024-07-23T22:57:42.7989479Z 2024-07-23T22:57:42.783+0000 [TRACE] tokio_util::codec::framed_impl: attempting to decode a frame
2024-07-23T22:57:42.7991208Z 2024-07-23T22:57:42.783+0000 [TRACE] tokio_util::codec::framed_impl: frame decoded from buffer
2024-07-23T22:57:42.7992927Z 2024-07-23T22:57:42.783+0000 [TRACE] tokio_util::codec::framed_impl: attempting to decode a frame
2024-07-23T22:57:42.7994621Z 2024-07-23T22:57:42.783+0000 [TRACE] tokio_util::codec::framed_impl: attempting to decode a frame
2024-07-23T22:57:42.7996292Z 2024-07-23T22:57:42.783+0000 [TRACE] tokio_util::codec::framed_impl: frame decoded from buffer
2024-07-23T22:57:42.7997950Z 2024-07-23T22:57:42.783+0000 [TRACE] tokio_util::codec::framed_impl: attempting to decode a frame
2024-07-23T22:57:42.7999643Z 2024-07-23T22:57:42.783+0000 [TRACE] tokio_util::codec::framed_impl: attempting to decode a frame
2024-07-23T22:57:42.8001646Z 2024-07-23T22:57:42.783+0000 [TRACE] tokio_util::codec::framed_impl: frame decoded from buffer
2024-07-23T22:57:42.8003331Z 2024-07-23T22:57:42.783+0000 [TRACE] tokio_util::codec::framed_impl: attempting to decode a frame
2024-07-23T22:57:42.8149136Z 2024-07-23T22:57:42.813+0000 [DEBUG] turborepo_repository::discovery: discovering packages using caching strategy
2024-07-23T22:57:42.8151446Z 2024-07-23T22:57:42.813+0000 [DEBUG] turborepo_lib::run::task_access: TASK ACCESS TRACE: unable to restore config from cache
2024-07-23T22:57:42.8163122Z • Packages in scope: dram
2024-07-23T22:57:42.8164407Z 2024-07-23T22:57:42.816+0000 [TRACE] turborepo_lib::task_hash: scm running in git mode
2024-07-23T22:57:42.8165504Z • Running test in 1 packages
2024-07-23T22:57:42.8166122Z • Remote caching disabled
2024-07-23T22:57:42.8167867Z 2024-07-23T22:57:42.816+0000 [DEBUG] turborepo_telemetry::config: Telemetry config path: /home/runner/.config/turborepo/telemetry.json
2024-07-23T22:57:42.8170291Z 2024-07-23T22:57:42.816+0000 [DEBUG] turborepo_telemetry::config: Telemetry config path: /home/runner/.config/turborepo/telemetry.json
2024-07-23T22:57:42.8172839Z 2024-07-23T22:57:42.816+0000 [DEBUG] turborepo_telemetry::config: Telemetry config path: /home/runner/.config/turborepo/telemetry.json
2024-07-23T22:57:42.8177327Z 2024-07-23T22:57:42.816+0000 [DEBUG] turborepo_telemetry::config: Telemetry config path: /home/runner/.config/turborepo/telemetry.json
2024-07-23T22:57:42.8179764Z 2024-07-23T22:57:42.816+0000 [DEBUG] turborepo_telemetry::config: Telemetry config path: /home/runner/.config/turborepo/telemetry.json
2024-07-23T22:57:42.8184584Z 2024-07-23T22:57:42.816+0000 [DEBUG] turborepo_telemetry::config: Telemetry config path: /home/runner/.config/turborepo/telemetry.json
2024-07-23T22:57:42.8190590Z 2024-07-23T22:57:42.818+0000 [DEBUG] turborepo_telemetry::config: Telemetry config path: /home/runner/.config/turborepo/telemetry.json
2024-07-23T22:57:42.8192785Z 2024-07-23T22:57:42.818+0000 [DEBUG] turborepo_telemetry: Starting telemetry event queue flush (num_events=10)
2024-07-23T22:57:42.8194485Z 2024-07-23T22:57:42.818+0000 [DEBUG] turborepo_telemetry: Done telemetry event queue flush
2024-07-23T22:57:42.8201417Z 2024-07-23T22:57:42.819+0000 [TRACE] log: signal: Want
2024-07-23T22:57:42.8205563Z 2024-07-23T22:57:42.817+0000 [DEBUG] turborepo_telemetry::config: Telemetry config path: /home/runner/.config/turborepo/telemetry.json
2024-07-23T22:57:42.8208150Z 2024-07-23T22:57:42.820+0000 [DEBUG] turborepo_telemetry::config: Telemetry config path: /home/runner/.config/turborepo/telemetry.json
2024-07-23T22:57:42.8211046Z 2024-07-23T22:57:42.820+0000 [TRACE] log: signal: Want
2024-07-23T22:57:42.8212916Z 2024-07-23T22:57:42.820+0000 [DEBUG] turborepo_telemetry::config: Telemetry config path: /home/runner/.config/turborepo/telemetry.json
2024-07-23T22:57:42.8322066Z 2024-07-23T22:57:42.832+0000 [DEBUG] turborepo_lib::run::global_hash: global hash env vars ["CI", "EDGEDB_BRANCH", "EDGEDB_INSTANCE", "EDGEDB_SECRET_KEY"]
2024-07-23T22:57:42.8339291Z 2024-07-23T22:57:42.833+0000 [DEBUG] globwalk: processing includes: [".editorconfig", ".eslintrc.cjs", ".prettierignore", "prettier.config.js", "tsconfig.json", "tsconfig.typecheck.json"]
2024-07-23T22:57:42.8342136Z 2024-07-23T22:57:42.833+0000 [DEBUG] globwalk: processing excludes: ["**/node_modules/**"]
2024-07-23T22:57:42.8344520Z 2024-07-23T22:57:42.833+0000 [DEBUG] globwalk: '/home/runner/work/apps/apps/prettier.config.js' doesn't have metadata
2024-07-23T22:57:42.8349144Z 2024-07-23T22:57:42.833+0000 [DEBUG] globwalk: processed includes: ["/home/runner/work/apps/apps/.editorconfig", "/home/runner/work/apps/apps/.eslintrc.cjs", "/home/runner/work/apps/apps/.prettierignore", "/home/runner/work/apps/apps/prettier.config.js", "/home/runner/work/apps/apps/tsconfig.json", "/home/runner/work/apps/apps/tsconfig.typecheck.json"]
2024-07-23T22:57:42.8351962Z 2024-07-23T22:57:42.833+0000 [DEBUG] globwalk: processed excludes: ["/home/runner/work/apps/apps/**/node_modules/**"]
2024-07-23T22:57:42.8407361Z 2024-07-23T22:57:42.840+0000 [DEBUG] turborepo_lib::run::global_hash: external deps hash: c8d5aac9959edec9
2024-07-23T22:57:42.8408971Z 2024-07-23T22:57:42.840+0000 [DEBUG] turborepo_lib::run: running visitor
2024-07-23T22:57:42.8410899Z 2024-07-23T22:57:42.840+0000 [DEBUG] turborepo_telemetry::config: Telemetry config path: /home/runner/.config/turborepo/telemetry.json
2024-07-23T22:57:42.8412941Z 2024-07-23T22:57:42.840+0000 [DEBUG] turborepo_telemetry::config: Telemetry config path: /home/runner/.config/turborepo/telemetry.json
2024-07-23T22:57:42.8414351Z 2024-07-23T22:57:42.840+0000 [DEBUG] turborepo_telemetry::config: Telemetry config path: /home/runner/.config/turborepo/telemetry.json
2024-07-23T22:57:42.8415693Z 2024-07-23T22:57:42.840+0000 [DEBUG] turborepo_telemetry::config: Telemetry config path: /home/runner/.config/turborepo/telemetry.json
2024-07-23T22:57:42.8416814Z 2024-07-23T22:57:42.840+0000 [DEBUG] turborepo_lib::task_hash: auto detected framework for dram
2024-07-23T22:57:42.8417805Z 2024-07-23T22:57:42.840+0000 [DEBUG] turborepo_lib::task_hash: framework: vite, env_prefix: ["VITE_*"]
2024-07-23T22:57:42.8422075Z 2024-07-23T22:57:42.842+0000 [DEBUG] turborepo_lib::task_hash: task hash env vars for dram:codegen
2024-07-23T22:57:42.8423053Z  vars: []
2024-07-23T22:57:42.8424373Z 2024-07-23T22:57:42.842+0000 [DEBUG] turborepo_lib::task_graph::visitor: task dram#codegen hash is a29164775e0e0a4c
2024-07-23T22:57:42.8426589Z 2024-07-23T22:57:42.842+0000 [DEBUG] turborepo_telemetry::config: Telemetry config path: /home/runner/.config/turborepo/telemetry.json
2024-07-23T22:57:42.8429058Z 2024-07-23T22:57:42.842+0000 [DEBUG] turborepo_telemetry::config: Telemetry config path: /home/runner/.config/turborepo/telemetry.json
2024-07-23T22:57:42.8430816Z 2024-07-23T22:57:42.842+0000 [TRACE] turborepo_lib::process: acquiring lock for spawning (/home/runner/work/apps/apps/apps/dram) /opt/hostedtoolcache/node/20.15.1/x64/bin/npm run codegen
2024-07-23T22:57:42.8432886Z 2024-07-23T22:57:42.842+0000 [TRACE] turborepo_lib::process: acquired lock for spawning (/home/runner/work/apps/apps/apps/dram) /opt/hostedtoolcache/node/20.15.1/x64/bin/npm run codegen
2024-07-23T22:57:42.8437072Z 2024-07-23T22:57:42.843+0000 [TRACE] log: registering event source with poller: token=Token(140667585312384), interests=READABLE | WRITABLE
2024-07-23T22:57:42.8439190Z 2024-07-23T22:57:42.843+0000 [TRACE] log: registering event source with poller: token=Token(140667585312896), interests=READABLE | WRITABLE
2024-07-23T22:57:42.8441618Z 2024-07-23T22:57:42.843+0000 [TRACE] log: registering event source with poller: token=Token(140667585313280), interests=READABLE | WRITABLE
2024-07-23T22:57:42.8443321Z 2024-07-23T22:57:42.843+0000 [TRACE] turborepo_lib::process: releasing lock for spawning (/home/runner/work/apps/apps/apps/dram) /opt/hostedtoolcache/node/20.15.1/x64/bin/npm run codegen
2024-07-23T22:57:42.8445288Z 2024-07-23T22:57:42.843+0000 [TRACE] turborepo_lib::process::child: watching exit channel of (/home/runner/work/apps/apps/apps/dram) /opt/hostedtoolcache/node/20.15.1/x64/bin/npm run codegen
2024-07-23T22:57:42.8446651Z 2024-07-23T22:57:42.843+0000 [DEBUG] turborepo_lib::process::child: waiting for task
2024-07-23T22:57:42.8804068Z 2024-07-23T22:57:42.880+0000 [TRACE] tokio_util::codec::framed_impl: attempting to decode a frame
2024-07-23T22:57:42.8805922Z 2024-07-23T22:57:42.880+0000 [TRACE] tokio_util::codec::framed_impl: frame decoded from buffer
2024-07-23T22:57:42.8807733Z 2024-07-23T22:57:42.880+0000 [TRACE] tokio_util::codec::framed_impl: attempting to decode a frame
2024-07-23T22:57:42.8809426Z 2024-07-23T22:57:42.880+0000 [TRACE] tokio_util::codec::framed_impl: frame decoded from buffer
2024-07-23T22:57:42.8810718Z 2024-07-23T22:57:42.880+0000 [TRACE] tokio_util::codec::framed_impl: attempting to decode a frame
2024-07-23T22:57:42.9080176Z 2024-07-23T22:57:42.907+0000 [TRACE] tokio_util::codec::framed_impl: attempting to decode a frame
2024-07-23T22:57:42.9082567Z 2024-07-23T22:57:42.907+0000 [TRACE] tokio_util::codec::framed_impl: frame decoded from buffer
2024-07-23T22:57:42.9084693Z 2024-07-23T22:57:42.907+0000 [TRACE] tokio_util::codec::framed_impl: attempting to decode a frame
2024-07-23T22:57:42.9086729Z 2024-07-23T22:57:42.907+0000 [TRACE] tokio_util::codec::framed_impl: frame decoded from buffer
2024-07-23T22:57:42.9087965Z 2024-07-23T22:57:42.907+0000 [TRACE] tokio_util::codec::framed_impl: attempting to decode a frame
2024-07-23T22:57:42.9810043Z 2024-07-23T22:57:42.980+0000 [TRACE] turborepo_lib::process::child: processing stdout line
2024-07-23T22:57:42.9812067Z 2024-07-23T22:57:42.980+0000 [TRACE] turborepo_lib::process::child: processing stdout line
2024-07-23T22:57:42.9815009Z 2024-07-23T22:57:42.980+0000 [TRACE] turborepo_lib::process::child: watching exit channel of (/home/runner/work/apps/apps/apps/dram) /opt/hostedtoolcache/node/20.15.1/x64/bin/npm run codegen
2024-07-23T22:57:42.9817557Z 2024-07-23T22:57:42.980+0000 [TRACE] turborepo_lib::process::child: processing stdout line
2024-07-23T22:57:42.9819373Z 2024-07-23T22:57:42.980+0000 [TRACE] turborepo_lib::process::child: processing stdout line
2024-07-23T22:57:42.9822329Z 2024-07-23T22:57:42.980+0000 [TRACE] turborepo_lib::process::child: watching exit channel of (/home/runner/work/apps/apps/apps/dram) /opt/hostedtoolcache/node/20.15.1/x64/bin/npm run codegen
2024-07-23T22:57:43.2092096Z 2024-07-23T22:57:43.208+0000 [TRACE] turborepo_lib::process::child: processing stdout line
2024-07-23T22:57:43.2094002Z 2024-07-23T22:57:43.208+0000 [TRACE] turborepo_lib::process::child: watching exit channel of (/home/runner/work/apps/apps/apps/dram) /opt/hostedtoolcache/node/20.15.1/x64/bin/npm run codegen
2024-07-23T22:57:43.2108436Z 2024-07-23T22:57:43.210+0000 [TRACE] turborepo_lib::process::child: processing stdout line
2024-07-23T22:57:43.2110423Z 2024-07-23T22:57:43.210+0000 [TRACE] turborepo_lib::process::child: watching exit channel of (/home/runner/work/apps/apps/apps/dram) /opt/hostedtoolcache/node/20.15.1/x64/bin/npm run codegen
2024-07-23T22:57:43.8417870Z 2024-07-23T22:57:43.841+0000 [DEBUG] turborepo_telemetry: Starting telemetry event queue flush (num_events=7)
2024-07-23T22:57:43.8419956Z 2024-07-23T22:57:43.841+0000 [DEBUG] turborepo_telemetry: Done telemetry event queue flush
2024-07-23T22:57:43.8420928Z 2024-07-23T22:57:43.841+0000 [TRACE] log: signal: Want
2024-07-23T22:57:43.8421653Z 2024-07-23T22:57:43.841+0000 [TRACE] log: signal: Want
2024-07-23T22:57:43.8472916Z 2024-07-23T22:57:43.847+0000 [TRACE] tokio_util::codec::framed_impl: attempting to decode a frame
2024-07-23T22:57:43.8474161Z 2024-07-23T22:57:43.847+0000 [TRACE] tokio_util::codec::framed_impl: frame decoded from buffer
2024-07-23T22:57:43.8475260Z 2024-07-23T22:57:43.847+0000 [TRACE] tokio_util::codec::framed_impl: attempting to decode a frame
2024-07-23T22:57:43.8985552Z 2024-07-23T22:57:43.898+0000 [TRACE] tokio_util::codec::framed_impl: attempting to decode a frame
2024-07-23T22:57:43.8987009Z 2024-07-23T22:57:43.898+0000 [TRACE] tokio_util::codec::framed_impl: frame decoded from buffer
2024-07-23T22:57:43.8988204Z 2024-07-23T22:57:43.898+0000 [TRACE] tokio_util::codec::framed_impl: attempting to decode a frame
2024-07-23T22:57:43.8989246Z 2024-07-23T22:57:43.898+0000 [TRACE] tokio_util::codec::framed_impl: frame decoded from buffer
2024-07-23T22:57:43.8990285Z 2024-07-23T22:57:43.898+0000 [TRACE] tokio_util::codec::framed_impl: attempting to decode a frame
2024-07-23T22:57:45.2814924Z 2024-07-23T22:57:45.281+0000 [TRACE] turborepo_lib::process::child: processing stdout line
2024-07-23T22:57:45.2817935Z 2024-07-23T22:57:45.281+0000 [TRACE] turborepo_lib::process::child: watching exit channel of (/home/runner/work/apps/apps/apps/dram) /opt/hostedtoolcache/node/20.15.1/x64/bin/npm run codegen
2024-07-23T22:57:45.2840925Z 2024-07-23T22:57:45.283+0000 [TRACE] turborepo_lib::process::child: processing stdout line
2024-07-23T22:57:45.2843958Z 2024-07-23T22:57:45.283+0000 [TRACE] turborepo_lib::process::child: watching exit channel of (/home/runner/work/apps/apps/apps/dram) /opt/hostedtoolcache/node/20.15.1/x64/bin/npm run codegen
2024-07-23T22:57:45.2909883Z 2024-07-23T22:57:45.290+0000 [TRACE] turborepo_lib::process::child: processing stdout line
2024-07-23T22:57:45.2913115Z 2024-07-23T22:57:45.290+0000 [TRACE] turborepo_lib::process::child: watching exit channel of (/home/runner/work/apps/apps/apps/dram) /opt/hostedtoolcache/node/20.15.1/x64/bin/npm run codegen
2024-07-23T22:57:45.2979814Z 2024-07-23T22:57:45.297+0000 [TRACE] turborepo_lib::process::child: processing stdout line
2024-07-23T22:57:45.2981431Z 2024-07-23T22:57:45.297+0000 [TRACE] turborepo_lib::process::child: watching exit channel of (/home/runner/work/apps/apps/apps/dram) /opt/hostedtoolcache/node/20.15.1/x64/bin/npm run codegen
2024-07-23T22:57:45.3037744Z 2024-07-23T22:57:45.303+0000 [TRACE] turborepo_lib::process::child: processing stdout line
2024-07-23T22:57:45.3040973Z 2024-07-23T22:57:45.303+0000 [TRACE] turborepo_lib::process::child: watching exit channel of (/home/runner/work/apps/apps/apps/dram) /opt/hostedtoolcache/node/20.15.1/x64/bin/npm run codegen
2024-07-23T22:57:45.3318071Z 2024-07-23T22:57:45.331+0000 [TRACE] turborepo_lib::process::child: processing stdout line
2024-07-23T22:57:45.3319682Z 2024-07-23T22:57:45.331+0000 [TRACE] turborepo_lib::process::child: watching exit channel of (/home/runner/work/apps/apps/apps/dram) /opt/hostedtoolcache/node/20.15.1/x64/bin/npm run codegen
2024-07-23T22:57:45.3572147Z 2024-07-23T22:57:45.356+0000 [TRACE] turborepo_lib::process::child: processing stdout line
2024-07-23T22:57:45.3575759Z 2024-07-23T22:57:45.356+0000 [TRACE] turborepo_lib::process::child: watching exit channel of (/home/runner/work/apps/apps/apps/dram) /opt/hostedtoolcache/node/20.15.1/x64/bin/npm run codegen
2024-07-23T22:57:45.3578516Z 2024-07-23T22:57:45.357+0000 [TRACE] turborepo_lib::process::child: processing stdout line
2024-07-23T22:57:45.3581940Z 2024-07-23T22:57:45.357+0000 [TRACE] turborepo_lib::process::child: watching exit channel of (/home/runner/work/apps/apps/apps/dram) /opt/hostedtoolcache/node/20.15.1/x64/bin/npm run codegen
2024-07-23T22:57:45.3584638Z 2024-07-23T22:57:45.357+0000 [TRACE] turborepo_lib::process::child: processing stdout line
2024-07-23T22:57:45.3587542Z 2024-07-23T22:57:45.357+0000 [TRACE] turborepo_lib::process::child: watching exit channel of (/home/runner/work/apps/apps/apps/dram) /opt/hostedtoolcache/node/20.15.1/x64/bin/npm run codegen
2024-07-23T22:57:45.4094379Z 2024-07-23T22:57:45.409+0000 [TRACE] turborepo_lib::process::child: processing stdout line
2024-07-23T22:57:45.4097391Z 2024-07-23T22:57:45.409+0000 [TRACE] turborepo_lib::process::child: watching exit channel of (/home/runner/work/apps/apps/apps/dram) /opt/hostedtoolcache/node/20.15.1/x64/bin/npm run codegen
2024-07-23T22:57:45.4100119Z 2024-07-23T22:57:45.409+0000 [TRACE] turborepo_lib::process::child: processing stdout line
2024-07-23T22:57:45.4103025Z 2024-07-23T22:57:45.409+0000 [TRACE] turborepo_lib::process::child: watching exit channel of (/home/runner/work/apps/apps/apps/dram) /opt/hostedtoolcache/node/20.15.1/x64/bin/npm run codegen
2024-07-23T22:57:45.4105764Z 2024-07-23T22:57:45.410+0000 [TRACE] turborepo_lib::process::child: processing stdout line
2024-07-23T22:57:45.4107353Z 2024-07-23T22:57:45.410+0000 [TRACE] turborepo_lib::process::child: watching exit channel of (/home/runner/work/apps/apps/apps/dram) /opt/hostedtoolcache/node/20.15.1/x64/bin/npm run codegen
2024-07-23T22:57:45.4108726Z 2024-07-23T22:57:45.410+0000 [TRACE] turborepo_lib::process::child: processing stdout line
2024-07-23T22:57:45.4109649Z 2024-07-23T22:57:45.410+0000 [TRACE] turborepo_lib::process::child: processing stdout line
2024-07-23T22:57:45.4111143Z 2024-07-23T22:57:45.410+0000 [TRACE] turborepo_lib::process::child: watching exit channel of (/home/runner/work/apps/apps/apps/dram) /opt/hostedtoolcache/node/20.15.1/x64/bin/npm run codegen
2024-07-23T23:00:42.7822192Z 2024-07-23T23:00:42.781+0000 [TRACE] log: signal: Closed
2024-07-23T23:00:42.7823682Z 2024-07-23T23:00:42.781+0000 [DEBUG] log: Sending warning alert CloseNotify
2024-07-23T23:00:42.7825159Z 2024-07-23T23:00:42.782+0000 [TRACE] log: deregistering event source from poller
2024-07-23T23:08:14.9324400Z ##[error]The operation was canceled.
chris-olszewski commented 4 months ago

Merged what I think should fix this and cutting a canary. Once that finishes if people could update 2.0.10-canary.0 and see if the issue persist that would be very helpful.

A full write up of what was happening can be found in #8838

jakubmazanec commented 4 months ago

It seems to be working for me now (version 2.0.10-canary.1). Great job @chris-olszewski, thank you for fixing this.

unional commented 4 months ago

I have a case that it hangs too. Tried both 2.0.9 and 2.0.10-canary.1. It works in 1.13.0

https://github.com/repobuddy/repobuddy/tree/turbo-2-hang

it hangs when running the test or coverage script in the ./testcases/electron-ts or ./testcases/electron-renderer-ts projects.

When running them directly, they produce these messages but the script passes and terminates properly:

[45390:0726/003734.893877:ERROR:bus.cc(407)] Failed to connect to the bus: Failed to connect to socket /run/user/1000/bus: No such file or directory
[45390:0726/003734.893902:ERROR:bus.cc(407)] Failed to connect to the bus: Failed to connect to socket /run/user/1000/bus: No such file or directory

This happens on both CI (ubuntu) and WSL (ubuntu)

chris-olszewski commented 4 months ago

@unional can you confirm that they also hang with --env-mode=loose? Could you also run with -vvv? If this happens locally what are the final outputs of the script before it hangs?

unional commented 3 months ago

can you confirm that they also hang with --env-mode=loose

Still hang.

Could you also run with -vvv? If this happens locally what are the final outputs of the script before it hangs?

The UI bounces between electron-ts:test and electron-renderer-ts:test. Here are the logs that might be relevant:

electron-ts:test: 
electron-ts:test: > electron-ts@ test /home/unional/code/repobuddy/repobuddy/testcases/electron-ts
electron-ts:test: > cross-env NODE_NO_WARNINGS=1 jest
electron-ts:test: 
electron-renderer-ts:test: 
electron-renderer-ts:test: > electron-renderer-ts@ test /home/unional/code/repobuddy/repobuddy/testcases/electron-renderer-ts
electron-renderer-ts:test: > cross-env NODE_NO_WARNINGS=1 jest
electron-renderer-ts:test: 

electron-ts:test:  RUNS  src/feature.spec.ts
electron-ts:test:  RUNS  src/feature.spec.ts
electron-ts:test: [14460:0726/213008.778683:ERROR:ozone_platform_x11.cc(244)] Missing X server or $DISPLAY
electron-ts:test: [14460:0726/213008.778736:ERROR:env.cc(257)] The platform failed to initialize.  Exiting.
electron-renderer-ts:test: [14475:0726/213008.804590:ERROR:ozone_platform_x11.cc(244)] Missing X server or $DISPLAY
electron-renderer-ts:test: [14475:0726/213008.804636:ERROR:env.cc(257)] The platform failed to initialize.  Exiting.
2024-07-26T21:30:08.881-0700 [DEBUG] turborepo_telemetry: Starting telemetry event queue flush (num_events=3)
2024-07-26T21:30:08.881-0700 [DEBUG] turborepo_telemetry: Done telemetry event queue flush
2024-07-26T21:30:08.882-0700 [TRACE] log: signal: Want
2024-07-26T21:30:08.882-0700 [TRACE] log: signal: Want
2024-07-26T21:30:09.089-0700 [TRACE] tokio_util::codec::framed_impl: attempting to decode a frame
2024-07-26T21:30:09.089-0700 [TRACE] tokio_util::codec::framed_impl: frame decoded from buffer
2024-07-26T21:30:09.089-0700 [TRACE] tokio_util::codec::framed_impl: attempting to decode a frame
2024-07-26T21:30:09.089-0700 [TRACE] tokio_util::codec::framed_impl: frame decoded from buffer
electron-ts:test:  RUNS  src/feature.spec.ts
^C2024-07-26T21:30:30.380-0700 [DEBUG] turborepo_lib::process::child: stopping child process
2024-07-26T21:30:30.380-0700 [DEBUG] turborepo_lib::process::child: starting shutdown
2024-07-26T21:30:30.380-0700 [DEBUG] turborepo_lib::process::child: sending SIGINT to child 14303
2024-07-26T21:30:30.380-0700 [DEBUG] turborepo_lib::process::child: waiting for child 14303
2024-07-26T21:30:30.380-0700 [DEBUG] turborepo_lib::process: waiting for 2 processes to exit
2024-07-26T21:30:30.380-0700 [DEBUG] turborepo_lib::process::child: stopping child process
>                                                                                                                                                                                                                                                                               2024-07-26T21:30:30.380-0700 [DEBUG] turborepo_lib::process::child: starting shutdown
2024-07-26T21:30:30.380-0700 [DEBUG] turborepo_lib::process::child: sending SIGINT to child 14342
>   ...Finishing writing to cache...                                                                                                                                                                                                                                            2024-07-26T21:30:30.380-0700 [DEBUG] turborepo_lib::process::child: waiting for child 14342
 ELIFECYCLE  Test failed. See above for more details.

2024-07-26T21:30:30.410-0700 [DEBUG] turborepo_lib::process::child: child process stopped                                                                                                                                                                                       
2024-07-26T21:30:30.410-0700 [TRACE] turborepo_lib::process: process exited: Ok(Some(Killed))
2024-07-26T21:30:30.410-0700 [DEBUG] turborepo_lib::process: waiting for 2 processes to exit
2024-07-26T21:30:30.410-0700 [TRACE] turborepo_lib::process: process exited: Ok(Some(Killed))
2024-07-26T21:30:30.412-0700 [DEBUG] turborepo_lib::process::child: child process stopped
2024-07-26T21:30:30.412-0700 [TRACE] turborepo_lib::process: process exited: Ok(Some(Killed))
2024-07-26T21:30:30.412-0700 [TRACE] turborepo_lib::process: process exited: Ok(Some(Killed))
2024-07-26T21:30:30.412-0700 [DEBUG] turborepo_lib::process: waiting for 0 processes to exit
cyberuni  zsh   repobuddy  main ≡  ~4  9 
    ERROR  run failed: command  exited (1)
2024-07-26T21:30:30.418-0700 [TRACE] tower::buffer::worker: worker polling for next message
2024-07-26T21:30:30.418-0700 [TRACE] tower::buffer::worker: buffer already closed
2024-07-26T21:30:30.418-0700 [DEBUG] turborepo_telemetry: Starting telemetry event queue flush (num_events=2)
2024-07-26T21:30:30.418-0700 [DEBUG] turborepo_telemetry: Done telemetry event queue flush
2024-07-26T21:30:30.418-0700 [TRACE] log: signal: Closed
2024-07-26T21:30:30.418-0700 [TRACE] log: deregistering event source from poller
2024-07-26T21:30:30.418-0700 [TRACE] log: signal: Want
2024-07-26T21:30:30.418-0700 [TRACE] log: signal: Want
2024-07-26T21:30:30.629-0700 [TRACE] tokio_util::codec::framed_impl: attempting to decode a frame
2024-07-26T21:30:30.629-0700 [TRACE] tokio_util::codec::framed_impl: frame decoded from buffer
2024-07-26T21:30:30.629-0700 [TRACE] tokio_util::codec::framed_impl: attempting to decode a frame
2024-07-26T21:30:30.629-0700 [TRACE] tokio_util::codec::framed_impl: frame decoded from buffer
2024-07-26T21:30:30.629-0700 [TRACE] tokio_util::codec::framed_impl: attempting to decode a frame
2024-07-26T21:30:30.630-0700 [TRACE] log: signal: Closed
2024-07-26T21:30:30.630-0700 [DEBUG] turborepo_telemetry: telemetry handle closed
2024-07-26T21:30:30.630-0700 [DEBUG] log: Sending warning alert CloseNotify
2024-07-26T21:30:30.630-0700 [TRACE] log: deregistering event source from poller
R-Bower commented 3 months ago

@unional can you confirm that they also hang with --env-mode=loose? Could you also run with -vvv? If this happens locally what are the final outputs of the script before it hangs?

fwiw, I have been experiencing this issue with my electron apps on WSL2 since upgrading to turborepo v2. --env-mode=loose fixes the problem for me on 2.0.10-canary.4.

Edit: this flag is also working on 2.0.10.

NullVoxPopuli commented 3 months ago

I was also running in to this issue, and found success after upgrading to 2.0.10 and setting --env-mode=loose

petergrau commented 3 months ago

@R-Bower For us with 2.0.11 and --env-mode=loose it still does not work at the moment.

The log out put before it hangs is the following:

2024-08-02T10:17:08.283+0000 [TRACE] turborepo_lib::process::child: processing stdout line
2024-08-02T10:17:08.283+0000 [TRACE] turborepo_lib::process::child: watching exit channel of (/home/runner/work/xxxx) /home/runner/setup-pnpm/node_modules/.bin/pnpm run lint --max-warnings=0
2024-08-02T10:17:08.283+0000 [TRACE] turborepo_lib::process::child: processing stdout line
2024-08-02T10:17:08.283+0000 [TRACE] turborepo_lib::process::child: processing stdout line
2024-08-02T10:17:08.283+0000 [TRACE] turborepo_lib::process::child: watching exit channel of (/home/runner/work/xxx) /home/runner/setup-pnpm/node_modules/.bin/pnpm run lint --max-warnings=0
kpervin commented 3 months ago

^2.0.14 seems to now be running fine in GH Actions, even without --env-mode=loose.

unional commented 3 months ago

Just tried with 2.0.14 on repobuddy. The issue still exists

MattisAbrahamsson commented 1 month ago

Having the same issue in 2.1.3, tested with --env-mode=loose but no luck

These are the last lines of logs before it hangs

@repo/form:typecheck
2024-10-02T13:09:19.957+0000 [TRACE] turborepo_telemetry::config: Telemetry config path: /home/github-runner/.config/turborepo/telemetry.json
2024-10-02T13:09:19.958+0000 [TRACE] turborepo_telemetry::config: Telemetry config path: /home/github-runner/.config/turborepo/telemetry.json
2024-10-02T13:09:19.958+0000 [DEBUG] turborepo_lib::task_hash: auto detected framework for @repo/components
2024-10-02T13:09:19.958+0000 [DEBUG] turborepo_lib::task_hash: framework: nextjs, env_prefix: ["NEXT_PUBLIC_*"]
2024-10-02T13:09:19.959+0000 [DEBUG] turborepo_lib::task_graph::visitor: task @repo/components#typecheck hash is bdcebe57e953309e
2024-10-02T13:09:19.959+0000 [TRACE] turborepo_telemetry::config: Telemetry config path: /home/github-runner/.config/turborepo/telemetry.json
2024-10-02T13:09:19.960+0000 [TRACE] turborepo_lib::process: acquiring lock for spawning (/home/github-runner/_work/platform/platform/frontend/packages/components) /tmp/xfs-1372de2d/yarn run typecheck
2024-10-02T13:09:19.960+0000 [TRACE] turborepo_lib::process: acquired lock for spawning (/home/github-runner/_work/platform/platform/frontend/packages/components) /tmp/xfs-1372de2d/yarn run typecheck
2024-10-02T13:09:19.974+0000 [TRACE] turborepo_lib::process: releasing lock for spawning (/home/github-runner/_work/platform/platform/frontend/packages/components) /tmp/xfs-1372de2d/yarn run typecheck
2024-10-02T13:09:19.982+0000 [TRACE] turborepo_lib::process::child: watching exit channel of (/home/github-runner/_work/platform/platform/frontend/packages/components) /tmp/xfs-1372de2d/yarn run typecheck
2024-10-02T13:09:19.982+0000 [DEBUG] turborepo_lib::process::child: waiting for task
2024-10-02T13:09:20.960+0000 [TRACE] turborepo_telemetry: Flushed telemetry event queue (num_events=1)
2024-10-02T13:09:20.960+0000 [TRACE] log: signal: Want
2024-10-02T13:09:20.960+0000 [TRACE] log: signal: Want
2024-10-02T13:09:21.272+0000 [TRACE] tokio_util::codec::framed_impl: attempting to decode a frame
2024-10-02T13:09:21.272+0000 [TRACE] tokio_util::codec::framed_impl: frame decoded from buffer
2024-10-02T13:09:21.272+0000 [TRACE] tokio_util::codec::framed_impl: attempting to decode a frame
2024-10-02T13:09:21.272+0000 [TRACE] tokio_util::codec::framed_impl: frame decoded from buffer
2024-10-02T13:09:21.272+0000 [TRACE] tokio_util::codec::framed_impl: attempting to decode a frame
petergrau commented 1 month ago

We are still having the issue - but only with the lint task. Is there anything we can do from a config perspective to make it work?

chris-olszewski commented 1 month ago

@petergrau Can you try forcibly closing stdin for your lint task and see if that changes the behavior? e.g. "lint": "my-linter < /dev/null"?

Just double checking a few things:

anthonyshew commented 1 month ago

In the interest of casting a wider net for more info, if folks can supply us with reproductions for these cases, it would be a huge help. 🙏 The earlier requests for -vvv logs looks like we captured and fixed one variant of reports but it looks like we're looking for something else now, and the -vvv logs aren't quite giving us enough.

petergrau commented 1 month ago

@chris-olszewski thanks for following up.

@petergrau Can you try forcibly closing stdin for your lint task and see if that changes the behavior? e.g. "lint": "my-linter < /dev/null"?

Tried that and seeing the same behaviour.

Just double checking a few things:

  • Is this only on GitHub Actions?

Yes - locally the tasks runs fine.

  • What OS are you using?

The workflow is using runs-on: ubuntu-latest

  • What program are you using to lint?

eslint

  • Just to double check you don't have your lint config enabled with "interactive": true or "persistent": true

nothing like that in the config

chris-olszewski commented 1 month ago

@petergrau

Very sorry for not recommending this earlier, but can you add --log-order=stream to your run command in CI and see if the hanging task is outputting anything suspect?

petergrau commented 1 month ago

@petergrau

Very sorry for not recommending this earlier, but can you add --log-order=stream to your run command in CI and see if the hanging task is outputting anything suspect?

@chris-olszewski no problem - do you want me to combine that with -vv or just that one option?

chris-olszewski commented 1 month ago

-vv would be great as well. If you'd like you could send me the logs at chris.olszewski@vercel.com.

MattisAbrahamsson commented 4 weeks ago

I have noticed that after upgrading to 2.1.3 the CI job fails much quicker, before 2.1.3 it would just stall and the job would timeout after around 15 minutes, but now it seems to fail after just a couple of minutes

Tofandel commented 3 weeks ago

FYI, I see that you are using process.report.getReport() to determine glibc, this has been known to cause freezes https://github.com/vercel/turborepo/blob/ba769dc14dbb556f6aace5c6c7c775ae894a2b1b/packages/turbo-repository/js/index.js#L29

https://github.com/nodejs/node/issues/55576#issuecomment-2443914070

This has caused freeze issues in boh yarn and npm already, I suggest you check out https://github.com/npm/npm-install-checks/pull/120 which is a viable fix for npm, basically the getReport has to run before anything else in the program (like fetch for telemetry) and not be used after that

Though I'm not sure this is directly the cause of this particular freeze issue

anthonyshew commented 3 weeks ago

@Tofandel From what we can tell at this point, this is happening because folks' tasks aren't closing. The program isn't freezing. Rather, it's continuing to run when we don't want it to.

However, that's only what we can intuit from the reports so far. We need dependable reproductions to look at. We've tried to make some ourselves but haven't been able to.

MattisAbrahamsson commented 3 weeks ago

@Tofandel From what we can tell at this point, this is happening because folks' tasks aren't closing. The program isn't freezing. Rather, it's continuing to run when we don't want it to.

However, that's only what we can intuit from the reports so far. We need dependable reproductions to look at. We've tried to make some ourselves but haven't been able to.

It's a bit difficult to create a reproduction for this, would it be helpful if I invited someone from the team to our repository? They would have to sign an NDA according to company policy, but other than that it should be all clear.

petergrau commented 1 week ago

@anthonyshew I saw that you remove the owned-by: turborepo label - does that mean work on this will be discontinued?

anthonyshew commented 1 week ago

I was doing some repository cleanup. Those labels are from when Turbopack was also in this repository, so aren't meaningful anymore. We still need figure out how to resolve this.