felixmosh / turborepo-gh-artifacts

A TurboRepo local cache server which uploads artifact cache to GH artifacts.
MIT License
122 stars 7 forks source link

context deadline exceeded (Client.Timeout exceeded while awaiting headers) #22

Open dstoyanoff opened 1 year ago

dstoyanoff commented 1 year ago

Hello, I am having trouble making this action work. After enabling turbo trace logs, I saw these errors:

2023-01-16T07:33:00.8209123Z 2023-01-16T07:33:00.820Z [ERROR] turbo: request failed: error="Get \"http://172.0.0.1:9080/v8/artifacts/48bf385e2de897f2?slug=magna\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" method=GET url=http://172.0.0.1:9080/v8/artifacts/48bf385e2de897f2?slug=magna

Any clue what the issue might be?

felixmosh commented 1 year ago

Hi, No I'm not familiar with this error... Maybe the local server didn't started? can you check its logs?

dstoyanoff commented 1 year ago

The actions logs this:

Run felixmosh/turborepo-gh-artifacts@v1
TURBO_LOCAL_SERVER_PID: 2486
Warning: The `save-state` command is deprecated and will be disabled soon. Please upgrade to using Environment Files. For more information see: https://github.blog/changelog/2022-10-[11](https://github.com/magna-eng/web/actions/runs/3928155787/jobs/6715472911#step:6:12)-github-actions-deprecating-save-state-and-set-output-commands/

Then the post action logs look like this:

Post job cleanup.
Found server pid: [2](https://github.com/magna-eng/web/actions/runs/3928155787/jobs/6715472911#step:11:2)486
Killing server pid: 2486
There is nothing to upload.
Server logs:
Cache dir: /home/runner/work/_temp/turbo-cache
Local Turbo server is listening at http://12[7](https://github.com/magna-eng/web/actions/runs/3928155787/jobs/6715472911#step:11:7).0.0.1:9080
felixmosh commented 1 year ago

From the error looks like your local server not started, and Turborepo fails on timeout, is you setup similar to this example? https://github.com/felixmosh/turborepo-gh-artifacts-example

dstoyanoff commented 1 year ago

Yes, I had a look at that example. Here is part of the workflow:

      - name: TurboRepo local server
        uses: felixmosh/turborepo-gh-artifacts@v1
        with:
          repo-token: ${{ secrets.GITHUB_TOKEN }}
          server-token: ${{ env.TURBO_TOKEN }}

      - name: Build & Test
        run: npx turbo run build test --filter investor --api="http://172.0.0.1:9080" --token="${{ env.TURBO_TOKEN }}" --team=magna -vvv
felixmosh commented 1 year ago

Try to run it without npx (as a script)... You can try to pass the server params as Env vars. (checkout the readme)

dstoyanoff commented 1 year ago

I actually started with env vars, but switched to arguments to see if it would help with the issue.

Unfortunately not different using yarn build directly.

Is there a way to add more verbosity to the action to understand what is going wrong? It seems to me the server is running as it yields a PID

felixmosh commented 1 year ago

There are logs on each request... https://github.com/felixmosh/turborepo-gh-artifacts/blob/master/src/turboServer.ts#L25

From your logs (they should apear in the post action part), looks like there are no requests...

What version of Turborepo / node are u using?

dstoyanoff commented 1 year ago

node: v16.19.0 npm: 8.19.3 yarn: 1.22.19 turbo: 1.6.3

dstoyanoff commented 1 year ago

No luck with turbo 1.7.0 as well. Btw, I see that there are a few newer releases of the action that are not shown in the marketplace. Perhaps there is a problem with it?

felixmosh commented 1 year ago

Let's try to debug it locally,

  1. Clone this repo
  2. Enter the cloned folder
  3. env "INPUT_REPO-TOKEN=test" "INPUT_SERVER-TOKEN=test" node dist/turboServer/index.js

This should start the proxy server... REPO-TOKEN suppose to be GITHUB_TOKEN, if you not provide a proper one it will mimic "always cache hit".

Enter to your projects build, and run the turborepo command with

yarn build --api="http://172.0.0.1:9080" --token="test" --team=magna

Let's see if it connects to the server + if it saves to the artifacts to local cache folder.

felixmosh commented 1 year ago

No luck with turbo 1.7.0 as well. Btw, I see that there are a few newer releases of the action that are not shown in the marketplace. Perhaps there is a problem with it?

Wow, this is weird, I've missed it, can you check with v2 tag?

felixmosh commented 1 year ago

Is upgrading to v2 solved your issue?

dstoyanoff commented 1 year ago

Not really, but I am double checking if there is any difference

dstoyanoff commented 1 year ago

For some reason turbo stopped showing me verbose logging, but there seems to be no big difference. The warning about the action using save-state is gone, but the output seems similar. Should I do anything for the cache dir? Perhaps tell turbo to use that dir for caching?

Post job cleanup.
Found server pid: [2](https://github.com/magna-eng/web/actions/runs/3936673484/jobs/6733417894#step:11:2)501
Killing server pid: 2501
There is nothing to upload.
Server logs:
Cache dir: /home/runner/work/_temp/turbo-cache
Local Turbo server is listening at http://12[7](https://github.com/magna-eng/web/actions/runs/3936673484/jobs/6733417894#step:11:7).0.0.1:9080
felixmosh commented 1 year ago

No, it is using a temp dir. Can you try the above explanation regarding local testing it?

dstoyanoff commented 1 year ago

Sorry, I have missed that comment.

It still yields no logs and fails with the same error:

cache miss, executing c0e1aec919923516
2023-01-17T18:30:45.751+0200 [ERROR] turbo: request failed: error="Get \"http://172.0.0.1:9080/v8/artifacts/5cc2cd831c72e046?slug=magna\": dial tcp 172.0.0.1:9080: i/o timeout (Client.Timeout exceeded while awaiting headers)" method=GET url=http://172.0.0.1:9080/v8/artifacts/5cc2cd831c72e046?slug=magna

And then all of a sudden it did work after re-running the server and not changing anything. Still not working in GH actions though. I am thinking there might be an issue on the first run in a clean environment..

felixmosh commented 1 year ago

Weird, it should make api calls... Maybe you have a wrong turbo config? Did you tried to run it without remote cache?

dstoyanoff commented 1 year ago

Locally without remote cache it's working. I haven't tested on github actions.

Here is my turbo config:

{
  "$schema": "https://turbo.build/schema.json",
  "pipeline": {
    "build": {
      "inputs": ["src/**/*.tsx", "src/**/*.ts"],
      "outputs": [".next/**", "dist/**"]
    },
    "test": {
      "dependsOn": ["build"],
      "inputs": ["src/**/*.tsx", "src/**/*.ts"],
      "outputs": []
    },
    "deploy": {
      "dependsOn": ["build"],
      "inputs": ["src/**/*.tsx", "src/**/*.ts"],
      "outputs": []
    }
  }
}
felixmosh commented 1 year ago

Yes, I had a look at that example. Here is part of the workflow:

      - name: TurboRepo local server
        uses: felixmosh/turborepo-gh-artifacts@v1
        with:
          repo-token: ${{ secrets.GITHUB_TOKEN }}
          server-token: ${{ env.TURBO_TOKEN }}

      - name: Build & Test
        run: npx turbo run build test --filter investor --api="http://172.0.0.1:9080" --token="${{ env.TURBO_TOKEN }}" --team=magna -vvv

What is the value of env.TURBO_TOKEN? can you put the token inside secrets and change it to secrets.TURBO_TOKEN on both places?