search

ducktors / turborepo-remote-cache

Open source implementation of the Turborepo custom remote cache server.
https://ducktors.github.io/turborepo-remote-cache/
MIT License
1.03k stars 95 forks source link

Error on use with Github Actions #114

Closed mnzsss closed 1 year ago

mnzsss commented 1 year ago

🐛 Bug Report

I tried use this repo with docker inside github actions, but I got error on during the artifact creation. I no have ideia what ocurring this error because this is generic.

I clone this repo locally for debug and this work normaly.

To Reproduce

Steps to reproduce the behavior:

name: Build and Deploy
on: [push]
jobs:
  build:
    name: Build and Deploy
    runs-on: ubuntu-latest

    services:
      turbo_cache:
        image: fox1t/turborepo-remote-cache
        env:
          NODE_ENV: production
          PORT: 3000
          TURBO_TOKEN: XXX
          STORAGE_PROVIDER: google-cloud-storage
          STORAGE_PATH: <storage-path>
          GCS_PROJECT_ID: <project-id>
          GCS_CLIENT_EMAIL: <email>
          GCS_PRIVATE_KEY: ${{ secrets.GCP_CREDENTIALS_JSON.private_key }}
        ports:
          - 3000:3000
//...

Error on Github Actions

Print service container logs: 21bde0cd5856471ca87367f9ae4b6d9d_fox1tturboreporemotecache_9b0647
/usr/bin/docker logs --details bb3a0a8794f47437bd7be173ed7db8f9f44e791998c3[2](https://github.com/beyounglabs/transformers/actions/runs/4084527958/jobs/7041334215#step:10:2)f5c606[3](https://github.com/beyounglabs/transformers/actions/runs/4084527958/jobs/7041334215#step:10:3)a36fae8e4355
 {"severity":"INFO","level":30,"time":1675430365053,"pid":7,"hostname":"bb3a0a8794f4","message":"Server listening at http://0.0.0.0:3000/"}
 {"severity":"INFO","level":30,"time":1675[4](https://github.com/beyounglabs/transformers/actions/runs/4084527958/jobs/7041334215#step:10:4)30[5](https://github.com/beyounglabs/transformers/actions/runs/4084527958/jobs/7041334215#step:10:5)59487,"pid":7,"hostname":"bb3a0a8794f4","reqId":"UT8iMrFOS8G-Gre72NQmrQ-0","req":{"method":"GET","url":"/v8/artifacts/7ca77c752709781f?slug=beyoung_transformers","hostname":"172.17.0.1:3000","remoteAddress":"172.18.0.1","remotePort":32910},"message":"incoming request"}
 {"severity":"WARNING","level":40,"time":1675430559579,"pid":7,"hostname":"bb3a0a8794f4","reqId":"UT8iMrFOS8G-Gre72NQmrQ-0","data":{},"isBoom":true,"isServer":false,"output":{"statusCode":404,"payload":{"statusCode":404,"error":"Not Found","message":"Artifact not found"},"headers":{}},"stack":"/home/app/node/src/plugins/remote-cache/routes/get-artifact.ts:29\n      throw notFound(`Artifact not found`, err)\n                    ^\n\nError: Artifact not found\n    at Object.handler (/home/app/node/src/plugins/remote-cache/routes/get-artifact.ts:29:21)\n    at processTicksAndRejections (node:internal/process/task_queues:96:5)","type":"Error","message":"Artifact not found"}
 {"severity":"INFO","level":30,"time":1[6](https://github.com/beyounglabs/transformers/actions/runs/4084527958/jobs/7041334215#step:10:6)[7](https://github.com/beyounglabs/transformers/actions/runs/4084527958/jobs/7041334215#step:10:7)5430559585,"pid":7,"hostname":"bb3a0a8794f4","reqId":"UT8iMrFOS8G-Gre72NQmrQ-0","res":{"statusCode":404},"responseTime":96.68799800000852,"message":"request completed"}
 {"severity":"INFO","level":30,"time":1675430559788,"pid":7,"hostname":"bb3a0a8794f4","reqId":"UT8iMrFOS8G-Gre72NQmrQ-1","req":{"method":"POST","url":"/v8/artifacts/events?slug=beyoung_transformers","hostname":"172.17.0.1:3000","remoteAddress":"172.18.0.1","remotePort":32926},"message":"incoming request"}
 {"severity":"INFO","level":30,"time":1675430559791,"pid":7,"hostname":"bb3a0a[8](https://github.com/beyounglabs/transformers/actions/runs/4084527958/jobs/7041334215#step:10:8)7[9](https://github.com/beyounglabs/transformers/actions/runs/4084527958/jobs/7041334215#step:10:9)4f4","reqId":"UT8iMrFOS8G-Gre72NQmrQ-1","res":{"statusCode":200},"responseTime":2.7053000000014435,"message":"request completed"}
 {"severity":"INFO","level":30,"time":1675430563280,"pid":7,"hostname":"bb3a0a8794f4","reqId":"UT8iMrFOS8G-Gre72NQmrQ-2","req":{"method":"GET","url":"/v8/artifacts/b3c1adf0c0c67da0?slug=beyoung_transformers","hostname":"172.17.0.1:3000","remoteAddress":"172.18.0.1","remotePort":34550},"message":"incoming request"}
 {"severity":"INFO","level":30,"time":1675430563284,"pid":7,"hostname":"bb3a0a8794f4","reqId":"UT8iMrFOS8G-Gre72NQmrQ-3","req":{"method":"PUT","url":"/v8/artifacts/7ca77c752709781f?slug=beyoung_transformers","hostname":"172.17.0.1:3000","remoteAddress":"172.18.0.1","remotePort":34560},"message":"incoming request"}
 {"severity":"WARNING","level":40,"time":1675430563298,"pid":7,"hostname":"bb3a0a8794f4","reqId":"UT8iMrFOS8G-Gre72NQmrQ-3","data":{},"isBoom":true,"isServer":false,"output":{"statusCode":412,"payload":{"statusCode":412,"error":"Precondition Failed","message":"Error during the artifact creation"},"headers":{}},"stack":"/home/app/node/src/plugins/remote-cache/routes/put-artifact.ts:33\n      throw preconditionFailed(`Error during the artifact creation`, err)\n                              ^\n\nError: Error during the artifact creation\n    at Object.handler (/home/app/node/src/plugins/remote-cache/routes/put-artifact.ts:33:31)\n    at processTicksAndRejections (node:internal/process/task_queues:96:5)","type":"Error","message":"Error during the artifact creation"}
 {"severity":"INFO","level":30,"time":1675430563299,"pid":7,"hostname":"bb3a0a8794f4","reqId":"UT8iMrFOS8G-Gre72NQmrQ-3","res":{"statusCode":412},"responseTime":15.39499900001[10](https://github.com/beyounglabs/transformers/actions/runs/4084527958/jobs/7041334215#step:10:10)12,"message":"request completed"}
 {"severity":"WARNING","level":40,"time":1675430563308,"pid":7,"hostname":"bb3a0a8794f4","reqId":"UT8iMrFOS8G-Gre72NQmrQ-2","data":{},"isBoom":true,"isServer":false,"output":{"statusCode":404,"payload":{"statusCode":404,"error":"Not Found","message":"Artifact not found"},"headers":{}},"stack":"/home/app/node/src/plugins/remote-cache/routes/get-artifact.ts:29\n      throw notFound(`Artifact not found`, err)\n                    ^\n\nError: Artifact not found\n    at Object.handler (/home/app/node/src/plugins/remote-cache/routes/get-artifact.ts:29:21)\n    at processTicksAndRejections (node:internal/process/task_queues:96:5)","type":"Error","message":"Artifact not found"}
 {"severity":"INFO","level":30,"time":1675430563309,"pid":7,"hostname":"bb3a0a8794f4","reqId":"UT8iMrFOS8G-Gre72NQmrQ-2","res":{"statusCode":404},"responseTime":28.97209799999837,"message":"request completed"}
 {"severity":"INFO","level":30,"time":1675430563512,"pid":7,"hostname":"bb3a0a8794f4","reqId":"UT8iMrFOS8G-Gre72NQmrQ-4","req":{"method":"POST","url":"/v8/artifacts/events?slug=beyoung_transformers","hostname":"172.17.0.1:3000","remoteAddress":"172.18.0.1","remotePort":34572},"message":"incoming request"}
 {"severity":"INFO","level":30,"time":1675430563513,"pid":7,"hostname":"bb3a0a8794f4","reqId":"UT8iMrFOS8G-Gre72NQmrQ-4","res":{"statusCode":200},"responseTime":1.[11](https://github.com/beyounglabs/transformers/actions/runs/4084527958/jobs/7041334215#step:10:11)73000000[12](https://github.com/beyounglabs/transformers/actions/runs/4084527958/jobs/7041334215#step:10:12)6194,"message":"request completed"}
 {"severity":"INFO","level":30,"time":1675430569545,"pid":7,"hostname":"bb3a0a8794f4","reqId":"UT8iMrFOS8G-Gre72NQmrQ-5","req":{"method":"GET","url":"/v8/artifacts/7a84019eaa8739de?slug=beyoung_transformers","hostname":"172.17.0.1:3000","remoteAddress":"172.18.0.1","remotePort":34582},"message":"incoming request"}
 {"severity":"INFO","level":30,"time":1675430569549,"pid":7,"hostname":"bb3a0a8794f4","reqId":"UT8iMrFOS8G-Gre72NQmrQ-6","req":{"method":"PUT","url":"/v8/artifacts/b3c1adf0c0c67da0?slug=beyoung_transformers","hostname":"172.17.0.1:3000","remoteAddress":"172.18.0.1","remotePort":34594},"message":"incoming request"}
 {"severity":"INFO","level":30,"time":1675430569549,"pid":7,"hostname":"bb3a0a8794f4","reqId":"UT8iMrFOS8G-Gre72NQmrQ-7","req":{"method":"GET","url":"/v8/artifacts/eaa815124d059847?slug=beyoung_transformers","hostname":"172.17.0.1:3000","remoteAddress":"172.18.0.1","remotePort":34578},"message":"incoming request"}
 {"severity":"WARNING","level":40,"time":1675430569556,"pid":7,"hostname":"bb3a0a8794f4","reqId":"UT8iMrFOS8G-Gre72NQmrQ-6","data":{},"isBoom":true,"isServer":false,"output":{"statusCode":412,"payload":{"statusCode":412,"error":"Precondition Failed","message":"Error during the artifact creation"},"headers":{}},"stack":"/home/app/node/src/plugins/remote-cache/routes/put-artifact.ts:33\n      throw preconditionFailed(`Error during the artifact creation`, err)\n                              ^\n\nError: Error during the artifact creation\n    at Object.handler (/home/app/node/src/plugins/remote-cache/routes/put-artifact.ts:33:31)\n    at processTicksAndRejections (node:internal/process/task_queues:96:5)","type":"Error","message":"Error during the artifact creation"}
 {"severity":"INFO","level":30,"time":1675430569557,"pid":7,"hostname":"bb3a0a8794f4","reqId":"UT8iMrFOS8G-Gre72NQmrQ-6","res":{"statusCode":412},"responseTime":8.76[14](https://github.com/beyounglabs/transformers/actions/runs/4084527958/jobs/7041334215#step:10:14)99000014737,"message":"request completed"}
 {"severity":"WARNING","level":40,"time":1675430569577,"pid":7,"hostname":"bb3a0a8794f4","reqId":"UT8iMrFOS8G-Gre72NQmrQ-5","data":{},"isBoom":true,"isServer":false,"output":{"statusCode":404,"payload":{"statusCode":404,"error":"Not Found","message":"Artifact not found"},"headers":{}},"stack":"/home/app/node/src/plugins/remote-cache/routes/get-artifact.ts:29\n      throw notFound(`Artifact not found`, err)\n                    ^\n\nError: Artifact not found\n    at Object.handler (/home/app/node/src/plugins/remote-cache/routes/get-artifact.ts:29:21)\n    at processTicksAndRejections (node:internal/process/task_queues:96:5)","type":"Error","message":"Artifact not found"}
 {"severity":"INFO","level":30,"time":1675430569579,"pid":7,"hostname":"bb3a0a8794f4","reqId":"UT8iMrFOS8G-Gre72NQmrQ-5","res":{"statusCode":404},"responseTime":33.89039800001774,"message":"request completed"}
 {"severity":"WARNING","level":40,"time":1675430569605,"pid":7,"hostname":"bb3a0a8794f4","reqId":"UT8iMrFOS8G-Gre72NQmrQ-7","data":{},"isBoom":true,"isServer":false,"output":{"statusCode":404,"payload":{"statusCode":404,"error":"Not Found","message":"Artifact not found"},"headers":{}},"stack":"/home/app/node/src/plugins/remote-cache/routes/get-artifact.ts:29\n      throw notFound(`Artifact not found`, err)\n                    ^\n\nError: Artifact not found\n    at Object.handler (/home/app/node/src/plugins/remote-cache/routes/get-artifact.ts:29:21)\n    at processTicksAndRejections (node:internal/process/task_queues:96:5)","type":"Error","message":"Artifact not found"}
 {"severity":"INFO","level":30,"time":1675430569606,"pid":7,"hostname":"bb3a0a8794f4","reqId":"UT8iMrFOS8G-Gre72NQmrQ-7","res":{"statusCode":404},"responseTime":56.34499599999981,"message":"request completed"}
 {"severity":"INFO","level":30,"time":1675430569808,"pid":7,"hostname":"bb3a0a8794f4","reqId":"UT8iMrFOS8G-Gre72NQmrQ-8","req":{"method":"POST","url":"/v8/artifacts/events?slug=beyoung_transformers","hostname":"172.17.0.1:3000","remoteAddress":"172.18.0.1","remotePort":34604},"message":"incoming request"}
 {"severity":"INFO","level":30,"time":1675430569810,"pid":7,"hostname":"bb3a0a8794f4","reqId":"UT8iMrFOS8G-Gre72NQmrQ-8","res":{"statusCode":200},"responseTime":0.8954000000085216,"message":"request completed"}
 {"severity":"INFO","level":30,"time":1675430588230,"pid":7,"hostname":"bb3a0a8794f4","reqId":"UT8iMrFOS8G-Gre72NQmrQ-9","req":{"method":"PUT","url":"/v8/artifacts/eaa8[15](https://github.com/beyounglabs/transformers/actions/runs/4084527958/jobs/7041334215#step:10:15)124d059847?slug=beyoung_transformers","hostname":"172.17.0.1:3000","remoteAddress":"172.18.0.1","remotePort":45336},"message":"incoming request"}
 {"severity":"WARNING","level":40,"time":[16](https://github.com/beyounglabs/transformers/actions/runs/4084527958/jobs/7041334215#step:10:16)75430588272,"pid":7,"hostname":"bb3a0a8794f4","reqId":"UT8iMrFOS8G-Gre72NQmrQ-9","data":{},"isBoom":true,"isServer":false,"output":{"statusCode":412,"payload":{"statusCode":412,"error":"Precondition Failed","message":"Error during the artifact creation"},"headers":{}},"stack":"/home/app/node/src/plugins/remote-cache/routes/put-artifact.ts:33\n      throw preconditionFailed(`Error during the artifact creation`, err)\n                              ^\n\nError: Error during the artifact creation\n    at Object.handler (/home/app/node/src/plugins/remote-cache/routes/put-artifact.ts:33:31)\n    at processTicksAndRejections (node:internal/process/task_queues:96:5)","type":"Error","message":"Error during the artifact creation"}
 {"severity":"INFO","level":30,"time":1675430588274,"pid":7,"hostname":"bb3a0a8794f4","reqId":"UT8iMrFOS8G-Gre72NQmrQ-9","res":{"statusCode":412},"responseTime":43.37320099998033,"message":"request completed"}
 {"severity":"INFO","level":30,"time":1675430621811,"pid":7,"hostname":"bb3a0a8794f4","reqId":"UT8iMrFOS8G-Gre72NQmrQ-10","req":{"method":"PUT","url":"/v8/artifacts/7a84019eaa8739de?slug=beyoung_transformers","hostname":"[17](https://github.com/beyounglabs/transformers/actions/runs/4084527958/jobs/7041334215#step:10:17)2.17.0.1:3000","remoteAddress":"172.[18](https://github.com/beyounglabs/transformers/actions/runs/4084527958/jobs/7041334215#step:10:18).0.1","remotePort":35098},"message":"incoming request"}
 {"severity":"WARNING","level":40,"time":16754306[21](https://github.com/beyounglabs/transformers/actions/runs/4084527958/jobs/7041334215#step:10:21)863,"pid":7,"hostname":"bb3a0a8794f4","reqId":"UT8iMrFOS8G-Gre72NQmrQ-10","data":{},"isBoom":true,"isServer":false,"output":{"statusCode":412,"payload":{"statusCode":412,"error":"Precondition Failed","message":"Error during the artifact creation"},"headers":{}},"stack":"/home/app/node/src/plugins/remote-cache/routes/put-artifact.ts:33\n      throw preconditionFailed(`Error during the artifact creation`, err)\n                              ^\n\nError: Error during the artifact creation\n    at Object.handler (/home/app/node/src/plugins/remote-cache/routes/put-artifact.ts:33:31)\n    at processTicksAndRejections (node:internal/process/task_queues:96:5)","type":"Error","message":"Error during the artifact creation"}
 {"severity":"INFO","level":30,"time":1675430621872,"pid":7,"hostname":"bb3a0a8794f4","reqId":"UT8iMrFOS8G-Gre72NQmrQ-10","res":{"statusCode":412},"responseTime":60.56[23](https://github.com/beyounglabs/transformers/actions/runs/4084527958/jobs/7041334215#step:10:23)04999970365,"message":"request completed"}

Turbo build Cache enabled

image

Cache storaged (from local test) in GCP Bucket

image

Again, I use console.error locally in this file to catch the error.

"Error: error:0909006C:PEM routines:get_name:no start line\n    at Sign.sign (node:internal/crypto/sig:131:29)\n    at Object.sign (/home/mnzs/tmp/turborepo-remote-cache/node_modules/.pnpm/jwa@2.0.0/node_modules/jwa/index.js:152:45)\n    at Object.jwsSign [as sign] (/home/mnzs/tmp/turborepo-remote-cache/node_modules/.pnpm/jws@4.0.0/node_modules/jws/lib/sign-stream.js:32:24)\n    at GoogleToken.requestToken (/home/mnzs/tmp/turborepo-remote-cache/node_modules/.pnpm/gtoken@6.1.1/node_modules/gtoken/build/src/index.js:232:31)\n    at GoogleToken.getTokenAsyncInner (/home/mnzs/tmp/turborepo-remote-cache/node_modules/.pnpm/gtoken@6.1.1/node_modules/gtoken/build/src/index.js:166:21)\n    at GoogleToken.getTokenAsync (/home/mnzs/tmp/turborepo-remote-cache/node_modules/.pnpm/gtoken@6.1.1/node_modules/gtoken/build/src/index.js:145:55)\n    at GoogleToken.getToken (/home/mnzs/tmp/turborepo-remote-cache/node_modules/.pnpm/gtoken@6.1.1/node_modules/gtoken/build/src/index.js:97:21)\n    at JWT.refreshTokenNoCache (/home/mnzs/tmp/turborepo-remote-cache/node_modules/.pnpm/google-auth-library@8.4.0/node_modules/google-auth-library/build/src/auth/jwtclient.js:172:36)\n    at JWT.refreshToken (/home/mnzs/tmp/turborepo-remote-cache/node_modules/.pnpm/google-auth-library@8.4.0/node_modules/google-auth-library/build/src/auth/oauth2client.js:152:24)\n    at JWT.getRequestMetadataAsync (/home/mnzs/tmp/turborepo-remote-cache/node_modules/.pnpm/google-auth-library@8.4.0/node_modules/google-auth-library/build/src/auth/oauth2client.js:284:28)"

This occour because I use base64 credentials of gcp, so I change that to json and pass in env vars and works locally.

But some error ocurring in Github Actions and I can't access real error, exists another way to get erros on storage error? Or another exemple to use this solution with Github Actions?

mnzsss commented 1 year ago

I found the error, the github actions don't parse json secrets. I need pass the value directly.

If anyone need that:

name: Build and Deploy
on: [push]
jobs:
  build:
    name: Build and Deploy
    runs-on: ubuntu-latest

    services:
      turbo_cache:
        image: fox1t/turborepo-remote-cache
        env:
          NODE_ENV: production
          PORT: 3000
          TURBO_TOKEN: XXX
          STORAGE_PROVIDER: google-cloud-storage
          STORAGE_PATH: <storage-path>
          GCS_PROJECT_ID: <project-id>
          GCS_CLIENT_EMAIL: <email>
          GCS_PRIVATE_KEY: ${{ secrets.GCP_CREDENTIALS_PRIVATE_KEY }}
        ports:
          - 3000:3000