ducktors / turborepo-remote-cache

Open source implementation of the Turborepo custom remote cache server.
https://ducktors.github.io/turborepo-remote-cache/
MIT License
1.03k stars 95 forks source link

Help request - configured with s3, issue in the server log #168

Closed imgalli closed 1 year ago

imgalli commented 1 year ago

Hi there, we're trying to setup a remote cache using AWS S3 and can see the following error in the log relating to the line of code below. Note using k8s, this image - ducktors/turborepo-remote-cache:1.14.1.

{"severity":"WARNING","level":40,"time":1684164667390,"pid":7,"hostname":"turborepo-cache-57d6c996db-m8ktx","reqId":"WmEucNymQiefEGn9CLNAYw-0","data":null,"isBoom":true,"isServer":false,"output":{"statusCode":400,"payload":{"statusCode":400,"error":"Bad Request","message":"Missing Authorization header"},"headers":{}},"stack":"Error: Missing Authorization header\n    at Object.<anonymous> (/home/app/node/src/plugins/remote-cache/index.ts:56:27)\n    at hookIterator (/home/app/node/node_modules/.pnpm/fastify@4.12.0/node_modules/fastify/lib/hooks.js:246:10)\n    at next (/home/app/node/node_modules/.pnpm/fastify@4.12.0/node_modules/fastify/lib/hooks.js:174:16)\n    at hookRunner (/home/app/node/node_modules/.pnpm/fastify@4.12.0/node_modules/fastify/lib/hooks.js:196:3)\n    at Object.routeHandler [as handler] (/home/app/node/node_modules/.pnpm/fastify@4.12.0/node_modules/fastify/lib/route.js:469:7)\n    at Router.callHandler (/home/app/node/node_modules/.pnpm/find-my-way@7.6.0/node_modules/find-my-way/index.js:398:14)\n    at Router.lookup (/home/app/node/node_modules/.pnpm/find-my-way@7.6.0/node_modules/find-my-way/index.js:376:17)\n    at Server.preRouting (/home/app/node/node_modules/.pnpm/fastify@4.12.0/node_modules/fastify/fastify.js:773:14)\n    at Server.emit (node:events:513:28)\n    at Server.emit (node:domain:489:12)","type":"Error","message":"Missing Authorization header"}

https://github.com/ducktors/turborepo-remote-cache/blob/main/src/plugins/remote-cache/index.ts#L56

The following env vars are set, key and secret via ~/.aws.

STORAGE_PROVIDER - s3
STORAGE_PATH - something
AWS_REGION - eu-west-2
S3_ENDPOINT - https://XXX.s3.eu-west-2.amazonaws.com

This is a result of trying to run a command against this server, local command being

turbo run ci:lint -vvv --remote-only

with .turbo/config.json of

{
  "teamId": "team_myteam",
  "apiUrl": "XXXXX",
  "token": "XXXXX"
}

Won't lie, not too sure what aspect of this is unhappy but expect it's something to do with our config :) any help would be appreciated please!

imgalli commented 1 year ago

Any thoughts @fox1t ?

fox1t commented 1 year ago

Hey. The project config file doesn’t support passing the token. This is a turbo limitation. You should use one of the 3 methods described here https://ducktors.github.io/turborepo-remote-cache/custom-remote-caching#config-file

imgalli commented 1 year ago

Hey @fox1t , thanks for the reply!

We tried using only ENV vars listed here and had the same result - https://ducktors.github.io/turborepo-remote-cache/custom-remote-caching#local-environment-variables

lperrious commented 1 year ago

Hey there ! Same error on our side.

fox1t commented 1 year ago

@lperrious hey! Can you please paste your turbo version and the output that says if the cache is enabled?

lperrious commented 1 year ago

Thanks for the reply @fox1t ! Turbo version is 1.10.3 and I get "Remote caching enabled" in my turbo log. I cleaned all local cache to be sure it tries to fetch on the server however I end up with that "Bad Request, Missing Authorization header" response.

lperrious commented 1 year ago

I had some time to work on it again. It turns out that the TURBO_TOKEN was causing problems due to a special character. It works perfectly now. Thanks for your work !