Support AWS S3 (and other Object Stores) as a Remote Cache

[dcherman]

Describe the feature you'd like to request

Rather than requiring an HTTP endpoint be used with remote caching, can we consider directly reading from/writing to an S3 bucket? That helps avoid needing to support yet another service that requires authentication/authorization and allows you to use your existing auth with your Cloud Provider whether it's static access keys, SSO, OIDC (from github actions for example) or something else.

Describe the solution you'd like

Adding support for something like turbo --cache-store=s3 --s3-bucket=my-bucket-name --s3-key-prefix="/foo/bar" which would read/write directly from an S3 bucket (or something API compatible with an S3 bucket) for a remote cache rather than an HTTP endpoint.

Describe alternatives you've considered

I've implemented this locally with a service that implements the HTTP API but reads/writes to S3 using the credentials of the user running the service, but it would be nice to have this built into turborepo since it seems like it'd be generally useful for others as well.

[samchouse]

Maybe plugins would be useful, kind of like Terraform. Users could use plugins and make their own for custom caches.

[weyert]

If there would be a way to disable changing this configuration on developer machines. Personally trying to avoid needing local access to block storage.

[samchouse]

It would be opt in, for most vercel remote cache is fine.

[ragrag]

I like the idea of plugins @Xenfo mentioned

[polRk]

It would be opt in, for most vercel remote cache is fine.

Maybe tomorrow, vercel may not available for me or for you. Linking to a specific platform rather than a solution is always a bad thing

[anthonyshew]

Turborepo can hook up to any Remote Cache that fulfills the requirements of the Remote Caching API. Many implementations are just a search away!