buchgr / bazel-remote

A remote cache for Bazel
https://bazel.build
Apache License 2.0
595 stars 154 forks source link

fix http regex string to add support for cas.v2 http requests #663

Open HunterCrosland opened 1 year ago

HunterCrosland commented 1 year ago

Adds support to http.go regex string to allow cas.v2 in http request

HunterCrosland commented 1 year ago

I am not sure why this is failing HTML requests, but we are getting error responses locally on latest container version.

CURL requests to valid SHAs using /cas/SHA returns resource name must be a SHA256 hash in hex. got '/cas.v2/SHA'

I can investigate this more next week.

HunterCrosland commented 1 year ago

I am not sure why this is failing HTML requests, but we are getting error responses locally on latest container version.

CURL requests to valid SHAs using /cas/SHA returns resource name must be a SHA256 hash in hex. got '/cas.v2/SHA'

I can investigate this more next week.

Realized regex match groups got messed up, fixed it.

HunterCrosland commented 1 year ago

Thanks for the contribution- I assume you would like this behaviour because you're chaining two bazel-remote instances with the http proxy backend? If that is the case, I wonder if it would be better to add a new option to the http proxy backend instead?

You are spot-on with the why -- and I really don't mind how. I am not entirely familiar with the routing on the backend, so that's probably something I will need to do over a weekend. In the meantime for our CI we've rolled back to a previous image version built prior to cas.v2 support. I built an image based on the changes I made here but it didn't have the correct entrypoint by default, and that's going to be another weekend issue.

mostynb commented 1 year ago

This used to work out of the box before bazel-remote had a compressed storage option (which is now the default). You should still be able to use a current build to chain together two bazel-remote instances with the http proxy backend by using --storage_mode uncompressed on the front end (and optionally on the back end).

If the front end uses the default --storage_mode zstd then it uploads compressed blobs using the /cas.v2/<sha256> path to distinguish them from regular/uncompressed blobs, and gives the error you probably saw. But if we know that the proxy backend can handle zstd compressed blobs correctly, then we could add a flag like --http_proxy.mode bazel-remote2 to make it upload both compressed and uncompressed blobs to /cas/<sha256> paths. I like this idea, because it doesn't expand the client-side API.

liam-baker-sm commented 3 days ago

--http_proxy.mode bazel-remote2 This would support our usecase as well. Bazel is now running --remote_cache_compression in production without error, so enabling compressed storage for the second bazel-remote instance would improve its capacity.