search

bazelbuild / bazel

a fast, scalable, multi-language and extensible build system
https://bazel.build
Apache License 2.0
23.22k stars 4.07k forks source link

Feature Request: Run custom command to download external dependency #19267

Open gonzojive opened 1 year ago

gonzojive commented 1 year ago

Description of the feature request:

I want to use a command like gsutil cp gs://my-bucket/thing.tar /tmp/thing.tar to download the @thing dependency in my workspace. I'd like this to work with the repository cache.

Which category does this issue belong to?

External Dependency

What underlying problem are you trying to solve with this feature?

Two use cases:

1) Using gsutil to download GCS files as external dependencies in a bazel workspace in a way that is just as performant as http_archive. 2) Using Coursier with a a private [Google Artifact Registry for maven]() and authenticating with Workload Identity Federation. This can seemingly be done with a plugin to Coursier that handles downloading from artifactregistry:// URLs.

Which operating system are you running Bazel on?

No response

What is the output of bazel info release?

No response

If bazel info release returns development version or (@non-git), tell us how you built Bazel.

No response

What's the output of git remote get-url origin; git rev-parse master; git rev-parse HEAD ?

No response

Have you found anything relevant by searching the web?

https://bazel.build/rules/lib/builtins/repository_ctx#download documents the only mechanism for adding entries to the repository cache using repository_ctx: By downloding them from URLs.

Related issues:

Any other information, logs, or outputs that you want to share?

No response

Wyverald commented 1 year ago

We're currently working on the "true" repository cache: https://github.com/bazelbuild/bazel/issues/12227 This is not exactly what you're asking for, but I imagine it would solve the issue nonetheless?

github-actions[bot] commented 2 weeks ago

Thank you for contributing to the Bazel repository! This issue has been marked as stale since it has not had any activity in the last 1+ years. It will be closed in the next 90 days unless any other activity occurs. If you think this issue is still relevant and should stay open, please post any comment here and the issue will no longer be marked as stale.

malt3 commented 2 weeks ago

For your specific problem there is now rules_gcs. It does use the repository cache. The true repository cache for a more general solution is still being worked on in #12227.