Vendor mode does not include enough content to run a MODULE-based build without internet

peakschris commented 5 months ago

Description of the bug:

Given a build with all internet repos defined in MODULE.bazel, running bazel vendor does not vendor enough content to subsequently run bazel without an internet connection.

/// Machine with internet access
<setup module-based build>
bazel vendor --vendor_dir=vendor

/// Machine without internet access
<setup module-based build>
bazel build --vendor_dir=vendor
<fails with errors about missing repos and missing bcr>

My closest (but still failing) attempt so far is:

/// Machine with internet access
<setup module-based build>
bazel vendor --vendor_dir=/offline/vendor
<prune unnecessary platforms from vendor dir>
bazel fetch @remotejdk_win/... --repository_cache=/offline/cache
cd src/build/offline/empty_workspace
bazel clean
bazel sync --repository_cache=/offline/cache
<download bazel-central-registry to /offline/bazel-central-registry>

/// Machine without internet access
bazel build //x --vendor_dir=/offline/vendor --repository_cache=/offline/cache --registry=file://offline/bazel-central-registry --nofetch
ERROR: Analysis of target '//x' failed; build aborted: to fix, run
        bazel fetch //...
External repository @@local_config_sh not found and fetching repositories is disabled.

Even if this worked, this is clunky, was hard to figure out, and the resulting set of directories is hard to manage across multiple build versions of our product. The ask is that all required internet content for a build that exclusively uses modules should be vendored by the vendor command.

I don't know if this is a bug or feature request, please assign as appropriate.

Which category does this issue belong to?

External Dependency

What's the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.

No response

Which operating system are you running Bazel on?

windows

What is the output of `bazel info release`?

release-7.2.0 latest (8c6220a19b65dac3e675a97a59a146b8e294c404)

If `bazel info release` returns `development version` or `(@non-git)`, tell us how you built Bazel.

No response

What's the output of `git remote get-url origin; git rev-parse HEAD` ?

No response

Is this a regression? If yes, please try to identify the Bazel commit where the bug was introduced.

No response

Have you found anything relevant by searching the web?

No response

Any other information, logs, or outputs that you want to share?

No response

peakschris commented 5 months ago

Update: Even the sequence of steps above fails, with this error. I'm stuck!

/// Machine without internet access
bazel build //x --vendor_dir=/offline/vendor --repository_cache=/offline/cache --registry=file://offline/bazel-central-registry --nofetch
ERROR: Analysis of target '//x' failed; build aborted: to fix, run
        bazel fetch //...
External repository @@local_config_sh not found and fetching repositories is disabled.

meteorcloudy commented 5 months ago

--nofetch is not the accurate way to test offline build, because some repo can be fetched without internet access, eg. local_config_sh. You should probably use --repository_disable_download to test the build or just set HTTP_PROXY=none and HTTPS_PROXY=none. The former makes sure the repository_ctx.download API cannot be used, the latter may also apply to tools called by repository.execute.

meteorcloudy commented 4 months ago

https://github.com/bazelbuild/bazel/commit/96b90baf08172bb55844c2366111a0c39b6f9e14 should make vendor mode fully offline. Please try with Bazel built @ HEAD

AlexanderAA commented 2 months ago

I created a new issue https://github.com/bazelbuild/bazel/issues/23243 and then found this one as well...

I do not think vendor mode is fully offline at the moment.

bazelbuild / bazel