search

bazel-contrib / rules_oci

Bazel rules for building OCI containers
Apache License 2.0
305 stars 159 forks source link

v2.0.0 on Windows #714

Open lexspoon opened 1 month ago

lexspoon commented 1 month ago

I'm running into two problems trying to use rules_oci 2.0.0 on Windows.

  1. It fails to download bsdtar, receiving a 404 code. This can be fixed by upgrading aspect_bazel_lib.
  2. It fails to access zstd. It's pulling zstd out of aspect_bazel_lib, but that module doesn't support Windows for zstd.

A workaround for zstd would be appreciated if anyone has one. As things stand, I can't upgrade due to this problem on Windows, and I can't use the previous version due to a problem with MacOS (https://github.com/bazel-contrib/rules_oci/issues/509).

As an idea for a workaround, would it be possible for me to register a zstd toolchain in my own build files? I see that Meta provides an exe for Windows. It's in a zip file, but it seems like sufficient Bazel-fu should be able to turn that into a toolchain. It's beyond me, though.

For a real fix:

thesayyn commented 1 month ago

Perhaps use the official zstd module from Facebook? I don't know the trade offs, but it seems odd for rules_oci to pull in zstd from aspect_bazel_lib.

facebook/zstd does not publish prebuilt binaries which we need for rules_oci to work fast. Building it from source is not an option for us as building from source breaks eventually due to differences in cc toolchains.

https://github.com/facebook/zstd/releases/tag/v1.5.6

What we do instead, is to build it from source with every release and publish it to github releases ourselves, see here https://github.com/aspect-build/zstd-prebuilt/releases

thesayyn commented 1 month ago

Can zstd be optional? If you publish a 2.0.1 with optional zstd, that would allow me to upgrade on Windows.

No, zstd is required for rules_oci to work

lexspoon commented 1 month ago

I understand these things about how it currently works and was looking for ways to repair the Windows support. I agree about the advantages of the pre-built binaries, although there are also clearly downsides, as this ticket shows.

Using the local CC is certainly a very common option with Bazel, so most users ought to have a working CC. There are 215 modules that depend on rules_cc!

https://registry.bazel.build/modules/rules_cc/0.0.9

thesayyn commented 1 month ago

Sorry, we have seen build-from-source fail miserably countless times it's simply not an option. If it's something we add support here the. prebuilt binaries are required, and build from source is a nice to have.

If you you'd like to workaround this, you can simply add dependency on zstd from BCR and register the windows toolchain from your workspace.

sluongng commented 1 month ago

may be I am missing something here but isn't the windows prebuilt binaries included in https://github.com/facebook/zstd/releases/tag/v1.5.6 as zstd-v1.5.6-win32.zip and zstd-v1.5.6-win64.zip? 🤔