Error applying patches with patches attribute in crate.annotation rule

[tetsuok]

crate.annotation rule supports patches and patch_args attributes, but applying patch failed. See error message below. It would be great if there is something wrong with the configuration (see below).

git clone git@github.com:tetsuok/rules_rust_cargo_universe_hashbrown.git
cd rules_rust_cargo_universe_hashbrown
bazel build //...

Output:

Loading: 
Loading: 0 packages loaded
Analyzing: target //:hello (1 packages loaded, 0 targets configured)
INFO: Repository crate_index__ahash-0.7.6 instantiated at:
  /home/t/rules_rust_cargo_universe_hashbrown/WORKSPACE:52:19: in <toplevel>
  /home/t/.cache/bazel/_bazel_t/4ba20afb3d7a89a1d3a105467b78f4a5/external/crate_index/defs.bzl:380:10: in crate_repositories
  /home/t/.cache/bazel/_bazel_t/4ba20afb3d7a89a1d3a105467b78f4a5/external/bazel_tools/tools/build_defs/repo/utils.bzl:233:18: in maybe
Repository rule http_archive defined at:
  /home/t/.cache/bazel/_bazel_t/4ba20afb3d7a89a1d3a105467b78f4a5/external/bazel_tools/tools/build_defs/repo/http.bzl:355:31: in <toplevel>
INFO: Repository 'crate_index__ahash-0.7.6' used the following cache hits instead of downloading the corresponding file.
 * Hash 'fcb51a0695d8f838b1ee009b3fbf66bda078cd64590202a864a8f3e8c4315c47' for https://crates.io/api/v1/crates/ahash/0.7.6/download
If the definition of 'crate_index__ahash-0.7.6' was updated, verify that the hashes were also updated.
ERROR: An error occurred during the fetch of repository 'crate_index__ahash-0.7.6':
   Traceback (most recent call last):
    File "/home/t/.cache/bazel/_bazel_t/4ba20afb3d7a89a1d3a105467b78f4a5/external/bazel_tools/tools/build_defs/repo/http.bzl", line 136, column 10, in _http_archive_impl
        patch(ctx, auth = auth)
    File "/home/t/.cache/bazel/_bazel_t/4ba20afb3d7a89a1d3a105467b78f4a5/external/bazel_tools/tools/build_defs/repo/utils.bzl", line 167, column 22, in patch
        ctx.patch(patchfile, strip)
Error in patch: Error applying patch /home/t/rules_rust_cargo_universe_hashbrown/ahash.patch: Incorrect Chunk: the chunk content doesn't match the target
**Original Position**: 69

**Original Content**:
const-random = { version = "0.1.12", optional = true }
serde = { version = "1.0.117", optional = true }

[target.'cfg(not(any(target_os = "linux", target_os = "android", target_os = "windows", target_os = "macos", target_os = "ios", target_os = "freebsd", target_os = "openbsd", target_os = "netbsd", target_os = "dragonfly", target_os = "solaris", target_os = "illumos", target_os = "fuchsia", target_os = "redox", target_os = "cloudabi", target_os = "haiku", target_os = "vxworks", target_os = "emscripten", target_os = "wasi")))'.dependencies]
const-random = { version = "0.1.12", optional = true }
serde = { version = "1.0.117", optional = true }

[target.'cfg(not(all(target_arch = "arm", target_os = "none")))'.dependencies]
once_cell = { version = "1.8", default-features = false, features = ["alloc"] }

**Revised Content**:
const-random = { version = "0.1.12", optional = true }
serde = { version = "1.0.117", optional = true }

# [target.'cfg(not(any(target_os = "linux", target_os = "android", target_os = "windows", target_os = "macos", target_os = "ios", target_os = "freebsd", target_os = "openbsd", target_os = "netbsd", target_os = "dragonfly", target_os = "solaris", target_os = "illumos", target_os = "fuchsia", target_os = "redox", target_os = "cloudabi", target_os = "haiku", target_os = "vxworks", target_os = "emscripten", target_os = "wasi")))'.dependencies]
# const-random = { version = "0.1.12", optional = true }
# serde = { version = "1.0.117", optional = true }

[target.'cfg(not(all(target_arch = "arm", target_os = "none")))'.dependencies]
once_cell = { version = "1.8", default-features = false, features = ["alloc"] }
ERROR: /home/t/rules_rust_cargo_universe_hashbrown/WORKSPACE:52:19: fetching http_archive rule //external:crate_index__ahash-0.7.6: Traceback (most recent call last):
    File "/home/t/.cache/bazel/_bazel_t/4ba20afb3d7a89a1d3a105467b78f4a5/external/bazel_tools/tools/build_defs/repo/http.bzl", line 136, column 10, in _http_archive_impl
        patch(ctx, auth = auth)
    File "/home/t/.cache/bazel/_bazel_t/4ba20afb3d7a89a1d3a105467b78f4a5/external/bazel_tools/tools/build_defs/repo/utils.bzl", line 167, column 22, in patch
        ctx.patch(patchfile, strip)
Error in patch: Error applying patch /home/t/rules_rust_cargo_universe_hashbrown/ahash.patch: Incorrect Chunk: the chunk content doesn't match the target
**Original Position**: 69

**Original Content**:
const-random = { version = "0.1.12", optional = true }
serde = { version = "1.0.117", optional = true }

[target.'cfg(not(any(target_os = "linux", target_os = "android", target_os = "windows", target_os = "macos", target_os = "ios", target_os = "freebsd", target_os = "openbsd", target_os = "netbsd", target_os = "dragonfly", target_os = "solaris", target_os = "illumos", target_os = "fuchsia", target_os = "redox", target_os = "cloudabi", target_os = "haiku", target_os = "vxworks", target_os = "emscripten", target_os = "wasi")))'.dependencies]
const-random = { version = "0.1.12", optional = true }
serde = { version = "1.0.117", optional = true }

[target.'cfg(not(all(target_arch = "arm", target_os = "none")))'.dependencies]
once_cell = { version = "1.8", default-features = false, features = ["alloc"] }

**Revised Content**:
const-random = { version = "0.1.12", optional = true }
serde = { version = "1.0.117", optional = true }

# [target.'cfg(not(any(target_os = "linux", target_os = "android", target_os = "windows", target_os = "macos", target_os = "ios", target_os = "freebsd", target_os = "openbsd", target_os = "netbsd", target_os = "dragonfly", target_os = "solaris", target_os = "illumos", target_os = "fuchsia", target_os = "redox", target_os = "cloudabi", target_os = "haiku", target_os = "vxworks", target_os = "emscripten", target_os = "wasi")))'.dependencies]
# const-random = { version = "0.1.12", optional = true }
# serde = { version = "1.0.117", optional = true }

[target.'cfg(not(all(target_arch = "arm", target_os = "none")))'.dependencies]
once_cell = { version = "1.8", default-features = false, features = ["alloc"] }
ERROR: /home/t/.cache/bazel/_bazel_t/4ba20afb3d7a89a1d3a105467b78f4a5/external/crate_index__hashbrown-0.12.3/BUILD.bazel:34:13: @crate_index__hashbrown-0.12.3//:hashbrown depends on @crate_index__ahash-0.7.6//:ahash in repository @crate_index__ahash-0.7.6 which failed to fetch. no such package '@crate_index__ahash-0.7.6//': Error applying patch /home/t/rules_rust_cargo_universe_hashbrown/ahash.patch: Incorrect Chunk: the chunk content doesn't match the target
**Original Position**: 69

**Original Content**:
const-random = { version = "0.1.12", optional = true }
serde = { version = "1.0.117", optional = true }

[target.'cfg(not(any(target_os = "linux", target_os = "android", target_os = "windows", target_os = "macos", target_os = "ios", target_os = "freebsd", target_os = "openbsd", target_os = "netbsd", target_os = "dragonfly", target_os = "solaris", target_os = "illumos", target_os = "fuchsia", target_os = "redox", target_os = "cloudabi", target_os = "haiku", target_os = "vxworks", target_os = "emscripten", target_os = "wasi")))'.dependencies]
const-random = { version = "0.1.12", optional = true }
serde = { version = "1.0.117", optional = true }

[target.'cfg(not(all(target_arch = "arm", target_os = "none")))'.dependencies]
once_cell = { version = "1.8", default-features = false, features = ["alloc"] }

**Revised Content**:
const-random = { version = "0.1.12", optional = true }
serde = { version = "1.0.117", optional = true }

# [target.'cfg(not(any(target_os = "linux", target_os = "android", target_os = "windows", target_os = "macos", target_os = "ios", target_os = "freebsd", target_os = "openbsd", target_os = "netbsd", target_os = "dragonfly", target_os = "solaris", target_os = "illumos", target_os = "fuchsia", target_os = "redox", target_os = "cloudabi", target_os = "haiku", target_os = "vxworks", target_os = "emscripten", target_os = "wasi")))'.dependencies]
# const-random = { version = "0.1.12", optional = true }
# serde = { version = "1.0.117", optional = true }

[target.'cfg(not(all(target_arch = "arm", target_os = "none")))'.dependencies]
once_cell = { version = "1.8", default-features = false, features = ["alloc"] }
ERROR: Analysis of target '//:hello' failed; build aborted: 
INFO: Elapsed time: 1.459s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (71 packages loaded, 437 targets configured)
FAILED: Build did NOT complete successfully (71 packages loaded, 437 targets configured)

Below is a set of related files on this issues. For the full contents, see https://github.com/tetsuok/rules_rust_cargo_universe_hashbrown.

WORKSPACE:

load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")

http_archive(
    name = "rules_rust",
    sha256 = "6bfe75125e74155955d8a9854a8811365e6c0f3d33ed700bc17f39e32522c822",
    urls = [
        "https://github.com/bazelbuild/rules_rust/releases/download/0.9.0/rules_rust-v0.9.0.tar.gz",
    ],
)

load("@rules_rust//rust:repositories.bzl", "rules_rust_dependencies", "rust_register_toolchains")

rules_rust_dependencies()

rust_register_toolchains(
    edition = "2021",
    version = "1.63.0",
)

load("@rules_rust//crate_universe:repositories.bzl", "crate_universe_dependencies")

crate_universe_dependencies()

load("@rules_rust//crate_universe:defs.bzl", "crate", "crates_repository", "render_config")

# after changing:
#   CARGO_BAZEL_REPIN=1 bazel sync --only=crate_index
crates_repository(
    name = "crate_index",
    annotations = {
        "ahash": [crate.annotation(
            patch_args = ["-p1"],
            patches = ["@//:ahash.patch"],
            version = "=0.7.6",
        )],
    },
    cargo_lockfile = "//:Cargo.Bazel.lock",
    lockfile = "//:cargo-bazel-lock.json",
    packages = {
        "hashbrown": crate.spec(
            version = "0.12.3",
        ),
    },
    render_config = render_config(
        default_package_name = "",
    ),
    supported_platform_triples = ["x86_64-unknown-linux-gnu"],
)

load("@crate_index//:defs.bzl", "crate_repositories")

crate_repositories()

ahash.patch:

diff --git a/Cargo.toml b/Cargo.toml
index 4b56472..5847f94 100644
--- a/Cargo.toml
+++ b/Cargo.toml
@@ -69,9 +69,9 @@ getrandom = { version = "0.2.3" }
 const-random = { version = "0.1.12", optional = true }
 serde = { version = "1.0.117", optional = true }

-[target.'cfg(not(any(target_os = "linux", target_os = "android", target_os = "windows", target_os = "macos", target_os = "ios", target_os = "freebsd", target_os = "openbsd", target_os = "netbsd", target_os = "dragonfly", target_os = "solaris", target_os = "illumos", target_os = "fuchsia", target_os = "redox", target_os = "cloudabi", target_os = "haiku", target_os = "vxworks", target_os = "emscripten", target_os = "wasi")))'.dependencies]
-const-random = { version = "0.1.12", optional = true }
-serde = { version = "1.0.117", optional = true }
+# [target.'cfg(not(any(target_os = "linux", target_os = "android", target_os = "windows", target_os = "macos", target_os = "ios", target_os = "freebsd", target_os = "openbsd", target_os = "netbsd", target_os = "dragonfly", target_os = "solaris", target_os = "illumos", target_os = "fuchsia", target_os = "redox", target_os = "cloudabi", target_os = "haiku", target_os = "vxworks", target_os = "emscripten", target_os = "wasi")))'.dependencies]
+# const-random = { version = "0.1.12", optional = true }
+# serde = { version = "1.0.117", optional = true }

 [target.'cfg(not(all(target_arch = "arm", target_os = "none")))'.dependencies]
 once_cell = { version = "1.8", default-features = false, features = ["alloc"] }

BUILD.bazel:

load("@rules_rust//rust:defs.bzl", "rust_binary")

rust_binary(
    name = "hello",
    srcs = ["hello.rs"],
    deps = [
        "@crate_index//:hashbrown",
    ],
)

hello.rs:

fn main() {
    println!("Hello World!");
}

Note that to use hashbrown 0.12.3 (latest as of writing) in crate_universe rule, the ahash crate 0.7.6 needs to be patched otherwise build fails with the following message:

ERROR: Traceback (most recent call last):
        File "/home/t/.cache/bazel/_bazel_t/4ba20afb3d7a89a1d3a105467b78f4a5/external/crate_index__ahash-0.7.6/BUILD.bazel", line 51, column 10, in <toplevel>
                ): [
Error: dictionary expression has duplicate key: ("@rules_rust//rust/platform:x86_64-unknown-linux-gnu",)
ERROR: /home/t/.cache/bazel/_bazel_t/4ba20afb3d7a89a1d3a105467b78f4a5/external/crate_index__hashbrown-0.12.3/BUILD.bazel:34:13: no such target '@crate_index__ahash-0.7.6//:ahash': target 'ahash' not declared in package '' defined by /home/t/.cache/bazel/_bazel_t/4ba20afb3d7a89a1d3a105467b78f4a5/external/crate_index__ahash-0.7.6/BUILD.bazel and referenced by '@crate_index__hashbrown-0.12.3//:hashbrown'
ERROR: Analysis of target '//:hello' failed; build aborted: 
INFO: Elapsed time: 1.936s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (17 packages loaded, 157 targets configured)

Below is a generated crate_index__ahash-0.7.6/BUILD.bazel. Maybe ahash crate 0.8.0 doesn't have issue, but hashbrown 0.12.3 crate specifies ahash 0.7.0 as a dependency in Cargo.toml

crate_index__ahash-0.7.6/BUILD.bazel:

###############################################################################
# @generated
# DO NOT MODIFY: This file is auto-generated by a crate_universe tool. To 
# regenerate this file, run the following:
#
#     bazel sync --only=crate_index
###############################################################################

load(
    "@bazel_skylib//lib:selects.bzl", 
    "selects",
)
load(
    "@rules_rust//cargo:defs.bzl",
    "cargo_build_script",
)
load(
    "@rules_rust//rust:defs.bzl",
    "rust_binary",
    "rust_library",
    "rust_proc_macro",
)

# buildifier: disable=bzl-visibility
load("@rules_rust//crate_universe/private:selects.bzl", "select_with_or")

package(default_visibility = ["//visibility:public"])

# licenses([
#     "TODO",  # MIT OR Apache-2.0
# ])

rust_library(
    name = "ahash",
    deps = [
    ] + select_with_or({
        # cfg(any(target_os = "linux", target_os = "android", target_os = "windows", target_os = "macos", target_os = "ios", target_os = "freebsd", target_os = "openbsd", target_os = "netbsd", target_os = "dragonfly", target_os = "solaris", target_os = "illumos", target_os = "fuchsia", target_os = "redox", target_os = "cloudabi", target_os = "haiku", target_os = "vxworks", target_os = "emscripten", target_os = "wasi"))
        (
            "@rules_rust//rust/platform:x86_64-unknown-linux-gnu",
        ): [
            # Target Deps
            "@crate_index__getrandom-0.2.7//:getrandom",

            # Common Deps
            "@crate_index__ahash-0.7.6//:build_script_build",
        ],
        # cfg(not(all(target_arch = "arm", target_os = "none")))
        (
            "@rules_rust//rust/platform:x86_64-unknown-linux-gnu",
        ): [
            # Target Deps
            "@crate_index__once_cell-1.13.1//:once_cell",

            # Common Deps
            "@crate_index__ahash-0.7.6//:build_script_build",
        ],
        "//conditions:default": [
            "@crate_index__ahash-0.7.6//:build_script_build",
        ],
    }),
    proc_macro_deps = [
    ] + select_with_or({
        "//conditions:default": [
        ],
    }),
    aliases = selects.with_or({
        "//conditions:default": {
        },
    }),
    compile_data = glob(include = ["**"], exclude = ["BUILD", "BUILD.bazel", "WORKSPACE", "WORKSPACE.bazel"]) + select_with_or({
        "//conditions:default": [
        ],
    }),
    crate_root = "src/lib.rs",
    crate_features = [
    ],
    data = select_with_or({
        "//conditions:default": [
        ],
    }),
    edition = "2018",
    rustc_env = {
    },
    rustc_env_files = select_with_or({
        "//conditions:default": [
        ],
    }),
    rustc_flags = [
        # In most cases, warnings in 3rd party crates are not interesting as
        # they're out of the control of consumers. The flag here silences 
        # warnings. For more details see: 
        # https://doc.rust-lang.org/rustc/lints/levels.html
        "--cap-lints=allow",
    ],
    srcs = glob(
        include = [
            "**/*.rs",
        ],
        exclude = [
        ],
    ),
    version = "0.7.6",
    tags = [
        "cargo-bazel",
        "manual",
        "noclippy",
        "norustfmt",
    ],

)

cargo_build_script(
    # See comment associated with alias. Do not change this name
    name = "ahash_build_script",
    aliases = selects.with_or({
        "//conditions:default": {
        },
    }),
    build_script_env = {
    },
    compile_data = select_with_or({
        "//conditions:default": [
        ],
    }),
    crate_name = "build_script_build",
    crate_root = "build.rs",
    crate_features = [
    ],
    data = glob(["**"]) + select_with_or({
        "//conditions:default": [
        ],
    }),
    deps = [
    ] + select_with_or({
        "//conditions:default": [
            "@crate_index__version_check-0.9.4//:version_check",
        ],
    }),
    edition = "2018",
    proc_macro_deps = [
    ] + select_with_or({
        "//conditions:default": [
        ],
    }),
    rustc_env = {
    },
    rustc_env_files = select_with_or({
        "//conditions:default": [
        ],
    }),
    rustc_flags = [
        # In most cases, warnings in 3rd party crates are not interesting as
        # they're out of the control of consumers. The flag here silences 
        # warnings. For more details see: 
        # https://doc.rust-lang.org/rustc/lints/levels.html
        "--cap-lints=allow",
    ],
    srcs = glob(
        include = [
            "**/*.rs",
        ],
        exclude = [
        ],
    ),
    tools = select_with_or({
        "//conditions:default": [
        ],
    }),
    version = "0.7.6",
    tags = [
        "cargo-bazel",
        "manual",
        "noclippy",
        "norustfmt",
    ],
    visibility = ["//visibility:private"],
)
alias(
    # Because `cargo_build_script` does some invisible target name mutating to
    # determine the package and crate name for a build script, the Bazel
    # target namename of any build script cannot be the Cargo canonical name
    # of `build_script_build` without losing out on having certain Cargo
    # environment variables set.
    name = "build_script_build",
    actual = "ahash_build_script",
    tags = [
        "manual",
    ],
)

[UebelAndre]

Note that to use hashbrown 0.12.3 (latest as of writing) in crate_universe rule, the ahash crate 0.7.6 needs to be patched otherwise build fails with the following message:

ERROR: Traceback (most recent call last):
        File "/home/t/.cache/bazel/_bazel_t/4ba20afb3d7a89a1d3a105467b78f4a5/external/crate_index__ahash-0.7.6/BUILD.bazel", line 51, column 10, in <toplevel>
                ): [
Error: dictionary expression has duplicate key: ("@rules_rust//rust/platform:x86_64-unknown-linux-gnu",)
ERROR: /home/t/.cache/bazel/_bazel_t/4ba20afb3d7a89a1d3a105467b78f4a5/external/crate_index__hashbrown-0.12.3/BUILD.bazel:34:13: no such target '@crate_index__ahash-0.7.6//:ahash': target 'ahash' not declared in package '' defined by /home/t/.cache/bazel/_bazel_t/4ba20afb3d7a89a1d3a105467b78f4a5/external/crate_index__ahash-0.7.6/BUILD.bazel and referenced by '@crate_index__hashbrown-0.12.3//:hashbrown'
ERROR: Analysis of target '//:hello' failed; build aborted: 
INFO: Elapsed time: 1.936s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (17 packages loaded, 157 targets configured)

This is the same issue as https://github.com/bazelbuild/rules_rust/issues/1236. I would advise not setting that value and letting Bazel be aware of other targets until this kind of filtering is supported (https://github.com/bazelbuild/rules_rust/issues/1417).

[UebelAndre]

crates_repository(
    name = "crate_index",
    annotations = {
        "ahash": [crate.annotation(
            patch_args = ["-p1"],
            patches = ["@//:ahash.patch"],
            version = "=0.7.6",
        )],
    },
    cargo_lockfile = "//:Cargo.Bazel.lock",
    lockfile = "//:cargo-bazel-lock.json",
    packages = {
        "hashbrown": crate.spec(
            version = "0.12.3",
        ),
    },
    render_config = render_config(
        default_package_name = "",
    ),
    supported_platform_triples = ["x86_64-unknown-linux-gnu"],
)

Could you try changing "@//:ahash.patch" to expand the repository name of your workspace?

[tetsuok]

Thanks, but why does applying patches in cargo.annotation fail? I thought it should work in the same way as http_archive.

[UebelAndre]

Thanks, but why does applying patches in cargo.annotation fail? I thought it should work in the same way as http_archive.

Ah, I see, the thing you're trying to do won't work. By the time an http_archive is generated for the crate, the dependency graph has already been resolved. What you'd want to do instead is somehow set [patch.crates-io] or something in the manifest crates_repository uses to query the dependency graph.

[tetsuok]

@UebelAndre Thanks for the suggestion. I'm now wondering why crate.annotation rule supports the 'patches', 'patch_args', and 'patch_tool' attributes. Cargo Universe is fantastic to enable "Cargo free" set up. It would be great to clarify the intended use cases of the attributes.

[tetsuok]

crates_repository(
    name = "crate_index",
    annotations = {
        "ahash": [crate.annotation(
            patch_args = ["-p1"],
            patches = ["@//:ahash.patch"],
            version = "=0.7.6",
        )],
    },
    cargo_lockfile = "//:Cargo.Bazel.lock",
    lockfile = "//:cargo-bazel-lock.json",
    packages = {
        "hashbrown": crate.spec(
            version = "0.12.3",
        ),
    },
    render_config = render_config(
        default_package_name = "",
    ),
    supported_platform_triples = ["x86_64-unknown-linux-gnu"],
)

Could you try changing "@//:ahash.patch" to expand the repository name of your workspace?

Sorry, I missed your prompt response. I just added the workspace name (commit https://github.com/tetsuok/rules_rust_cargo_universe_hashbrown/commit/719a5d0cff1daf7b8ab01a7d3e576bfff60dde6a), but applying the patch still fails, unfortunately.

[UebelAndre]

@UebelAndre Thanks for the suggestion. I'm now wondering why crate.annotation rule supports the 'patches', 'patch_args', and 'patch_tool' attributes. Cargo Universe is fantastic to enable "Cargo free" set up. It would be great to clarify the intended use cases of the attributes.

Rust has it's own build system and public registry where developers can publish versions of their code using said build system. 99.9% of the available libraries you could use in a project are built and tested using Cargo. The way crate_universe works is by relying on Cargo to go describe dependency graphs and where to download crates from so that it can generate some repository rules and BUILD files to represent the graph. Cargo still plays a part here but ideally users won't have to think about that if they don't want to.

With that said, the patches functionality of crates.annotation is specifically for the generated http_archive/git_repository rules so that developers can modify code (not dependency graphs) in case the dependency (originally built and tested with Cargo) does not work with Bazel or your uses.

The fix here is likely to add a patch section to crates_repository which would allow you to pass the same info to Bazel (and ultimately Cargo) that you'd be able to in a standard Cargo.toml file.

Alternatively, what you could do if you needed this working today, was to make a fork of the dependency you want and change, make a commit with the patches, and pull the project in via git.

Does this address any confusion you had and help get you unblocked?

[tetsuok]

Yes, thank you for the detailed explanation! I think I understand your point.