Tracking crates with exotic build.rs

[mfarrugi]

Moving https://github.com/acmcarther/cargo-raze/issues/16 over here.

Bazel needs some extra help with build.rs scripts that:

• link cc libs to *-sys crates
• use exotic cargo environment variables
• use compile time file includes
• produce --cfg flags
• and do other bespoke logic

Conveniently I haven't run into many additional problems, and our set of overrides hasn't grown.

[mfarrugi]

https://github.com/briansmith/ring/blob/master/build.rs is a doozy, it's upstream of actix-web :|

[acmcarther]

Got an initial read of that thing? Does it look like we can run it if we provide the right env vars?

[mfarrugi]

It might, wanted to keep track before I got the chance to take a deeper look.

[vitalyd]

A few notes from my attempt at looking at ring that @mfarrugi brought up:

• https://github.com/briansmith/ring/blob/master/build.rs#L270 looks for the CARGO_PKG_NAME envvar - this isn't set in the build script wrapper that cargo raze generates (but is set in bazel rust, for example: https://github.com/bazelbuild/rules_rust/blob/master/rust/toolchain.bzl#L23). I'm pretty sure the intended build path here is to end up calling the ring_build_rs_main() fn, rather than enter the pregeneration step via pregenerate_asm_main(). I hand edited cargo raze's ring_build_script_executor genrule to get past this point.

• Once I forced execution into ring_build_rs_main, we once again get blocked on missing envvars. In all, I had to add this to the same genrule:

  + " export CARGO_CFG_TARGET_ARCH=x86_64;"
      + " export CARGO_CFG_TARGET_OS=linux;"
      + " export CARGO_CFG_TARGET_ENV=;"
      + " export DEBUG=false;"
      + " export OPT_LEVEL=3;"
      + " export HOST=;"

  The latter 3 are needed by the gcc invocation. Apparently, cargo sets this all up for build scripts, and I suspect cargo raze will eventually need to reimplement a whole bunch (if not all) of this.

• After this, I hit include_bytes macro issues where the macro couldn't find the requested files, e.g. https://github.com/briansmith/ring/blob/d249aa57a08891a55254b303f628c87aeab84559/src/ec/suite_b/ecdsa/signing.rs#L350. The working dir is different and so it fails. I ran out of time to experiment further, and stopped here (for now).

HTH

[photex]

Howdy!

I didn't see any special handling for backtrace or backtrace-sys in the crater or other examples, but with my dependency on the failure crate I'm getting backtrace 0.3.9 and backtrace-sys 0.1.23 as transitive dependencies.

This version of backtrace-sys has been renovated compared to 0.1.16: https://github.com/alexcrichton/backtrace-rs/blob/master/backtrace-sys/build.rs

I'm unable to link or build any crate (aka all of my crates at the moment) due to missing symbols:

<snip gobs of linker output/>
undefined reference to `__rbt_backtrace_create_state'
undefined reference to `__rbt_backtrace_pcinfo'
undefined reference to `__rbt_backtrace_syminfo'
          collect2: error: ld returned 1 exit status

Adding the following to my raze Cargo.toml has had no effect at all:

[raze.crates.backtrace-sys.'0.1.23']
gen_buildrs = true

[mfarrugi]

This requires building it's c portion explicitly, we've done this internally I'll try to share it

On Wed, Aug 1, 2018, 09:42 Chip Collier notifications@github.com wrote:

Howdy!

I didn't see any special handling for backtrace or backtrace-sys in the crater or other examples, but with my dependency on the failure crate I'm getting backtrace 0.3.9 and backtrace-sys 0.1.23 as transitive dependencies.

This version of backtrace-sys has been renovated compared to 0.1.16:

https://github.com/alexcrichton/backtrace-rs/blob/master/backtrace-sys/build.rs

I'm unable to link or build any crate (aka all of my crates at the moment) due to missing symbols:

undefined reference to __rbt_backtrace_create_state' undefined reference torbt_backtrace_pcinfo' undefined reference to `rbt_backtrace_syminfo' collect2: error: ld returned 1 exit status

Adding the following to my raze Cargo.toml has had no effect at all:

[raze.crates.backtrace-sys.'0.1.23'] gen_buildrs = true

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/google/cargo-raze/issues/41#issuecomment-409578996, or mute the thread https://github.com/notifications/unsubscribe-auth/ACLjeKoayjaDkpHFR7SXXjkBUjvMpNvTks5uMbBfgaJpZM4TCCJs .

[photex]

I see. I'd just be adding a new rule more or less replicating the effect of the build.rs then? And does that require fooling with output from cargo-raze?

[mfarrugi]

@photex

cargo/Cargo.toml

[dependencies]
backtrace = "0.3.9"

# ... 

[raze.crates.backtrace.'0.3.9']
# Overriden to hand-written BUILD file, because libloading is building its own c portion.
skipped_deps = [
    "backtrace-sys-0.1.23",
]
additional_deps = [
    "//cargo/overrides/backtrace-sys-0.1.23:backtrace_sys",
]

cargo/overrides/backtrace-sys-0.1.23/ contains a copy of the crate, with the following added

cargo/overrides/backtrace-sys-0.1.23/BUILD which is a modification of the generated file

package(default_visibility=["//ext/public/rust/cargo:__subpackages__"])

licenses([
    "notice",  # "MIT,Apache-2.0"
])

load(
    "@io_bazel_rules_rust//rust:rust.bzl",
    "rust_library",
    "rust_binary",
    "rust_test",
    "rust_bench_test",
)

genrule(
    name="touch_config_header",
    outs=[
        "config.h",
    ],
    cmd="touch $@",
)

genrule(
    name="touch_backtrace_supported_header",
    outs=[
        "backtrace-supported.h",
    ],
    cmd="touch $@",
)

cc_library(
    name="backtrace_native",
    srcs=[
        ":touch_config_header",
        ":touch_backtrace_supported_header",
        "src/libbacktrace/internal.h",
        "src/libbacktrace/backtrace.h",
        "src/libbacktrace/alloc.c",
        "src/libbacktrace/dwarf.c",
        "src/libbacktrace/fileline.c",
        "src/libbacktrace/posix.c",
        "src/libbacktrace/read.c",
        "src/libbacktrace/sort.c",
        "src/libbacktrace/state.c",
        "src/libbacktrace/elf.c",
    ],
    includes=["."],
    copts=[
        "-fvisibility=hidden",
        "-fPIC",
    ],
    defines=[
        "BACKTRACE_ELF_SIZE=64",
        "BACKTRACE_SUPPORTED=1",
        "BACKTRACE_USES_MALLOC=1",
        "BACKTRACE_SUPPORTS_THREADS=0",
        "BACKTRACE_SUPPORTS_DATA=0",
        "HAVE_DL_ITERATE_PHDR=1",
        "_GNU_SOURCE=1",
        "_LARGE_FILES=1",
        "backtrace_full=__rbt_backtrace_full",
        "backtrace_dwarf_add=__rbt_backtrace_dwarf_add",
        "backtrace_initialize=__rbt_backtrace_initialize",
        "backtrace_pcinfo=__rbt_backtrace_pcinfo",
        "backtrace_syminfo=__rbt_backtrace_syminfo",
        "backtrace_get_view=__rbt_backtrace_get_view",
        "backtrace_release_view=__rbt_backtrace_release_view",
        "backtrace_alloc=__rbt_backtrace_alloc",
        "backtrace_free=__rbt_backtrace_free",
        "backtrace_vector_finish=__rbt_backtrace_vector_finish",
        "backtrace_vector_grow=__rbt_backtrace_vector_grow",
        "backtrace_vector_release=__rbt_backtrace_vector_release",
        "backtrace_close=__rbt_backtrace_close",
        "backtrace_open=__rbt_backtrace_open",
        "backtrace_print=__rbt_backtrace_print",
        "backtrace_simple=__rbt_backtrace_simple",
        "backtrace_qsort=__rbt_backtrace_qsort",
        "backtrace_create_state=__rbt_backtrace_create_state",
        "backtrace_uncompress_zdebug=__rbt_backtrace_uncompress_zdebug",
    ],
)

rust_library(
    name="backtrace_sys",
    crate_root="src/lib.rs",
    crate_type="lib",
    srcs=glob(["**/*.rs"]),
    deps=[
        "//cargo/vendor/libc-0.2.42:libc",
        ":backtrace_native",
    ],
    rustc_flags=[
        "--cap-lints allow",
        "--target=x86_64-unknown-linux-gnu",
    ],
    crate_features=[
    ],
)

This magical series of steps should be improved by #58

[photex]

Thanks for so quickly sharing your solution @mfarrugi. I would not have worked this out so fast on my own.

[mfarrugi]

Ran into new build scripts!

proc-macro2 lazy-static

Our solution was to hardcode the --cfg flags in the raze Cargo.toml that cargo would have come up with.

Edit: More precisely, we added:

[raze.crates.lazy_static.'1.1.0']
additional_flags = [
    "--cfg=lazy_static_inline_impl",
]

[raze.crates.proc-macro2.'0.4.19']
additional_flags = [
    "--cfg use_proc_macro",
]

[acmcarther]

I needed to link libsqlite3-sys, and found that it uses some interesting bindgen options that I couldn't replicate via bindgen executable flags:

https://github.com/jgallagher/rusqlite/blob/bd1756adeff7a67a13f00b39325301f5935d8267/libsqlite3-sys/build.rs#L204-L226

Specifically, it overrides the integer type inference via a callback passed to the bindgen library to cause the #define decls in sqlite3.h to be i32 instead of u32. I'll be needing to write a bindgen rule that uses the bindgen library internally rather than the binary.

[mainrs]

@acmcarther I am currently trying to get diesel to compile through bazel but I always get weird errors with the following message: error: linking with /bin/gcc failed: exit code 1. Not sure what I am actually doing wrong. IIRC diesel does use libsqlite3-sys for the sqlite3 feature. Do I have to do any additional configuration? I have added the gen_buildrs flag but it still fails :sob:

Did you somehow solve the problem?

[acmcarther]

@SirWindfield If your problem is the libsqlite3-sys, I can help with that, but you'll need to convert these instructions from rusqlite to diesel. This is also a pretty clunky process because I didn't actually write a bindgen rule, i just made this work manually:

1) First, I vendored sqlite3, with the following additions: A "bindgen runner" that invokes bindgen via Rust over sqlite3.h. Note that this would probably be better placed into libsqlite3-sys and run over wrapper.h, but I was having a hard time making bindgen's clang include path work correctly. The build for sqlite3 also generates an out_dir_tar for libsqlite3-sys

2) Next, I brought down libsqlite3-sys into a local directory so that I could override it's BUILD file. This overridden build file is identical to what raze produces, but it depends on my local sqlite3, and has sqlite3's out_tar as its out_dir_tar.

3. Finally, I overrode the libsqlite3-sys dependency for rustqlite with my local libsqlite3-sys

[mfarrugi]

For completeness, here's our current raze Cargo.toml:

third_party/rust/cargo/Cargo.toml

[package]
name = "cargo_via_bazel"
version = "0.0.0"

[lib]
path = "fake_lib.rs"

[dependencies]
# Mostly libraries recommended by https://github.com/brson/stdx
bitflags            = "1.0.4"
backtrace           = "0.3.9"
base64              = "0.9.0"
bindgen             = "0.40.0"
byteorder           = "1.2.6"
bytes               = "0.4.10"
chrono              = "0.4.6"
clap                = "2.32.0"
crossbeam-channel   = "0.2.6"
csv                 = "1.0.1"
env_logger          = "0.5.13"
error-chain         = "0.12.0"
failure             = "0.1.2"
fnv                 = "1.0.6"
flate2              = "1.0.2"
futures             = "0.1.24"
hyper               = "0.11.22"
hyper-tls           = "0.1.2"
itertools           = "0.7.8"
lazy_static         = "1.0.1"
libc                = "0.2.43"
log                 = "0.4.5"
memmap              = "0.6.2"
native-tls          = "0.1.5"
ndarray             = "0.12.0"
num                 = "0.2.0"
num_cpus            = "1.8.0"
pretty_env_logger   = "0.2.4"
protobuf            = "1.4.4"
rand                = "0.5.5"
rayon               = "1.0.2"
regex               = "1.0.1"
reqwest             = "0.9.0"
semver              = "0.9.0"
serde               = "1.0.79"
serde_derive        = "1.0.79"
serde_json          = "1.0.28"
structopt           = "0.2.10"
structopt-derive    = "0.2.10"
tar                 = "0.4.13"
tempdir             = "0.3.5"
threadpool          = "1.7.1"
tokio               = "0.1.7"
tokio-codec         = "0.1.0"
tokio-core          = "0.1.17"
tokio-io            = "0.1.6"
toml                = "0.4.5"
tower-web           = "0.2.2"
url                 = "1.6.0"
walkdir             = "2.0.1"
zip                 = "0.3.1"

[raze]
workspace_path = "//third_party/rust/cargo"
target = "x86_64-unknown-linux-gnu"

[raze.crates.unicase.'1.4.2']
additional_flags = [
  "--cfg=__unicase__iter_cmp",
  "--cfg=__unicase__defauler_hasher",
]

[raze.crates.unicase.'2.1.0']
additional_flags = [
  "--cfg=__unicase__iter_cmp",
  "--cfg=__unicase__defauler_hasher",
]

[raze.crates.crc.'1.8.1']
gen_buildrs = true

[raze.crates.mime_guess.'2.0.0-alpha.6']
gen_buildrs = true

[raze.crates.openssl-sys.'0.9.36']
additional_deps = [
  "@openssl//:lib",
]
# openssl-sys is painful.  The magic incantation of flags was determined by doing the following:
#   1) git clone https://github.com/sfackler/rust-openssl
#   2) cd rust-openssl/openssl-sys
#   3) export OPENSSL_DIR=/path/to/openssl
#   4) cargo build -vv
# and observing the cfg's passed/used by its build.rs
additional_flags = [
    # ...
]

[raze.crates.openssl.'0.10.12']
additional_flags = [
    # ...
]

[raze.crates.openssl.'0.9.24']
additional_flags = [
    # ...
]

[raze.crates.bzip2-sys.'0.1.6']
additional_deps = [
  "//third_party/bz2/1/0/6:bz2",
]

[raze.crates.bindgen.'0.40.0']
gen_buildrs = true
extra_aliased_targets = [
  "cargo_bin_bindgen"
]

[raze.crates.clang-sys.'0.23.0']
gen_buildrs = false
# Overriden to hand-written BUILD file, because libloading is building its own c portion.
skipped_deps = [
    "libloading-0.5.0"
]
additional_deps = [
    "//third_party/rust/cargo/overrides/libloading-0.5.0:libloading",
]

[raze.crates.backtrace.'0.3.9']
# Similar to `clang-sys` above, we build backtrace-sys manually with Bazel overrides
# to compile libbacktrace from packaged sources
skipped_deps = [
    "backtrace-sys-0.1.24",
]
additional_deps = [
    "//third_party/rust/cargo/overrides/backtrace-sys-0.1.23:backtrace_sys",
]

[raze.crates.lazy_static.'1.1.0']
additional_flags = [
    "--cfg=lazy_static_inline_impl",
]

[raze.crates.proc-macro2.'0.4.19']
additional_flags = [
    "--cfg use_proc_macro",
]

third_party/rust/cargo/overrides/backtrace-sys-0.1.23/BUILD

"""
OVERRIDDEN:
cargo-raze crate build file.

- `backtrace-sys` comes with `libloading`, which we build via targets here rather
   than using backtrace-sys's build.rs
"""
package(default_visibility=["//third_party/rust/cargo:__subpackages__"])

licenses([
    "notice",  # "MIT,Apache-2.0"
])

load(
    "@io_bazel_rules_rust//rust:rust.bzl",
    "rust_library",
    "rust_binary",
    "rust_test",
    "rust_benchmark",
)

genrule(
    name="touch_config_header",
    outs=[
        "config.h",
    ],
    cmd="touch $@",
)

genrule(
    name="touch_backtrace_supported_header",
    outs=[
        "backtrace-supported.h",
    ],
    cmd="touch $@",
)

# This should roughly follow the Rust build script: https://github.com/alexcrichton/backtrace-rs/blob/master/backtrace-sys/build.rs
cc_library(
    name="backtrace_native",
    srcs=[
        ":touch_config_header",
        ":touch_backtrace_supported_header",
        "src/libbacktrace/internal.h",
        "src/libbacktrace/backtrace.h",
        "src/libbacktrace/alloc.c",
        "src/libbacktrace/dwarf.c",
        "src/libbacktrace/fileline.c",
        "src/libbacktrace/posix.c",
        "src/libbacktrace/read.c",
        "src/libbacktrace/sort.c",
        "src/libbacktrace/state.c",
        "src/libbacktrace/elf.c",
    ],
    includes=["."],
    copts=[
        "-fvisibility=hidden",
        "-fPIC",
    ],
    defines=[
        "BACKTRACE_ELF_SIZE=64",
        "BACKTRACE_SUPPORTED=1",
        "BACKTRACE_USES_MALLOC=1",
        "BACKTRACE_SUPPORTS_THREADS=0",
        "BACKTRACE_SUPPORTS_DATA=0",
        "HAVE_DL_ITERATE_PHDR=1",
        "_GNU_SOURCE=1",
        "_LARGE_FILES=1",
        "backtrace_full=__rbt_backtrace_full",
        "backtrace_dwarf_add=__rbt_backtrace_dwarf_add",
        "backtrace_initialize=__rbt_backtrace_initialize",
        "backtrace_pcinfo=__rbt_backtrace_pcinfo",
        "backtrace_syminfo=__rbt_backtrace_syminfo",
        "backtrace_get_view=__rbt_backtrace_get_view",
        "backtrace_release_view=__rbt_backtrace_release_view",
        "backtrace_alloc=__rbt_backtrace_alloc",
        "backtrace_free=__rbt_backtrace_free",
        "backtrace_vector_finish=__rbt_backtrace_vector_finish",
        "backtrace_vector_grow=__rbt_backtrace_vector_grow",
        "backtrace_vector_release=__rbt_backtrace_vector_release",
        "backtrace_close=__rbt_backtrace_close",
        "backtrace_open=__rbt_backtrace_open",
        "backtrace_print=__rbt_backtrace_print",
        "backtrace_simple=__rbt_backtrace_simple",
        "backtrace_qsort=__rbt_backtrace_qsort",
        "backtrace_create_state=__rbt_backtrace_create_state",
        "backtrace_uncompress_zdebug=__rbt_backtrace_uncompress_zdebug",
    ],
)

rust_library(
    name="backtrace_sys",
    crate_root="src/lib.rs",
    crate_type="lib",
    srcs=glob(["**/*.rs"]),
    deps=[
        "//third_party/rust/cargo/vendor/libc-0.2.43:libc",
        ":backtrace_native",
    ],
    rustc_flags=[
        "--cap-lints allow",

    ],
    crate_features=[
    ],
)

third_party/rust/cargo/overrides/libloading-0.5.0/BUILD

"""
OVERRIDDEN:
cargo-raze crate build file.

- Libloading has a CC dep that needs to be brought in
"""

package(default_visibility = ["//visibility:public"])

licenses([
    "notice",  # "ISC"
])

load(
    "@io_bazel_rules_rust//rust:rust.bzl",
    "rust_library",
    "rust_binary",
    "rust_test",
    "rust_benchmark",
)

cc_library(
    name = "global_whatever",
    srcs = [
        "src/os/unix/global_static.c",
    ],
    copts = [ "-fPIC" ],
)

rust_library(
    name = "libloading",
    srcs = glob(["**/*.rs"]),
    crate_features = [
    ],
    crate_root = "src/lib.rs",
    crate_type = "lib",
    rustc_flags = [
        "--cap-lints allow",

    ],
    deps = [
        ":global_whatever",
    ],
)

this also serves as some additional context for https://github.com/google/cargo-raze/issues/58

[akhilles]

A few notes from my attempt at looking at ring that @mfarrugi brought up:

• https://github.com/briansmith/ring/blob/master/build.rs#L270 looks for the CARGO_PKG_NAME envvar - this isn't set in the build script wrapper that cargo raze generates (but is set in bazel rust, for example: https://github.com/bazelbuild/rules_rust/blob/master/rust/toolchain.bzl#L23). I'm pretty sure the intended build path here is to end up calling the ring_build_rs_main() fn, rather than enter the pregeneration step via pregenerate_asm_main(). I hand edited cargo raze's ring_build_script_executor genrule to get past this point.
• Once I forced execution into ring_build_rs_main, we once again get blocked on missing envvars. In all, I had to add this to the same genrule:

 + " export CARGO_CFG_TARGET_ARCH=x86_64;"
        + " export CARGO_CFG_TARGET_OS=linux;"
        + " export CARGO_CFG_TARGET_ENV=;"
        + " export DEBUG=false;"
        + " export OPT_LEVEL=3;"
        + " export HOST=;"

The latter 3 are needed by the gcc invocation. Apparently, cargo sets this all up for build scripts, and I suspect cargo raze will eventually need to reimplement a whole bunch (if not all) of this.

• After this, I hit include_bytes macro issues where the macro couldn't find the requested files, e.g. https://github.com/briansmith/ring/blob/d249aa57a08891a55254b303f628c87aeab84559/src/ec/suite_b/ecdsa/signing.rs#L350. The working dir is different and so it fails. I ran out of time to experiment further, and stopped here (for now).

HTH

@vitalyd covered almost all of the changes needed to get ring to compile. In order to address the include_bytes macro issue, some data directories need to be included in the sandbox. I was too lazy to find exactly what's needed, so I just added data_attr = "glob([\"src/**\"])" to Cargo.toml and this worked.

[prestonvanloon]

I tried the suggestions from https://github.com/google/cargo-raze/issues/41#issuecomment-407743039 and https://github.com/google/cargo-raze/issues/41#issuecomment-491547697 could not get ring to build.

ERROR: /home/preston/.cache/bazel/_bazel_preston/2bf51e8690558a83a0a0514ab422fd4b/external/raze__ring__0_14_6/BUILD.bazel:46:1: Executing genrule @raze__ring__0_14_6//:ring_build_script_executor failed (Exit 101)
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Os { code: 17, kind: AlreadyExists, message: "File exists" }', src/libcore/result.rs:997:5
stack backtrace:
   0: std::sys::unix::backtrace::tracing::imp::unwind_backtrace
             at src/libstd/sys/unix/backtrace/tracing/gcc_s.rs:39
   1: std::sys_common::backtrace::_print
             at src/libstd/sys_common/backtrace.rs:71
   2: std::panicking::default_hook::{{closure}}
             at src/libstd/sys_common/backtrace.rs:59
             at src/libstd/panicking.rs:197
   3: std::panicking::default_hook
             at src/libstd/panicking.rs:211
   4: std::panicking::rust_panic_with_hook
             at src/libstd/panicking.rs:474
   5: std::panicking::continue_panic_fmt
             at src/libstd/panicking.rs:381
   6: rust_begin_unwind
             at src/libstd/panicking.rs:308
   7: core::panicking::panic_fmt
             at src/libcore/panicking.rs:85
   8: core::result::unwrap_failed
   9: ring_build_script::main
  10: std::rt::lang_start::{{closure}}
  11: std::panicking::try::do_call
             at src/libstd/rt.rs:49
             at src/libstd/panicking.rs:293
  12: __rust_maybe_catch_panic
             at src/libpanic_unwind/lib.rs:87
  13: std::rt::lang_start_internal
             at src/libstd/panicking.rs:272
             at src/libstd/panic.rs:388
             at src/libstd/rt.rs:48
  14: main
  15: __libc_start_main
  16: _start

[jonhoo]

I think it'd be really useful to have an example in this repository for how to write a -sys crate that fits into an existing Bazel pipeline using cargo raze. Even one that has a relatively straightforward build.rs that just runs bindgen on the output of a cc_library would go a long way! This thread is the only real source of such examples at the moment, and they are all for relatively complicated builds.

[nhyne]

@prestonvanloon were you able to make any more progress on ring? I'm running into the same issue you have.

[prestonvanloon]

I don’t recall reaching a solution. Gave up on rust with bazel for now

[joprice]

@mfarrugi Did you try getting openssl working with boringssl? It's especially convenient since they maintain a bazel branch.

[mfarrugi]

Nope, was addressing a specific internal requirement by linking to system openssl.

On Fri, Dec 20, 2019 at 11:08 PM Joseph Price notifications@github.com wrote:

@mfarrugi https://github.com/mfarrugi Did you try getting openssl working with boringssl? It's especially convenient since they maintain a bazel branch.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/google/cargo-raze/issues/41?email_source=notifications&email_token=AAROG6B4FF2GCKYXNNHY35TQZWJDVA5CNFSM4EYIEJWKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEHOUS6A#issuecomment-568150392, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAROG6EVNFEFJ7VIMT4MEA3QZWJDVANCNFSM4EYIEJWA .

[joprice]

No problem. I'll keep smashing my head against it. I'm also hitting a glorious infinite loop when trying to compile unicase with the error "calls in statics are limited to constant functions, tuple structs and tuple variants".

[gluax]

Has anyone gotten backtrace-0.3.4 to compile with bazel? I keep getting the below errors.

Loading: 
Loading: 0 packages loaded
Analyzing: target //rust/game:game (0 packages loaded, 0 targets configured)
INFO: Analyzed target //rust/game:game (0 packages loaded, 0 targets configured).
INFO: Found 1 target...
[0 / 7] [Prepa] BazelWorkspaceStatusAction stable-status.txt
ERROR: /home/gluax/docs/projects/wander-interactive-fiction-engine/cargo/vendor/backtrace-0.3.40/BUILD:29:1: error executing shell command: '/bin/bash -c CARGO_MANIFEST_DIR=$(pwd)/cargo/vendor/backtrace-0.3.40 external/rust_linux_x86_64/bin/rustc "$@" --remap-path-prefix="$(pwd)"=__bazel_redacted_pwd  cargo/vendor/backtrace-0.3.40/src/l...' failed (Exit 1)
error[E0308]: mismatched types
   --> cargo/vendor/backtrace-0.3.40/src/symbolize/libbacktrace.rs:100:45
    |
96  |     pub fn addr(&self) -> Option<*mut c_void> {
    |                           ------------------- expected `core::option::Option<*mut libc::unix::c_void>` because of return type
...
100 |             Symbol::Dladdr(ref s) => return s.addr(),
    |                                             ^^^^^^^^ expected enum `libc::unix::c_void`, found enum `core::ffi::c_void`
    |
    = note: expected type `core::option::Option<*mut libc::unix::c_void>`
               found type `core::option::Option<*mut core::ffi::c_void>`

error[E0308]: mismatched types
   --> cargo/vendor/backtrace-0.3.40/src/symbolize/libbacktrace.rs:433:32
    |
433 |         return dladdr_fallback(what.address_or_ip(), cb);
    |                                ^^^^^^^^^^^^^^^^^^^^ expected enum `libc::unix::c_void`, found enum `core::ffi::c_void`
    |
    = note: expected type `*mut libc::unix::c_void`
               found type `*mut core::ffi::c_void`

error[E0308]: mismatched types
   --> cargo/vendor/backtrace-0.3.40/src/symbolize/libbacktrace.rs:461:25
    |
461 |         dladdr_fallback(what.address_or_ip(), cb);
    |                         ^^^^^^^^^^^^^^^^^^^^ expected enum `libc::unix::c_void`, found enum `core::ffi::c_void`
    |
    = note: expected type `*mut libc::unix::c_void`
               found type `*mut core::ffi::c_void`

error[E0308]: mismatched types
   --> cargo/vendor/backtrace-0.3.40/src/symbolize/libbacktrace.rs:466:21
    |
466 |     dladdr::resolve(addr, &mut |sym| {
    |                     ^^^^ expected enum `core::ffi::c_void`, found enum `libc::unix::c_void`
    |
    = note: expected type `*mut core::ffi::c_void`
               found type `*mut libc::unix::c_void`

error: aborting due to 4 previous errors

For more information about this error, try `rustc --explain E0308`.
Target //rust/game:game failed to build
INFO: Elapsed time: 0.568s, Critical Path: 0.51s
INFO: 0 processes.
FAILED: Build did NOT complete successfully
ERROR: Build failed. Not running target
FAILED: Build did NOT complete successfully

and my cargo.toml is as follows.

package]
name = "interactive-fiction"
version = "0.0.0"

# Mandatory (or Cargo tooling is unhappy)
[lib]
path = "fake_lib.rs"

[raze]
# The WORKSPACE relative path to the Cargo.toml working directory.
workspace_path = "//cargo"

# The target to generate BUILD rules for.
target = "x86_64-unknown-linux-gnu"

[dependencies]
serde_json = "1.0.41"
diesel = { version = "1.4.3", features = ["sqlite"] }
serde = { version = "1.0.104", features = ["derive"] }
toml = "0.5.5"
gluon = "0.13.1"

[raze.crates.backtrace.'0.3.40']
skipped_deps = [
    "backtrace-sys-0.1.32",
]
additional_deps = [
    "//cargo/overrides/backtrace-sys-0.1.32:backtrace_sys",
]

[raze.crates.indexmap.'1.3.0']
additional_flags = [
"--cfg=has_std",
]

[raze.crates.'libsqlite3-sys'.'0.16.0']
gen_buildrs = true

[raze.crates.proc-macro2.'1.0.6'] 
additional_flags = [
"--cfg=use_proc_macro", 
]

[zoidyzoidzoid]

@gluaxspeed I got further than that, just using:

[raze.crates.backtrace.'0.3.42']
gen_buildrs = true

However then I got stuck on mime_guess, so I'm giving up for the time being.

P.S.: I might have only got further on Linux and not Mac.

[lopter]

However then I got stuck on mime_guess, so I'm giving up for the time being.

Yes, I was stuck there too and that's because build.rs for unicase makes some println outputs that cargo-raze doesn't capture/understand (see @acmcarther reply in #116). You can get around with this in Cargo.toml:

[raze.crates.unicase.'1.4.2']
# build.rs should set those flags for us but Bazel with cargo-raze doesn't
# understand/capture the println outputs it makes (see GH#116), so we have to
# set those flags by hand:
additional_flags = [
    "--cfg=__unicase__iter_cmp", # rustc >= 1.5.0
    "--cfg=__unicase__default_hasher", # rustc >= 1.13
]

[raze.crates.unicase.'2.6.0']
additional_flags = [
    "--cfg=__unicase__iter_cmp", # rustc >= 1.5.0
    "--cfg=__unicase__default_hasher", # rustc >= 1.13
    "--cfg=__unicase__const_fns", # rustc >= 1.31.0
    "--cfg=__unicase__core_and_alloc", # rustc >= 1.36.0
]

[lopter]

This pattern of emitting rustc cfg options from build.rs seems to be fairly common, indexmap was also problematic through its use of autocfg which is actually another crate to help you do exactly that from build.rs.

[KaiserKarel]

wasmer will compile some included files, using a target depended compiler. Any idea how to get this working?

[GregBowyer]

No problem. I'll keep smashing my head against it. I'm also hitting a glorious infinite loop when trying to compile unicase with the error "calls in statics are limited to constant functions, tuple structs and tuple variants".

Which version of ring, I think I got it working a while back

My Cargo.toml for the brave

Some notes:

• Semver matching is hanging out on #127 so that wont work with off the shelf cargo raze
• Aliases are used for futures in here and need #123 so again ... wont work with off the shelf cargo raze

[package]
name = "rust_deps"
version = "0.1.0"

[lib]
path = "fake_lib.rs"

[dependencies]
# Error handling {{{
failure = "=0.1.6"
backtrace = "=0.3.40"
anyhow = "1.0.0"
thiserror = "1.0.0"
# }}}

# Quality of life utils {{{
matches = "0.1.8"
static_assertions = "1.1.0"
# }}}

# FFI {{{
libc = "0.2.66"
# }}}

# Date and time helpers {{{
chrono = { version = "=0.4.10", features = ["serde"] }
# }}}

# Types & Units of measure {{{
byte-unit = "3.0.3"
# }}}

# Memory mallocs {{{
jemallocator = "0.3.2"
jemalloc-sys = "0.3.2"
jemalloc-ctl = "0.3.3"
mimalloc = "0.1.12"
libmimalloc-sys = "0.1.9"
# }}}

# Low-Level and Byte utils {{{
byteorder = "1.3.2"
bytes = "0.5.4"
bytes_old = { version = "0.4.8", package = "bytes" }
bitflags = "1.2.1"
# }}}

# Data structures and algorithms {{{
path-tree = "0.1.9"
smallvec = "1.2.0"
# }}}

# Random number generation {{{
rand = "=0.7.2"
rand_chacha = "=0.2.1"
rand_xoshiro = "=0.4.0"
rand_xorshift = "=0.2.0"
rand_isaac = "=0.2.0"
rand_pcg = "=0.2.1"
rand_hc = "=0.2.0"
getrandom = "0.1.3"
# }}}

# Proc-macro & rust syntax {{{
quote = "1.0.2"
syn = { version = "1.0.11", features = ["full"] }
proc-macro2 = "1.0.8"
synstructure = "=0.10.1"
# }}}

# String and text processing {{{
regex = "1.3.4"
textwrap = { version = "=0.11.0", features = ["term_size"] }
patch = { git = "https://github.com/GregBowyer/patch-rs", branch="update-nom" }
bytesize = { version = "1.0.0", features = ["serde"] }
hex = "0.4.0"
punycode = "0.4.1"
# }}}

# Syntax and parsing reporting {{{
codemap = "0.1.2"
codemap-diagnostic = "0.1.1"
codespan = "0.5.0"
codespan-reporting = "0.5.0"
darling = { version = "0.10.2", features = ["suggestions"] }
# }}}

# Command line program handling {{{
structopt = { version = "=0.3.5", features = ["suggestions", "color", "wrap_help", "doc", "yaml"] }
exitfailure = "=0.5.1"
rustyline = "5.0.4"
# }}}

# Derive helpers {{{
derive_builder = "0.9.0"
derive-new = "0.5.8"
enum-primitive-derive = "0.1.2"
# }}}

# Serialisation and file format tooling {{{
serde = "1.0.103"
serde_derive = "1.0.103"
serde_json = "1.0.44"
serde_test = "1.0.103"
serde_qs = "0.5.2"
serde_yaml = "0.8.11"
serde_repr = "0.1.5"

protobuf = { version = "2.8.1", features = ["with-bytes"] }
protobuf-codegen = "2.8.1"
# }}}

# Concurrency {{{
crossbeam = "0.7.3"
once_cell = "1.3.1"
# }}}

# Async landscape {{{
futures = { version = "0.3.1", features = ["default", "compat", "io-compat", "executor", "thread-pool"] }
futures01 = "0.1.29"
tokio = { version = "0.2.11", features = ["full"] }
async-trait = "0.1.22"
# }}}

# GRPC {{{
tonic = { version = "0.1.1", features = ["transport", "tls", "tls-roots"] }
# }}}

# Cloud utils {{{
rusoto_core = "0.43.0-beta.1"
rusoto_s3 = "0.43.0-beta.1"
# }}}

# Http {{{
hyper = "0.13.1"
hyper-tls = "0.4.1"
hyper-rustls = "0.19.1"
http = "0.2.0"
# Currently needed to make async shims for rusoto
http_old = { version = "0.1", package = "http" }
reqwest = "0.9.9"
percent-encoding = "2.1.0"
hyper-old-types = "0.11.0"
# }}}

# Crypto libs {{{
ring = "0.16.9"
# TODO - We would potentially want to replace with something like mesalink
openssl = "=0.10.26"
openssl-sys = "=0.9.53"
# }}}

# Security libs {{{
openssh-keys = "0.4.1"
# }}}

# Hash Functions {{{
digest = "0.8.1"
sha2 = "0.8.0"
farmhash = "1.1.5"
# }}}

# Logging and tracing {{{
tracing = "0.1.12"
tracing-log = "0.1.1"
tracing-futures = {version = "0.2.1", features = ["futures-03"]}
tracing-subscriber = {version = "0.2.0-alpha.4", features = ["env-filter", "fmt", "registry"]}
tracing-opentelemetry = "0.1.0"
opentelemetry = "0.1.5"
# }}}

# Storage {{{
lmdb-rkv = "0.12.3"
# }}}

# Parser generators {{{
# Uncomment this iff you need to trace a rust-peg generated parser
#peg = { version = "0.6.0", features = ["trace"] }
peg = "0.6.0"
peg-runtime = "0.6.0"
# }}}

# Data structures {{{
phf = { version = "0.8.0", features = ["macros"] }
# }}}

# Higher traits {{{
num-traits = "0.2.11"
# }}}

[dev-dependencies]
# Util macros {{{
indoc = "0.3"
# }}}

# Testing and assertion tooling {{{
float-cmp = "=0.6.0"
proptest = "=0.9.4"
proptest-derive = "0.1.2"
pretty_assertions = "0.6.1"
criterion = "0.3.0"
# }}}

[raze]
# The WORKSPACE relative path to the Cargo.toml working directory.
workspace_path = "//third_party/rust"
# The target to generate BUILD rules for.
target = "x86_64-unknown-linux-gnu"
# Have Bazel pull the dependencies down
genmode = "Remote"

[raze.crates.protobuf.'2.8.1']
gen_buildrs = true
patches = ["@bedrock//third_party/rust/patch:protobuf-2.8.1-rustc.patch"]

[raze.crates.protobuf-codegen.'2.8.1']
extra_aliased_targets = ["cargo_bin_protoc_gen_rust"]

[raze.crates.bstr.'0.2.8']
data_attr = 'glob(["**/*.dfa"])'

[raze.crates.derive_builder.'0.9.0']
data_attr = 'glob(["src/doc_tpl/**/*"])'

[raze.crates.quote.'0.6.13']
additional_flags = [
    "--cfg=integer128",
]

[raze.crates.syn.'0.15.44']
additional_flags = [
    "--cfg=syn_can_use_thread_id",
    "--cfg=syn_can_call_macro_by_path",
    "--cfg=syn_disable_nightly_tests",
]

[raze.crates.proc-macro2.'0.4.27']
additional_flags = [
    "--cfg=u128",
    "--cfg=use_proc_macro",
    "--cfg=wrap_proc_macro",
]

[raze.crates.openssl.'0.10.26']
additional_flags = [
    # Vendored version is libressl 2.8.3
    "--cfg=libressl",
    "--cfg=libressl261",
    "--cfg=libressl270",
    "--cfg=libressl273",
    "--cfg=libressl280",
    "--cfg=libressl283",
]
additional_deps = [
    "@libressl//:ssl",
    "@libressl//:crypto",
]

[raze.crates.openssl-sys.'0.9.53']
additional_flags = [
    # Vendored version is libressl 2.8.3
    "--cfg=libressl",
    "--cfg=libressl251",
    "--cfg=libressl261",
    "--cfg=libressl270",
    "--cfg=libressl273",
    "--cfg=libressl280",
    "--cfg=libressl281",
    "--cfg=libressl283",
]
additional_deps = [
    "@libressl//:ssl",
    "@libressl//:crypto",
]

[raze.crates.criterion.'0.3.0']
data_attr = 'glob(["**/src/html/*"])'

[raze.crates.mime_guess.'2.0.1']
gen_buildrs = true

[raze.crates.unicase.'2.6.0']
additional_flags = [
    "--cfg=__unicase__iter_cmp",
    "--cfg=__unicase__default_hasher",
    "--cfg=__unicase__const_fns",
    "--cfg=__unicase__core_and_alloc",
]

[raze.crates.serde.'1.0.103']
additional_flags = [
    "--cfg=ops_bound",
    "--cfg=core_reverse",
    "--cfg=de_boxed_c_str",
    "--cfg=de_rc_dst",
    "--cfg=core_duration",
    "--cfg=integer128",
    "--cfg=range_inclusive",
    "--cfg=num_nonzero",
    "--cfg=core_try_from",
    "--cfg=num_nonzero_signed",
    "--cfg=std_atomic64",
    "--cfg=std_atomic",
]

[raze.crates.nom.'5.0.1']
additional_flags = [
    "--cfg=stable_i128"
]

[raze.crates.typenum.'1.11.2']
gen_buildrs = true
patches = ["@bedrock//third_party/rust/patch:typenum-1.11.2-env-vars.patch"]
patch_args = ["-p1"]

[raze.crates.proc-macro2.'1.0.8']
additional_flags = [
    "--cfg=u128",
    "--cfg=use_proc_macro",
    "--cfg=wrap_proc_macro",
]

[raze.crates.rand.'0.6.5']
additional_flags = [
    "--cfg=rustc_1_25",
    "--cfg=rustc_1_26",
    "--cfg=rustc_1_27",
    "--cfg=rust_1_25",
    "--cfg=rust_1_26",
    "--cfg=rust_1_27",
]

[raze.crates.rand_chacha.'0.1.1']
additional_flags = [
    "--cfg=rustc_1_26",
]

[raze.crates.rand_pcg.'0.1.2']
additional_flags = [
    "--cfg=rustc_1_26",
    "--cfg=rust_1_26",
]

[raze.crates.backtrace-sys.'0.1.32']
additional_deps = [
  "@libbacktrace//:failure_backtrace"
]

[raze.crates.failure.'0.1.6']
additional_deps = [
  "@libbacktrace//:failure_backtrace"
]

[raze.crates.libc.'0.2.66']
additional_flags = [
    "--cfg=libc_priv_mod_use",
    "--cfg=libc_union",
    "--cfg=libc_const_size_of",
    "--cfg=libc_align",
    "--cfg=libc_core_cvoid",
    "--cfg=libc_packedN",
]

[raze.crates.rand.'0.7.2']
additional_deps = [
    "@bedrock//third_party/rust/patch/getrandom_package:getrandom_package"
]

[raze.crates.lexical-core.'0.4.6']
additional_flags = [
    "--cfg=has_range_bounds",
    "--cfg=has_slice_index",
    "--cfg=has_full_range_inclusive",
    "--cfg=has_const_index",
    "--cfg=has_i128",
    "--cfg=has_ops_bound",
    "--cfg=has_pointer_methods",
    "--cfg=has_range_inclusive",
    "--cfg=limb_width_64",
]

[raze.crates.indexmap.'1.3.0']
additional_flags = [
    "--cfg=has_std",
]

[raze.crates.log.'0.4.8']
additional_flags = [
    "--cfg=atomic_cas"
]

[raze.crates.proc-macro-nested.'0.1.3']
gen_buildrs = true

[raze.crates.lmdb-rkv-sys.'0.9.6']
additional_deps = [
    "@bedrock//third_party/cpp/lmdb"
]

[raze.crates.ring.'~0.16.9']
data_attr = 'glob(["**/src/data/*", "**/src/ec/**/*.der"])'

[raze.crates.webpki.'^0.21.0']
data_attr = 'glob(["**/src/data/*"])'

[raze.crates.jemalloc-sys.'^0.3.3']
additional_deps = [
    "@jemalloc//:jemalloc_impl"
]

[raze.crates.mimalloc.'^0.1.12']
additional_deps = [
    "@mimalloc//:mimalloc"
]

[raze.crates.rustversion.'^1.0.2']
gen_buildrs = true

[raze.crates.rusoto-core.'0.43.0-beta.1']
patches = ["@bedrock//third_party/rust/patch:rusoto-core-0.43.0-rustc_version.patch"]
patch_args = ["-p1"]

[Cytro54]

@GregBowyer Could you please upload a Gist of the BUILD file at @bedrock//third_party/patch? I'm running into some issues with typenum and this would greatly help!

(btw , sorry for the bad English, I'm from overseas)

[lopter]

@GregBowyer Could you please upload a Gist of the BUILD file at @bedrock//third_party/patch? I'm running into some issues with typenum and this would greatly help!

Does #138 help?

[Cytro54]

Yes it does, Thanks!

[thomcc]

One of the maintainers of libsqlite3-sys here: I found this when helping someone get it working with bazel. If there's anything we could adjust or add to make this work better for y'all, let me know (or file an issue).

[dfreese]

@GregBowyer how did you end up patching protobuf in @bedrock//third_party/rust/patch:protobuf-2.8.1-rustc.patch? I was looking at bypassing the build.rs, and just patching the library as so:

diff --git a/src/lib.rs b/src/lib.rs
index b487e6e..7a99866 100644
--- a/src/lib.rs
+++ b/src/lib.rs
@@ -114,9 +114,9 @@ mod protobuf {

 /// This symbol is in generated `version.rs`, include here for IDE
 #[cfg(never)]
-pub const VERSION: &str = "";
+pub const VERSION: &'static str = "2.8.2";
 /// This symbol is in generated `version.rs`, include here for IDE
 #[cfg(never)]
 #[doc(hidden)]
-pub const VERSION_IDENT: &str = "";
-include!(concat!(env!("OUT_DIR"), "/version.rs"));
+pub const VERSION_IDENT: &'static str = "VERSION_2_8_2";
+pub const VERSION_2_8_2: () = ();

which is what would result from the include!, but wanted to see what other approaches there were. This works for me, since I don't run nightly, so I only need the version.rs file generated by protobuf's build.rs.

[a1ph]

Hello. I got a compile error on mio. The versions of libc used by both mio and iovec seems to be the same. What is a solution for it? Thanks.

error[E0308]: mismatched types
  --> external/raze__mio__0_6_21/src/sys/unix/uio.rs:20:34
   |
20 | ...                   slice.as_ptr(),
   |                       ^^^^^^^^^^^^^^ expected struct `libc::iovec`, found struct `libc::unix::iovec`
   |
   = note: expected raw pointer `*const libc::iovec`
              found raw pointer `*const libc::unix::iovec`
   = note: perhaps two different versions of crate `libc` are being used?

[kornholi]

@a1ph You're most likely missing cfg flags for libc that unify the types:

[raze.crates.libc.'0.2.69']
additional_flags = [
    "--cfg=libc_priv_mod_use",
    "--cfg=libc_union",
    "--cfg=libc_const_size_of",
    "--cfg=libc_align",
    "--cfg=libc_core_cvoid",
    "--cfg=libc_packedN",
]

Make sure the version matches your libc exactly and re-run cargo raze.

[dfreese]

So with cargo-raze 0.3.0 and later, below now works for the ring crate. Thanks @kornholi for adding the additional_build_file functionality.

Cargo.toml

[dependencies]
ring = "=0.16.4"

[raze.crates.ring.'0.16.14']
data_attr = 'glob(["**/src/data/*", "**/src/ec/**/*.der"])'
additional_deps = [
    # provided by the additional_build_file
    ":ring-core",
]
additional_build_file = "builds/ring-0.16.BUILD"

builds/ring-0.16.BUILD

load("@rules_cc//cc:defs.bzl", "cc_library")

# Based off of ring's build.rs file:
# https://github.com/briansmith/ring/blob/master/build.rs
cc_library(
    name = "ring-core",
    srcs = glob(
        [
            "**/*.h",
            "**/*.c",
            "**/*.inl",
            "**/*x86_64*-elf.S",
        ],
        exclude = ["crypto/constant_time_test.c"],
    ),
    copts = [
        "-fno-strict-aliasing",
        "-fvisibility=hidden",
    ],
    includes = [
        "include",
    ],
)

[a1ph]

@kornholi Thanks for your suggestion, but it did not help.

[a1ph]

How do I deal with package aliases, e.g. a crate has the following in Cargo.toml:

[dependencies.xml_rs]
version = "0.8.0"
package = "xml-rs"

And the error looks like:

  |
6 | use xml_rs::{
  |     ^^^^^^ use of undeclared type or module `xml_rs`

Thanks

[kornholi]

@a1ph aliases are now handled correctly in the 0.3.3 release from yesterday. Did you figure out your issue with mio?

[a1ph]

@kornholi I tweaked mio dependency crates versions with additional_deps.

Then I stuck with openssl, but finally made it work today, although it required changes to rules_rust and cargo-raze

[a1ph]

With the latest changes to rules_rust and cargo-raze I was able to remove all the tweaks and build with just default_gen_buildrs = true. 🎉

[hexdae]

The latest changes are amazing and solved most of the issues in my project. However, for some reason the default build script generated by cargo raze for libusb1-sys 0.3.7 hangs forever while building. (context, I am trying to build a project that depends on rusb 0.6

I tried adding the system libusb as a dependency using new_local_repository but I got the same result. Any suggestions would be very welcome.

Example `Cargo.toml` file to reproduce

```toml [package] name = "LibUSB_App" version = "0.1.0" [lib] path = "fake_lib.rs" [dependencies] rusb = "0.6" byte_struct = "0.4.2" [raze] workspace_path = "//cargo" output_buildfile_suffix = "BUILD.bazel" default_gen_buildrs = true ```

[parisholley]

couple notes for people who have some of the issues I had:

• don't use create vendoring, lots of headaches that just aren't supported well if your dependencies require building c' libs
• if you use default_gen_buildrs = true and run into some builds that seemingly fail to compile C code due to arch related issues, try adding gen_buildrs = false to the override, like i had to do for ring based on the example above:

[raze.crates.ring.'0.16.14']
gen_buildrs = false
data_attr = 'glob(["**/src/data/*", "**/src/ec/**/*.der"])'
additional_deps = [
    # provided by the additional_build_file
    ":ring-core",
]
additional_build_file = "builds/ring-0.16.BUILD"

for anyone running into issues with native-tls (macos build), i had to manually import all of it's extern crate dependencies:

security-framework = "0.4.4" # bazel fix
security-framework-sys = "0.4.3"
tempfile = "3.1.0"
libc = "0.2.76"

and add them as additional deps

[raze.crates.native-tls.'0.2.4']
additional_deps = [
    "@raze__lazy_static__1_4_0//:lazy_static",
    "@raze__security_framework__0_4_4//:security_framework",
    "@raze__security_framework_sys__0_4_3//:security_framework_sys",
    "@raze__tempfile__3_1_0//:tempfile",
    "@raze__libc__0_2_76//:libc"
]

[epigramengineer]

An interesting example I've found amethyst which has a lot of sys dependencies. I thought it would be clever to pull them in rules_nixpkgs:

rules_nixpkgs_version="dc24090573d74adcf38730422941fd69b87682c7"
rules_nixpkgs_sha256="aca86baa64174478c57f74ed09d5c2313113abe94aa3af030486d1b14032d3ed"

http_archive(
    name = "io_tweag_rules_nixpkgs",
    strip_prefix = "rules_nixpkgs-{0}".format(rules_nixpkgs_version),
    url = "https://github.com/tweag/rules_nixpkgs/archive/{0}.tar.gz".format(rules_nixpkgs_version),
    sha256 = rules_nixpkgs_sha256,
)

load("@io_tweag_rules_nixpkgs//nixpkgs:repositories.bzl", "rules_nixpkgs_dependencies")
rules_nixpkgs_dependencies()

load("@io_tweag_rules_nixpkgs//nixpkgs:nixpkgs.bzl", "nixpkgs_git_repository", "nixpkgs_package")
nixpkgs_git_repository(
    name = "nixpkgs",
    revision = nixpkgs_version,
    sha256 = nixpkgs_sha256,
)

nixpkgs_package(name = "alsa-lib", repository = "@nixpkgs")
nixpkgs_package(name = "cmake", repository = "@nixpkgs")
nixpkgs_package(name = "expat", repository = "@nixpkgs")
nixpkgs_package(name = "openssl", repository = "@nixpkgs")
nixpkgs_package(name = "pkg-config", repository = "@nixpkgs")
nixpkgs_package(name = "python3", repository = "@nixpkgs")
nixpkgs_package(name = "vulkan-validation-layers", repository = "@nixpkgs")
nixpkgs_package(name = "xorg.libX11", repository = "@nixpkgs")

However actually configuring them as dependencies on each of the sys crates has proven difficult. Are there any recommendations on how debugging / identifying configuration options and specific flags. So far I've been using `bazel query deps(//3rdparty/cargo:amethyst)' to find the quickly identify dependent crates, but I don't know how to go about actually finding all the necessary flags or options.

[mfarrugi]

Do you have nix-shell --command 'cargo build' working for amethyst? If that works, you can run cargo build -v to print out the flags.

On Thu, Sep 10, 2020 at 10:03 PM epigramengineer notifications@github.com wrote:

An interesting example I've found amethyst https://crates.io/crates/amethyst which has a lot of sys dependencies. I thought it would be clever to pull them in rules_nixpkgs https://github.com/tweag/rules_nixpkgs:

rules_nixpkgs_version="dc24090573d74adcf38730422941fd69b87682c7" rules_nixpkgs_sha256="aca86baa64174478c57f74ed09d5c2313113abe94aa3af030486d1b14032d3ed"

http_archive( name = "io_tweag_rules_nixpkgs", strip_prefix = "rules_nixpkgs-{0}".format(rules_nixpkgs_version), url = "https://github.com/tweag/rules_nixpkgs/archive/{0}.tar.gz".format(rules_nixpkgs_version), sha256 = rules_nixpkgs_sha256, )

load("@io_tweag_rules_nixpkgs//nixpkgs:repositories.bzl", "rules_nixpkgs_dependencies") rules_nixpkgs_dependencies()

load("@io_tweag_rules_nixpkgs//nixpkgs:nixpkgs.bzl", "nixpkgs_git_repository", "nixpkgs_package") nixpkgs_git_repository( name = "nixpkgs", revision = nixpkgs_version, sha256 = nixpkgs_sha256, )

nixpkgs_package(name = "alsa-lib", repository = "@nixpkgs") nixpkgs_package(name = "cmake", repository = "@nixpkgs") nixpkgs_package(name = "expat", repository = "@nixpkgs") nixpkgs_package(name = "openssl", repository = "@nixpkgs") nixpkgs_package(name = "pkg-config", repository = "@nixpkgs") nixpkgs_package(name = "python3", repository = "@nixpkgs") nixpkgs_package(name = "vulkan-validation-layers", repository = "@nixpkgs") nixpkgs_package(name = "xorg.libX11", repository = "@nixpkgs")

However actually configuring them as dependencies on each of the sys crates has proven difficult. Are there any recommendations on how debugging / identifying configuration options and specific flags. So far I've been using `bazel query deps(//3rdparty/cargo:amethyst)' to find the quickly identify dependent crates, but I don't know how to go about actually finding all the necessary flags or options.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/google/cargo-raze/issues/41#issuecomment-690828606, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAROG6DQ43QWQC5HTC7X73TSFGAPDANCNFSM4EYIEJWA .

[cherryland]

We dropped google/cargo-raze and adapted the build-strategy of project-oak, by simply referencing the generated artifacts (cargo-build)

# These files are built via cargo outside of Bazel
exports_files(srcs = glob(["target/x86_64-unknown-linux-musl/release/*"]))

It just works.

Messing around Cargo, bazelbuild/rules_rust and google/cargo-raze is an absolute waste of time, energy, resources and money.

[mfarrugi]

The trade-off is ease of use and interop (rust <=> c, etc).

At a glance it looks like oak dropped the target they needed rules_rust & raze for recently and so removed unused dependencies. Their exports_files implies they are only using binary outputs, which would be straightforward.

On Sat, Oct 10, 2020 at 7:47 AM cherryland notifications@github.com wrote:

We dropped google/cargo-raze and adapted the build-strategy of project-oak https://github.com/project-oak/oak, by simply referencing the generated artifacts (cargo-build https://doc.rust-lang.org/cargo/commands/cargo-build.html)

These files are built via cargo outside of Bazel

exports_files(srcs = glob(["target/x86_64-unknown-linux-musl/release/*"]))

It just works.

Messing around Cargo, bazelbuild/rules_rust and google/cargo-raze is an absolute waste of time, energy, resources and money.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/google/cargo-raze/issues/41#issuecomment-706536431, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAROG6H5LIFRULSUISTHISLSKBCUTANCNFSM4EYIEJWA .

[nikclayton-dfinity]

I'm trying to get logos-derive building (https://github.com/maciejhirsz/logos/blob/master/logos-derive/Cargo.toml), it's a dependency's dependency.

But the build fails with:

% bazel build :all
INFO: Analyzed 2 targets (5 packages loaded, 127 targets configured).
INFO: Found 2 targets...
INFO: From Compiling Rust proc-macro logos_derive v0.11.5 (23 files):
error[E0432]: unresolved import `proc_macro`
  --> external/raze__logos_derive__0_11_5/src/lib.rs:25:5
   |
25 | use proc_macro::TokenStream;
   |     ^^^^^^^^^^ help: a similar path exists: `syn::proc_macro`

error: aborting due to previous error

For more information about this error, try `rustc --explain E0432`.
ERROR: /private/var/tmp/_bazel_nik/b7eabcb2312aee3132f61301201aeb4a/external/raze__logos_derive__0_11_5/BUILD.bazel:31:13: output 'external/raze__logos_derive__0_11_5/liblogos_derive-1722838883.dylib' was not created
ERROR: /private/var/tmp/_bazel_nik/b7eabcb2312aee3132f61301201aeb4a/external/raze__logos_derive__0_11_5/BUILD.bazel:31:13: not all outputs were created or valid
INFO: Elapsed time: 2.108s, Critical Path: 1.64s
INFO: 11 processes: 10 internal, 1 darwin-sandbox.
FAILED: Build did NOT complete successfully

This is with

[raze.crates.logos-derive.'0.11.5']
additional_flags = [
    "--cfg=use_proc_macro",
    "--cfg=wrap_proc_macro",
]

in the Cargo.toml file. The generated BUILD file for it is:

"""
@generated
cargo-raze crate build file.

DO NOT EDIT! Replaced on runs of cargo-raze
"""

# buildifier: disable=load
load(
    "@io_bazel_rules_rust//rust:rust.bzl",
    "rust_binary",
    "rust_library",
    "rust_test",
)

package(default_visibility = [
    # Public for visibility by "@raze__crate__version//" targets.
    #
    # Prefer access through "//cargo", which limits external
    # visibility to explicit Cargo.toml dependencies.
    "//visibility:public",
])

licenses([
    "notice",  # MIT from expression "MIT OR Apache-2.0"
])

# Generated targets

# buildifier: leave-alone
rust_library(
    name = "logos_derive",
    crate_type = "proc-macro",
    deps = [
        "@raze__beef__0_4_4//:beef",
        "@raze__fnv__1_0_7//:fnv",
        "@raze__proc_macro2__1_0_24//:proc_macro2",
        "@raze__quote__1_0_7//:quote",
        "@raze__regex_syntax__0_6_20//:regex_syntax",
        "@raze__syn__1_0_44//:syn",
        "@raze__utf8_ranges__1_0_4//:utf8_ranges",
    ],
    srcs = glob(["**/*.rs"]),
    crate_root = "src/lib.rs",
    edition = "2018",
    rustc_flags = [
        "--cap-lints=allow",
        "--cfg=use_proc_macro",
        "--cfg=wrap_proc_macro",
    ],
    version = "0.11.5",
    tags = [
        "cargo-raze",
        "manual",
    ],
    crate_features = [
    ],
)

I assume I've missed something obvious, but for the life of me I can't see what it is. Anyone got any pointers?

[mfarrugi]

This was fixed earlier today by https://github.com/bazelbuild/rules_rust/pull/439

[nikclayton-dfinity]

@mfarrugi That's great, thanks for the pointer.

Do you think it would be worthwhile adding a repository where people can submit the overrides that they've determined need to happen, and integrating that in to cargo-raze?

For example, I've just discovered that to build the json5 crate you need:

[raze.crates.json5.'0.2.8']
data_attr = "['src/json5.pest']"

To discover that I had (a) diagnose the build failure, (b) poke through the cargo-raze code to discover how to add a custom data attribute, (c) actually add the attribute.

The fix above is going to need to be done by anyone who wants to use that crate with cargo-raze (and I've posted it here so people searching for cargo-raze json5 in the future have a chance of finding it.

---

Note that the above works with a version of io_bazel_rules_rust from 2020-10-15, but not 2020-10-26.

I.e., with this in the WORKSPACE file it works:

http_archive(
    name = "io_bazel_rules_rust",
    sha256 = "74bcc568a2cb8e410967eae7b44b8303062257ec2a69f59737173be5eab48b65",
    strip_prefix = "rules_rust-8a8be5834e5f226e9f01ecc77929e12563f18cb5",
    urls = [
        # Master branch as of 2020-10-15
        # Includes #439, see https://github.com/google/cargo-raze/issues/41#issuecomment-709600426
        "https://github.com/bazelbuild/rules_rust/archive/8a8be5834e5f226e9f01ecc77929e12563f18cb5.tar.gz",
    ],
)

And with this it doesn't:

http_archive(
    name = "io_bazel_rules_rust",
    sha256 = "1211f57f6cbc5d68aff40d78d2548c8adee8f2921b54dca6ec9a9e7df60db13b",
    strip_prefix = "rules_rust-224fe6a81b8ffa9c585c75f49a41f9b9924d516d",
    urls = [
        # Master branch as of 2020-10-26
        # Includes #439, see https://github.com/google/cargo-raze/issues/41#issuecomment-709600426
        "https://github.com/bazelbuild/rules_rust/archive/224fe6a81b8ffa9c585c75f49a41f9b9924d516d.tar.gz",
    ],
)

---

224fe6a81b8ffa9c585c75f49a41f9b9924d516d over there is the commit that breaks it, I've tested other commits up to and including the one immediately preceding it, 9c889b057ddf4feddae7c5ae6913b7282154aa24, and they work.

---

More details in https://github.com/bazelbuild/rules_rust/issues/467