tokio-rs / prost

PROST! a Protocol Buffers implementation for the Rust Language
Apache License 2.0
3.65k stars 476 forks source link

Support arch-os-prefixed PROTOC in `prost_build` (e.g: AARCH64_LINUX_PROTOC) #1083

Open daprilik opened 3 weeks ago

daprilik commented 3 weeks ago

283 introduced the PROTOC env var, which our project uses to great effect via the [env] table in .cargo/config.toml, ensuring that all folks working on the project use a consistent pre-packaged version of protoc (that we clone as part of our repo dev-env setup).

Unfortunately, our project is multi-platform and multi-arch, and needs to build across both windows and linux, as well as x64 and aarch64.

This fact makes it a bit awkward to use PROTOC as we are using it, as until https://github.com/rust-lang/cargo/issues/10273 is resolved, we are forced to use some sort of external script to symlink the correct protoc binary for a given build platform into a known-good directory, and then have PROTOC point at the symlink, which can get a bit finicky across multiple platforms.

Would it be possible to support arch-os-prefixed variants of of PROTOC in prostbuild? i.e: if PROTOC isn't found, fall-back to checking for the existence of a `$ARCH$OS_PROTOC` env-var?

This would allow us to do something like this in our [env] table:

[env]
X86_64_LINUX_PROTOC = { value = ".packages/protobuf/tools/x64-linux/protoc", relative = true }
X86_64_WINDOWS_PROTOC = { value = ".packages/protobuf/tools/x64-windows/protoc.exe", relative = true }
AARCH64_LINUX_PROTOC = { value = ".packages/protobuf/tools/aarch64-linux/protoc", relative = true }
AARCH64_WINDOWS_PROTOC = { value = ".packages/protobuf/tools/aarch64-windows/protoc.exe", relative = true }

...thereby avoiding the need to do any additional symlinking.

For some prior art: the openssl crate supports this exact feature: https://docs.rs/openssl/latest/openssl/#manual

caspermeijn commented 4 days ago

I believe the openssl example is different from that is needed for prost. The needed protoc binary is based on the host architecture and not the target architecture, right?

Could you solve your problem with a build.rs script? With cargo::rustc-env=VAR=VALUE you can set and environment variable. In the build script your can detect which protoc binary is needed.

daprilik commented 4 days ago

Unfortunately, a build-script based approach doesn't work here, as if the prost dependency is in a downstream crate, setting the env as part of a upstream build.rs won't percolate down the dependency chain.

The needed protoc binary is based on the host architecture and not the target architecture, right?

Right, but that doesn't actually change the usefulness of this proposed solution. We still want some way to uniformly set a path to a host-specific pre-packaged protoc executable via the [env] table.