search

apache / arrow-rs

Official Rust implementation of Apache Arrow
https://arrow.apache.org/
Apache License 2.0
2.6k stars 794 forks source link

protoc failed: Unknown flag: `--experimental_allow_proto3_optional` #1574

Closed andygrove closed 2 years ago

andygrove commented 2 years ago

Describe the bug

I get an error when running cargo test. This is apparently due to a recent upgrade to prost and likely related to having an old protoc installed (I am using libprotoc 3.11.4). I am filing this issue to help others that hit the same problem.

error: failed to run custom build command for `arrow-flight v12.0.0 (/tmp/arrow-12.0.0.PitgR/apache-arrow-rs-12.0.0/arrow-flight)`

Caused by:
  process didn't exit successfully: `/tmp/arrow-12.0.0.PitgR/apache-arrow-rs-12.0.0/target/debug/build/arrow-flight-32d1e9d97d680e87/build-script-build` (exit status: 1)
  --- stdout
  cargo:rerun-if-changed=../format/Flight.proto

  --- stderr
  Error: Custom { kind: Other, error: "protoc failed: Unknown flag: --experimental_allow_proto3_optional\n" }

To Reproduce Run cargo test

Expected behavior Should not fail.

Additional context Add any other context about the problem here.

tustvold commented 2 years ago

Yes, protoc no longer bundles protoc binaries and instead uses the system versions if available, or builds from source if not - see https://github.com/tokio-rs/prost/pull/610

Unfortunately the arrow-flight specification is using the somewhat new explicit presence feature of protobuf, and this will cause issues if compiling with an older version of protoc which doesn't support it.

Perhaps we could check the version of the protoc binary and return a more helpful error message if it is too old 🤔

ismailmaj commented 2 years ago

Building protobuf from source solved this problem for me, I had to also uninstall the ubuntu package protobuf-compiler.

tustvold commented 2 years ago

FWIW I believe if you uninstall protobuf-compiler and have cmake installed, prost will automatically compile the protobuf-compiler from source.

alamb commented 2 years ago

I think we have concluded this is not a bug but a "feature" of newer versions of prost

XinyuZeng commented 1 year ago

The issue seems still happen today on my AWS Ubuntu 20.04 machine.

FWIW I believe if you uninstall protobuf-compiler and have cmake installed, prost will automatically compile the protobuf-compiler from source.

Seems like simply uninstall the old protobuf-compiler does not work. I need manually install the newest one globally. Following https://github.com/hyperium/tonic/issues/1047 solved my issue. Probably some instructions can be added to contributing guide to smooth future ramp ups.

alamb commented 1 year ago

Thanks for the report @XinyuZeng -- if you can contribute such an improvement that would be great ❤️