search

google / riegeli

Riegeli/records is a file format for storing a sequence of string records, typically serialized protocol buffers.
Apache License 2.0
418 stars 53 forks source link

Upgrade protobuf to 3.19 to overcome build issue with bazel 4.2 #20

Closed mingzhao-db closed 2 years ago

mingzhao-db commented 2 years ago

Upgrade protobuf to 3.19 to overcome build issue with bazel 4.2

ERROR: /home/ming.zhao/.cache/bazel/_bazel_ming.zhao/c3581a885d2b26d39a1165f8c5a47b89/external/com_google_protobuf/BUILD:979:21: in proto_lang_toolchain rule @com_google_protobuf//:cc_toolchain: '@com_google_protobuf//:cc_toolchain' does not have mandatory provider 'ProtoInfo'.
ERROR: Analysis of target '//python/riegeli/records/examples:write_read_records' failed; build aborted: Analysis of target '@com_google_protobuf//:cc_toolchain' failed
INFO: Elapsed time: 0.077s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (0 packages loaded, 0 targets configured)
QrczakMK commented 2 years ago

Unfortunately that would break building the TensorFlow binding (and thus also examples), because even newest released TensorFlow versions include binary code compatible with protobuf 3.9.0–3.9.2 only.

Ideally Riegeli should have a different build mode which strips everything which depends on TensorFlow.

QrczakMK commented 2 years ago

There is now a workaround for building with an old protobuf: https://github.com/google/riegeli/commit/58945ac448cc0fc11629efb7d3627277c7d9f822, so this should no longer be urgent.

Since TensorFlow was not yet able to update its protobuf version (and applied the workaround to the old protobuf instead, which I applied now here), updating the protobuf version as specified in Riegeli’s WORKSPACE will need to wait.

Of course you can depend on parts of Riegeli which do not depend on TensorFlow, substituting your own protobuf version — dependency versions are ultimately decided by the final project.