QrczakMK
commented
1 year ago For C++ usage of Riegeli/records it should be sufficient to import Riegeli along with required libraries. From https://github.com/google/riegeli/blob/master/WORKSPACE: com_google_absl, org_brotli, net_zstd, snappy, highwayhash, com_google_protobuf, and rules_cc. Other libraries are used only by separate Riegeli libraries providing access to them or by example binaries.
I agree that this should be documented.
I don’t know how to provide macros which package that, because versions of the libraries should agree among other dependencies which also use the same libraries, and the final project should decide about the versions. A simple macro would work if a project uses only Riegeli, but it might use other libraries with shared dependencies. What is the common bazel practice for this? What if another dependency of the final project also uses Abseil, perhaps provides a macro for importing its dependencies, and the versions of e.g. Abseil or protobuf conflict?
This gets more complicated with Python or TensorFlow, to the point that I don’t know yet how this should be done. Riegeli uses TensorFlow only for separate libraries and for example binaries, but in practice TensorFlow brings the Python configuration to use, constrains versions of shared dependencies, and the setup of that is in a complicated and brittle state. Riegeli uses a precompiled TensorFlow, because it turned out that building TensorFlow from sources has even larger problems. Using precompiled TensorFlow resulted in a https://github.com/google/riegeli/blob/master/configure script to find out TensorFlow and Python flags and paths, importing the generated file to https://github.com/google/riegeli/blob/master/.bazelrc, and referring to the variables in https://github.com/google/riegeli/blob/master/tf_dependency/tf_configure.bzl. This is a mess, I am sorry for that, and I don’t know how to clean it up. Riegeli should be buildable without TensorFlow. I did not manage to make Riegeli building with TF-2.12 yet, which uses a newer protobuf version, which requires a different setup, in particular because it now depends on Abseil, and its Python setup changed as well.
dpetrou-continua
SanjayVas
I'd prefer to not manually vendor (clone) Riegeli into my source tree, but rather use it as an external Bazel dependency (either by modifying our WORKSPACE or MODULE.bazel). Might you consider providing that functionality and document it?
(Apologies if this already exists and I missed it. I didn't find any documentation like, to pick a random example, https://github.com/hedronvision/bazel-compile-commands-extractor#first-do-the-usual-workspace-setup. Nor did I find any macros for pulling in Riegeli's deps as is typical in any non-bzlmod repo that's used as an external dependency.)
David