Use own version of protobuf locally

[davidbrogan13]

Hello,

I am trying to use fleetbench to do some benchmarking, however I want to point to my own version of protobuf that I have locally on my machine. I tried changing URL archive for protobuf to local repository in the WORKSPACE file ( see below)

However, I then get this error

I am wondering is it possible to use my own version of protobuf locally and if so what would be the easiest way to implement this?

Thanks in advance, David

[rjogrady]

Hi David, What you have should work. I've just tried it with grpc v1.49.0 locally.

First I would check that your bazel cache contains a symlink to your local repository. That is, ls -l /root/.cache/bazel/_bazel_root/.../external/com_google_protobuf should be pointing to /root/grpc-1.49.0/third_party/protobuf.

If it's not, you might try running "bazel clean --expunge" to force bazel to clear its cache of downloaded repos, then rebuild.

If it is, maybe your local protobuf library has a problem. grpc 1.49.0 should be using protobuf version 3.21.5, which seems to work for me.

Hope that helps

[davidbrogan13]

Hi Richard,

Thanks for the rapid response!

Sorry I should have mentioned that I have made some modifications to the protobuf library locally, however when I build it with grpc it all works okay so I was hoping it would be the same with this. I am not overly familiar with Basel, are there changes that may need to be made somewhere else to help basel build and run successfully?

Thanks again for your help on this, David

[rjogrady]

From the error message, it seems like the changes to protobuf introduced a circular dependency in the protobuf build rules.

@com_google_protobuf//:protobuf 
| @com_google_protobuf//:src/google/protobuf/unittest_lite_imports_nonlite.pb.h
| @com_google_protobuf//:cc_test_protos_genproto
...

Something has gone wrong here. I'm not an expert on protobuf build rules, but they are complex. I note that my version of protobuf doesn't depend on these. bazel query 'deps(@com_google_protobuf//:protobuf)' prints the list of all dependencies, which could be helpful for figuring out why. Or you can try bazel query 'somepath(@com_google_protobuf//:protobuf, @com_google_protobuf//:empty_proto)' to see if that tells you why it has taken on that dependency.

I might suggest seeing if you can get it working with a clean copy of protobuf and then reintroduce your changes one-by-one to identify which one causes the problem.

[davidbrogan13]

Thank you for your help! I managed to get it working! One quick thing, I noticed when I run the benchmark it doesn't give me bytes_per_second as an output field, is there anyway to enable it?

Thanks, David

[rjogrady]

We need to implement the bytes processed counter which depends on how much throughput it is achieving, and that is specific to what each benchmark does. You should see it for libc/mem_benchmark, but it's not there for the others yet.

Would you mind opening a new issue requesting bytes_per_second and which benchmark(s) you would want to see that for? Thanks!