Unable to pass bazel tests

[DianCh]

Hi, I'm following the Quick Start to build the command line tool for evaluation. However I get the following error:

`INFO: Writing tracer profile to '/home/users/myname/.cache/bazel/_bazel_myname/94779e27959809881a12fee9a754d13c/command.profile.gz' INFO: Analyzed 18 targets (46 packages loaded, 1546 targets configured). INFO: Found 10 targets and 8 test targets... INFO: Deleting stale sandbox base /home/users/myname/.cache/bazel/_bazel_myname/94779e27959809881a12fee9a754d13c/sandbox ERROR: /home/users/myname/.cache/bazel/_bazel_myname/94779e27959809881a12fee9a754d13c/external/com_google_protobuf/BUILD:388:1: Linking of rule '@com_google_protobuf//:protoc' failed (Exit 1) gcc failed: error executing command /opt/rh/devtoolset-4/root/usr/bin/gcc @bazel-out/host/bin/external/com_google_protobuf/protoc-2.params

Use --sandbox_debug to see verbose messages from the sandbox bazel-out/host/bin/external/com_google_protobuf/_objs/protoc_lib/code_generator.o:code_generator.cc:function google::protobuf::compiler::ParseGeneratorParameter(std::string const&, std::vector<std::pair<std::string, std::string>, std::allocator<std::pair<std::string, std::string> > >): error: undefined reference to 'std::__throw_out_of_range_fmt(char const, ...)' bazel-out/host/bin/external/com_google_protobuf/_objs/protoc_lib/command_line_interface.o:command_line_interface.cc:function google::protobuf::compiler::(anonymous namespace)::PluginName(std::string const&, std::string const&): error: undefined reference to 'std::throw_out_of_range_fmt(char const*, ...)' bazel-out/host/bin/external/com_google_protobuf/_objs/protoc_lib/command_line_interface.o:command_line_interface.cc:function google::protobuf::compiler::CommandLineInterface::InterpretArgument(std::string const&, std::string const&): error: undefined reference to 'std::throw_out_of_range_fmt(char const, ...)' bazel-out/host/bin/external/com_google_protobuf/_objs/protoc_lib/command_line_interface.o:command_line_interface.cc:function google::protobuf::compiler::CommandLineInterface::ParseArguments(int, char const const): error: undefined reference to 'std::__throw_out_of_range_fmt(char const, ...)' bazel-out/host/bin/external/com_google_protobuf/_objs/protoc_lib/java_helpers.o:java_helpers.cc:function google::protobuf::compiler::java::SortFieldsByNumber(google::protobuf::Descriptor const): error: undefined reference to '__cxa_throw_bad_array_new_length' bazel-out/host/bin/external/com_google_protobuf/_objs/protoc_lib/objectivec_message.o:objectivec_message.cc:function google::protobuf::compiler::objectivec::(anonymous namespace)::SortFieldsByNumber(google::protobuf::Descriptor const): error: undefined reference to '__cxa_throw_bad_array_new_length' bazel-out/host/bin/external/com_google_protobuf/_objs/protoc_lib/objectivec_message.o:objectivec_message.cc:function google::protobuf::compiler::objectivec::MessageGenerator::GenerateSource(google::protobuf::io::Printer): error: undefined reference to '__cxa_throw_bad_array_new_length' bazel-out/host/bin/external/com_google_protobuf/_objs/protobuf/dynamic_message.o:dynamic_message.cc:function google::protobuf::DynamicMessageFactory::GetPrototypeNoLock(google::protobuf::Descriptor const): error: undefined reference to '__cxa_throw_bad_array_new_length' collect2: error: ld returned 1 exit status INFO: Elapsed time: 91.393s, Critical Path: 33.57s INFO: 639 processes: 639 processwrapper-sandbox. FAILED: Build did NOT complete successfully //waymo_open_dataset/metrics:breakdown_generator_test NO STATUS //waymo_open_dataset/metrics:detection_metrics_test NO STATUS //waymo_open_dataset/metrics:hungarian_test NO STATUS //waymo_open_dataset/metrics:iou_test NO STATUS //waymo_open_dataset/metrics:matcher_test NO STATUS //waymo_open_dataset/metrics:metrics_utils_test NO STATUS //waymo_open_dataset/metrics:mot_test NO STATUS //waymo_open_dataset/metrics:tracking_metrics_test NO STATUS

FAILED: Build did NOT complete successfully`

g++ info: g++ (GCC) 7.3.1 20180303 (Red Hat 7.3.1-5) Copyright (C) 2017 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

tensorflow info: 1.14.0 gpu version

OS info: CentOS 7.6.1810 Core

I'm using the right version in the requirement but still struggling to get the bazel build run correctly. Can someone help me out here? Thank you so much!

[peisun1115]

It looks like it failed to build protobuf. I have never tested it in CentOS. I do not have time to play with it for now. Will try it when i have time.

Do you have access to a ubuntu machine?

Also i tried with tensorflow cpu 1.14 though i do not think that matters here. TF is not needed to build the metrics computation tool

[DianCh]

Hi, thanks for the quick reply. Unfortunately I only have access to a CentOS machine which is used in my lab. I tried building it several times and still no success. And now I keep seeing this error related to hungarian:

`-bash-4.2$ bazel test waymo_open_dataset/metrics:all INFO: Writing tracer profile to '/home/users/myname/.cache/bazel/_bazel_myname/94779e27959809881a12fee9a754d13c/command.profile.gz' INFO: Analyzed 18 targets (46 packages loaded, 1546 targets configured). INFO: Found 10 targets and 8 test targets... INFO: Deleting stale sandbox base /home/users/myname/.cache/bazel/_bazel_myname/94779e27959809881a12fee9a754d13c/sandbox ERROR: /home/users/myname/waymo-open-dataset/waymo_open_dataset/metrics/BUILD:186:1: Linking of rule '//waymo_open_dataset/metrics:hungarian_test' failed (Exit 1) gcc failed: error executing command /opt/rh/devtoolset-4/root/usr/bin/gcc @bazel-out/k8-opt/bin/waymo_open_dataset/metrics/hungarian_test-2.params

Use --sandbox_debug to see verbose messages from the sandbox bazel-out/k8-opt/bin/_solib_k8/libexternal_Scom_Ugoogle_Ugoogletest_Slibgtest.so: error: undefined reference to 'std::__throw_out_of_range_fmt(char const, ...)' bazel-out/k8-opt/bin/_solib_k8/libexternal_Scom_Ugoogle_Ugoogletest_Slibgtest.so: error: undefined reference to 'std::runtime_error::runtime_error(char const)' collect2: error: ld returned 1 exit status INFO: Elapsed time: 17.830s, Critical Path: 11.61s INFO: 630 processes: 630 processwrapper-sandbox. FAILED: Build did NOT complete successfully //waymo_open_dataset/metrics:breakdown_generator_test NO STATUS //waymo_open_dataset/metrics:detection_metrics_test NO STATUS //waymo_open_dataset/metrics:iou_test NO STATUS //waymo_open_dataset/metrics:matcher_test NO STATUS //waymo_open_dataset/metrics:metrics_utils_test NO STATUS //waymo_open_dataset/metrics:mot_test NO STATUS //waymo_open_dataset/metrics:tracking_metrics_test NO STATUS

FAILED: Build did NOT complete successfully`

Could you give some hints on this? And, any general guide for debugging bazel build (I was struggling when building things using bazel whose error messages seem hard to interpret)?

Thank you so much for the support!

[peisun1115]

How about trying docker? I have not got time to try this on centos

Try the following command: (assuming docker is installed on your os)

docker build --tag=open_dataset_pip -f pip_pkg_scripts/build.Dockerfile . sudo docker run -it --entrypoint=/bin/bash -e "GITHUB_BRANCH=master" -e "PYTHON_VERSION=3" -e "PYTHON_MINOR_VERSION=5" -e "PIP_MANYLINUX2010=1" -e "TF_VERSION=1.15.0" open_dataset_pip (you can change env if you like)