google / benchmark

A microbenchmark support library
Apache License 2.0
8.6k stars 1.57k forks source link

[BUG] complexity_test crashes with flag --benchmark_list_tests=true #1666

Closed ylxxwx closed 9 months ago

ylxxwx commented 9 months ago

Describe the bug There is a crash when I verify the latest benchmark installation.

benchmark/build/test$ ./complexity_test --benchmark_list_tests=true

System Which OS, compiler, and compiler version are you using:

To reproduce Steps to reproduce the behavior:

  1. sync to commit ... commit 344117638c8ff7e239044fd0fa7085839fc03021 (HEAD -> main, tag: v1.8.3, origin/main, origin/HEAD) Author: Dominic Hamon dominic+github@google.com Date: Thu Aug 31 13:16:50 2023 +0100

    bump version to 1.8.3

  2. cmake/bazel... Following the ReadMe, build install and run test.

  3. make ...

  4. See error benchmark/build/test$ ./complexity_test --benchmark_list_tests=true

BM_ComplexityCaptureArgs/capture_test/1/4 BM_ComplexityCaptureArgs/capture_test/2/4

Testing ConsoleReporter Output

/usr/local/google/home/lipingliu/82_benchmark/benchmark/test/output_test_helper.cc:121: CheckCase: Check `remaining_output.eof() == false' failed. End of output reached before match for regex "^%bigo_name %bigo_str %bigo_str[ ]$" was found actual regex string "^BM_Complexity_O1_BigO [ ] [0-9][.]?[0-9]+([eE][-+][0-9]+)? ([0-9]+) [ ] [0-9][.]?[0-9]+([eE][-+][0-9]+)? ([0-9]+)[ ]$" started matching near: Aborted

Expected behavior No crash

Screenshots If applicable, add screenshots to help explain your problem.

Additional context Add any other context about the problem here.

$ ./complexity_test --benchmark_list_tests=true Running benchmarks... BM_Complexity_O1/1 BM_Complexity_O1/8 BM_Complexity_O1/64 BM_Complexity_O1/512 BM_Complexity_O1/4096 BM_Complexity_O1/32768 BM_Complexity_O1/262144 BM_Complexity_O1/1 BM_Complexity_O1/8 BM_Complexity_O1/64 BM_Complexity_O1/512 BM_Complexity_O1/4096 BM_Complexity_O1/32768 BM_Complexity_O1/262144 BM_Complexity_O1/1 BM_Complexity_O1/8 BM_Complexity_O1/64 BM_Complexity_O1/512 BM_Complexity_O1/4096 BM_Complexity_O1/32768 BM_Complexity_O1/262144 BM_Complexity_O_N/1024 BM_Complexity_O_N/2048 BM_Complexity_O_N/4096 BM_Complexity_O_N/8192 BM_Complexity_O_N/16384 BM_Complexity_O_N/32768 BM_Complexity_O_N/65536 BM_Complexity_O_N/1024 BM_Complexity_O_N/2048 BM_Complexity_O_N/4096 BM_Complexity_O_N/8192 BM_Complexity_O_N/16384 BM_Complexity_O_N/32768 BM_Complexity_O_N/65536 BM_Complexity_O_N/1024 BM_Complexity_O_N/2048 BM_Complexity_O_N/4096 BM_Complexity_O_N/8192 BM_Complexity_O_N/16384 BM_Complexity_O_N/32768 BM_Complexity_O_N/65536 BM_Complexity_O_N_log_N/1024 BM_Complexity_O_N_log_N/2048 BM_Complexity_O_N_log_N/4096 BM_Complexity_O_N_log_N/8192 BM_Complexity_O_N_log_N/16384 BM_Complexity_O_N_log_N/32768 BM_Complexity_O_N_log_N/65536 BM_Complexity_O_N_log_N/1024 BM_Complexity_O_N_log_N/2048 BM_Complexity_O_N_log_N/4096 BM_Complexity_O_N_log_N/8192 BM_Complexity_O_N_log_N/16384 BM_Complexity_O_N_log_N/32768 BM_Complexity_O_N_log_N/65536 BM_Complexity_O_N_log_N/1024 BM_Complexity_O_N_log_N/2048 BM_Complexity_O_N_log_N/4096 BM_Complexity_O_N_log_N/8192 BM_Complexity_O_N_log_N/16384 BM_Complexity_O_N_log_N/32768 BM_Complexity_O_N_log_N/65536 BM_ComplexityCaptureArgs/capture_test/1/3 BM_ComplexityCaptureArgs/capture_test/2/3 BM_ComplexityCaptureArgs/capture_test/1/4 BM_ComplexityCaptureArgs/capture_test/2/4

Testing ConsoleReporter Output

/usr/local/google/home/lipingliu/82_benchmark/benchmark/test/output_test_helper.cc:121: CheckCase: Check `remaining_output.eof() == false' failed. End of output reached before match for regex "^%bigo_name %bigo_str %bigo_str[ ]$" was found actual regex string "^BM_Complexity_O1_BigO [ ] [0-9][.]?[0-9]+([eE][-+][0-9]+)? ([0-9]+) [ ] [0-9][.]?[0-9]+([eE][-+][0-9]+)? ([0-9]+)[ ]$" started matching near: Aborted

LebedevRI commented 9 months ago

This is expected for internal tests, they are only designed to be used the way they are used as tests through CMake/CTest, and will not work well with other usage scenarios.