google / benchmark

A microbenchmark support library
Apache License 2.0
9.07k stars 1.63k forks source link

Some tests failing on mipsel #880

Open mr-c opened 5 years ago

mr-c commented 5 years ago

Here is the patch that we apply for the Debian package, but this disables the tests for all architectures

https://salsa.debian.org/science-team/benchmark/blob/262160dd0586f73f27c6199cdf18987e87604fbb/debian/patches/02_disable_2_tests.patch

-------------------------------
Testing ConsoleReporter Output
-------------------------------
2019-09-22 22:52:05
Running /<<PKGBUILDDIR>>/obj-mipsel-linux-gnu/test/user_counters_test
Run on (4 X 718.84 MHz CPU s)
CPU Caches:
  L1 Data 64K (x4)
  L1 Instruction 64K (x4)
  L2 Unified 1024K (x1)
Load Average: 2.92, 4.19, 3.97
***WARNING*** Library was built as DEBUG. Timings may be affected.
/<<PKGBUILDDIR>>/test/output_test_helper.cc:112: CheckCase: Check `remaining_output.eof() == false' failed. End of output reached before match for regex "^BM_Counters_WithBytesAndItemsPSec %console_report bar=%hrfloat bytes_per_second=%hrfloat/s foo=%hrfloat items_per_second=%hrfloat/s$" was found
    actual regex string "^BM_Counters_WithBytesAndItemsPSec [ ]*([0-9]+[.])?[0-9]+[ ]+ns [ ]*([0-9]+[.])?[0-9]+[ ]+ns [ ]*[0-9]+ bar=[0-9]*[.]?[0-9]+([eE][-+][0-9]+)?[kMGTPEZYmunpfazy]? bytes_per_second=[0-9]*[.]?[0-9]+([eE][-+][0-9]+)?[kMGTPEZYmunpfazy]?/s foo=[0-9]*[.]?[0-9]+([eE][-+][0-9]+)?[kMGTPEZYmunpfazy]? items_per_second=[0-9]*[.]?[0-9]+([eE][-+][0-9]+)?[kMGTPEZYmunpfazy]?/s$"
    started matching near: BM_Counters_WithBytesAndItemsPSec          0.000 ns        0.000 ns   1000000000 bar=10 bytes_per_second=inf/s foo=1 items_per_second=inf/s
-------------------------------
Testing ConsoleReporter Output
-------------------------------
2019-09-22 22:52:06
Running /<<PKGBUILDDIR>>/obj-mipsel-linux-gnu/test/user_counters_tabular_test
Run on (4 X 718.84 MHz CPU s)
CPU Caches:
  L1 Data 64K (x4)
  L1 Instruction 64K (x4)
  L2 Unified 1024K (x1)
Load Average: 2.92, 4.19, 3.97
***WARNING*** Library was built as DEBUG. Timings may be affected.
/<<PKGBUILDDIR>>/test/output_test_helper.cc:106: CheckCase: Check `TC.match_rule != MR_Next' failed. Expected line "BM_CounterRates_Tabular/threads:1       0.000 ns        0.000 ns   1000000000      inf/s      inf/s      inf/s      inf/s      inf/s      inf/s" to match regex "^BM_CounterRates_Tabular/threads:%int %console_report [ ]*%hrfloat/s [ ]*%hrfloat/s [ ]*%hrfloat/s [ ]*%hrfloat/s [ ]*%hrfloat/s [ ]*%hrfloat/s$"
    actual regex string "^BM_CounterRates_Tabular/threads:[ ]*[0-9]+ [ ]*([0-9]+[.])?[0-9]+[ ]+ns [ ]*([0-9]+[.])?[0-9]+[ ]+ns [ ]*[0-9]+ [ ]*[0-9]*[.]?[0-9]+([eE][-+][0-9]+)?[kMGTPEZYmunpfazy]?/s [ ]*[0-9]*[.]?[0-9]+([eE][-+][0-9]+)?[kMGTPEZYmunpfazy]?/s [ ]*[0-9]*[.]?[0-9]+([eE][-+][0-9]+)?[kMGTPEZYmunpfazy]?/s [ ]*[0-9]*[.]?[0-9]+([eE][-+][0-9]+)?[kMGTPEZYmunpfazy]?/s [ ]*[0-9]*[.]?[0-9]+([eE][-+][0-9]+)?[kMGTPEZYmunpfazy]?/s [ ]*[0-9]*[.]?[0-9]+([eE][-+][0-9]+)?[kMGTPEZYmunpfazy]?/s$"
    started matching near: BM_CounterRates_Tabular/threads:1       0.000 ns        0.000 ns   1000000000      inf/s      inf/s      inf/s      inf/s      inf/s      inf/s

60/61 Test #56: complexity_benchmark .......................Child aborted***Exception:   2.74 sec
-------------------------------
Testing ConsoleReporter Output
-------------------------------
2019-09-22 22:52:07
Running /<<PKGBUILDDIR>>/obj-mipsel-linux-gnu/test/complexity_test
Run on (4 X 718.84 MHz CPU s)
CPU Caches:
  L1 Data 64K (x4)
  L1 Instruction 64K (x4)
  L2 Unified 1024K (x1)
Load Average: 2.92, 4.19, 3.97
***WARNING*** Library was built as DEBUG. Timings may be affected.
/<<PKGBUILDDIR>>/test/output_test_helper.cc:106: CheckCase: Check `TC.match_rule != MR_Next' failed. Expected line "BM_ComplexityCaptureArgs/capture_test_RMS          2 %           nan %    " to match regex "^%rms_name %rms %rms[ ]*$"
    actual regex string "^BM_ComplexityCaptureArgs/capture_test_RMS [ ]*[0-9]+ % [ ]*[0-9]+ %[ ]*$"
    started matching near: BM_ComplexityCaptureArgs/capture_test_RMS          2 %           nan %    

https://buildd.debian.org/status/fetch.php?pkg=benchmark&arch=mipsel&ver=1.5.0-1&stamp=1569192757&raw=0

dmah42 commented 5 years ago

looks like DoNotOptimize might not be working for mipsel (https://github.com/google/benchmark/blob/master/test/user_counters_test.cc#L67)