t/porting/bench.t: Failures on Fedora Linux 38 / gcc and g++ builds

jkeenan commented 1 year ago

One of our steady smoke-testers has recently begun testing on Fedora Linux 38 (Server Edition). t/porting/bench.t is now failing on both gcc and g++ builds. Here is the list of failed runs:

https://perl5.test-smoke.org/submatrix?test=../t/porting/bench.t&pversion=5.37.12

Sample g++ failure: https://perl5.test-smoke.org/report/5034442

This test was passing on the previous version of this OS, where the build was done with g++-12. So I don't know whether this is a Fedora-specific problem or a problem with version 13 of g++ and gcc.

I don't have access to version 13 of gcc or g++, so if anyone else has those compilers, please build blead with them and run t/porting/bench.t.

@cjg-cguevara

Thank you very much. Jim Keenan

jkeenan commented 1 year ago

Failures with using gcc-13 for build: https://perl5.test-smoke.org/report/5034471

If the test were passing, this is what the output of the failing unit tests would look like:

ok 52 - --write should produce no output (1 perl)
ok 53 - --write should create a non-empty results file (1 perl)
ok 54 - --write should produce no output (2 perls)
ok 55 - --write should create a non-empty results file (2 perls)
ok 56 - basic cachegrind raw format; 1 perl
ok 57 - basic cachegrind raw compact format; 1 perl
ok 58 - basic cachegrind raw average format; 1 perl
ok 59 - basic cachegrind --fields; 1 perl
ok 60 - basic cachegrind percent format; 2 perls
ok 61 - basic cachegrind percent format, norm; 2 perls
ok 62 - basic cachegrind percent format, norm -2; 2 perls
ok 63 - basic cachegrind percent format, sort; 2 perls
ok 64 - basic cachegrind percent format, sort, norm; 2 perls
ok 65 - basic cachegrind raw format; 2 perls
ok 66 - basic cachegrind raw format, norm; 2 perls
ok 67 - basic cachegrind raw format, sort, norm; 2 perls
ok 68 - basic cachegrind raw format, sort, norm; 2 perls
ok 69 - basic cachegrind compact format; 2 perls
ok 70 - basic cachegrind average format; 2 perls
ok 71 - basic cachegrind --fields; 2 perls
ok 72 - basic cachegrind compact, fields; 2 perls
ok 73 - basic cachegrind 1 field; 2 perls

Leont commented 1 year ago

I can reproduce the issue on arch with gcc 13 (but I don't know if it failed previously)

ok 1 - Porting/bench.pl exists and is executable
ok 2 - croak: basic unknown option
ok 3 - croak: unknown --field
ok 4 - croak: unknown --action
ok 5 - croak: invalid --sort
ok 6 - croak: unknown --sort field
ok 7 - croak: --action-selftest with executable
ok 8 - croak: invalid --tests regex
ok 9 - croak: unknown test in --tests
ok 10 - croak: unknown test in --tests --verbose
ok 11 - croak: no --tests to run 
ok 12 - croak: non-existent --benchfile 
ok 13 - croak: --benchfile with syntax error
ok 14 - croak: --benchfile which returns 0
ok 15 - croak: --benchfile with odd number of entries
ok 16 - croak: --benchfile with invalid test name
ok 17 - croak: --benchfile with invalid test hash key
ok 18 - croak: select-a-perl out of range
ok 19 - croak: select-a-perl out of range
ok 20 - croak: select-a-perl out of range
ok 21 - croak: select-a-perl unrecognised
ok 22 - croak: select-a-perl ambiguous
ok 23 - croak: ./perl --foo
ok 24 - croak: --args without perl
ok 25 - croak: --env without perl
ok 26 - croak: --args without value
ok 27 - croak: --env without value
ok 28 - croak: --env without =value
ok 29 - croak: duplicate label ./perl ./perl
ok 30 - croak: duplicate label =A =A
ok 31 - croak: duplicate label --read=... --read=...
ok 32 - croak: duplicate label --read=... ./perl
ok 33 - croak:  no such perl
ok 34 - croak: cachegrind output format 
ok 35 - croak: --bisect=Ir
ok 36 - croak: --bisect=Ir,1
ok 37 - croak: --bisect=Ir,1,2,3
ok 38 - croak: --bisect=Ir,1,x
ok 39 - croak: --bisect=Ir,x,2
ok 40 - croak: --bisect=boz,1,2
ok 41 - croak: --bisect=boz,2,1
ok 42 - croak: non-existent --read file 
ok 43 - croak: --read version
ok 44 - croak: benchfile with read
ok 45 - croak: no input
ok 46 - croak: need 2 perls
ok 47 - croak: --bisect, need 1 perls
ok 48 - croak: --bisect one test only
ok 49 - croak: --write open error
ok 50 - croak: --read callsub, callsub2
ok 51 - croak: --read callsub, callsub2 --verbose
# running cachegrind for 1st perl; may be slow...
not ok 52 - --write should produce no output (1 perl)
# Failed test 52 - --write should produce no output (1 perl) at t/porting/bench.t line 408
#      got "Error: can\'t parse \'Ir_mm\' field from cachegrind output:\n==708421== Cachegrind, a cache and branch-prediction profiler\n==708421== Copyright (C) 2002-2017, and GNU GPL\'d, by Nicholas Nethercote et al.\n==708421== Using Valgrind-3.21.0 and LibVEX; rerun with -h for copyright info\n==708421== Command: /home/leont/Code/perl/perl - 20\n==708421== \n==708421== \n==708421== I refs:        1,344,629\n==708421== \n==708421== Branches:        274,990  (267,676 cond + 7,314 ind)\n==708421== Mispredicts:      17,718  ( 15,587 cond + 2,131 ind)\n==708421== Mispred rate:        6.4% (    5.8%     +  29.1%   )\n"
# expected ""
#  diff at 0
#    after "Error: can\'t parse \'Ir_mm\' field from ca"
#     have "Error: can\'t parse \'Ir_mm\' field from ca"
#     want ""
not ok 53 - --write should create a non-empty results file (1 perl)
# Failed test 53 - --write should create a non-empty results file (1 perl) at t/porting/bench.t line 409
# running cachegrind for 2nd perl; may be slow...
not ok 54 - --write should produce no output (2 perls)
# Failed test 54 - --write should produce no output (2 perls) at t/porting/bench.t line 416
#      got "Error: can\'t open \'t/tmp_ANGTM_B\' for reading: No such file or directory\n"
# expected ""
#  diff at 0
#    after "Error: can\'t open \'t/tmp_ANGTM_B\' for re"
#     have "Error: can\'t open \'t/tmp_ANGTM_B\' for re"
#     want ""
# got: Error: can't open 't/tmp_ANGTM_B' for reading: No such file or directory
not ok 55 - --write should create a non-empty results file (2 perls)
# Failed test 55 - --write should create a non-empty results file (2 perls) at t/porting/bench.t line 418
not ok 56 - basic cachegrind raw format; 1 perl
# Failed test 56 - basic cachegrind raw format; 1 perl at t/porting/bench.t line 425
#      got 'Error: can\'t open \'t/tmp_ANGTM_B\' for reading: No such file or directory\n'
# expected /(?^:\AKey\:\
#  *Ir +Instruction +read\
#  *Dr +Data +read\
#  *Dw +Data +write\
#  *COND +conditional +branches\
#  *IND +indirect +branches\
#  *_m +branch +predict +miss\
#  *_m1 +level +1 +cache +miss\
#  *_mm +last +cache +\(e\.g\. +L3\) +miss\
#  *\- +indeterminate +percentage +\(e\.g\. +1\/0\)\
# \
# The +numbers +represent +raw +counts +per +loop +iteration\.\
# \
# call\:\:sub\:\:empty\
# function +call +with +no +args +or +body\
# \
#  *p0\
#  *-+\
#  *Ir +(\s*-?\d+\.\d|\s*-)\
#  *Dr +(\s*-?\d+\.\d|\s*-)\
#  *Dw +(\s*-?\d+\.\d|\s*-)\
#  *COND +(\s*-?\d+\.\d|\s*-)\
#  *IND +(\s*-?\d+\.\d|\s*-)\
# \
# COND_m +(\s*-?\d+\.\d|\s*-)\
#  *IND_m +(\s*-?\d+\.\d|\s*-)\
# \
#  *Ir_m1 +(\s*-?\d+\.\d|\s*-)\
#  *Dr_m1 +(\s*-?\d+\.\d|\s*-)\
#  *Dw_m1 +(\s*-?\d+\.\d|\s*-)\
# \
#  *Ir_mm +(\s*-?\d+\.\d|\s*-)\
#  *Dr_mm +(\s*-?\d+\.\d|\s*-)\
#  *Dw_mm +(\s*-?\d+\.\d|\s*-)\
# \z)/
not ok 57 - basic cachegrind raw compact format; 1 perl
# Failed test 57 - basic cachegrind raw compact format; 1 perl at t/porting/bench.t line 430
#      got 'Error: can\'t open \'t/tmp_ANGTM_B\' for reading: No such file or directory\n'
# expected /(?^:\AKey\:\
#  *Ir +Instruction +read\
#  *Dr +Data +read\
#  *Dw +Data +write\
#  *COND +conditional +branches\
#  *IND +indirect +branches\
#  *_m +branch +predict +miss\
#  *_m1 +level +1 +cache +miss\
#  *_mm +last +cache +\(e\.g\. +L3\) +miss\
#  *\- +indeterminate +percentage +\(e\.g\. +1\/0\)\
# \
# The +numbers +represent +raw +counts +per +loop +iteration\.\
# \
# Results +for +p0\
# \
#  *Ir +Dr +Dw +COND +IND +COND_m +IND_m +Ir_m1 +Dr_m1 +Dw_m1 +Ir_mm +Dr_mm +Dw_mm\
#  *-+ +-+ +-+ +-+ +-+ +-+ +-+ +-+ +-+ +-+ +-+ +-+ +-+\
#  *(\s*-?\d+\.\d|\s*-) +(\s*-?\d+\.\d|\s*-) +(\s*-?\d+\.\d|\s*-) +(\s*-?\d+\.\d|\s*-) +(\s*-?\d+\.\d|\s*-) +(\s*-?\d+\.\d|\s*-) +(\s*-?\d+\.\d|\s*-) +(\s*-?\d+\.\d|\s*-) +(\s*-?\d+\.\d|\s*-) +(\s*-?\d+\.\d|\s*-) +(\s*-?\d+\.\d|\s*-) +(\s*-?\d+\.\d|\s*-) +(\s*-?\d+\.\d|\s*-) +call\:\:sub\:\:empty +function +call +with +no +args +or +body\
# \z)/
not ok 58 - basic cachegrind raw average format; 1 perl
# Failed test 58 - basic cachegrind raw average format; 1 perl at t/porting/bench.t line 435
#      got 'Error: can\'t open \'t/tmp_ANGTM_B\' for reading: No such file or directory\n'
# expected /(?^:\AKey\:\
#  *Ir +Instruction +read\
#  *Dr +Data +read\
#  *Dw +Data +write\
#  *COND +conditional +branches\
#  *IND +indirect +branches\
#  *_m +branch +predict +miss\
#  *_m1 +level +1 +cache +miss\
#  *_mm +last +cache +\(e\.g\. +L3\) +miss\
#  *\- +indeterminate +percentage +\(e\.g\. +1\/0\)\
# \
# The +numbers +represent +raw +counts +per +loop +iteration\.\
# \
# AVERAGE\
# \
#  *p0\
#  *-+\
#  *Ir +(\s*-?\d+\.\d|\s*-)\
#  *Dr +(\s*-?\d+\.\d|\s*-)\
#  *Dw +(\s*-?\d+\.\d|\s*-)\
#  *COND +(\s*-?\d+\.\d|\s*-)\
#  *IND +(\s*-?\d+\.\d|\s*-)\
# \
# COND_m +(\s*-?\d+\.\d|\s*-)\
#  *IND_m +(\s*-?\d+\.\d|\s*-)\
# \
#  *Ir_m1 +(\s*-?\d+\.\d|\s*-)\
#  *Dr_m1 +(\s*-?\d+\.\d|\s*-)\
#  *Dw_m1 +(\s*-?\d+\.\d|\s*-)\
# \
#  *Ir_mm +(\s*-?\d+\.\d|\s*-)\
#  *Dr_mm +(\s*-?\d+\.\d|\s*-)\
#  *Dw_mm +(\s*-?\d+\.\d|\s*-)\
# \z)/
not ok 59 - basic cachegrind --fields; 1 perl
# Failed test 59 - basic cachegrind --fields; 1 perl at t/porting/bench.t line 440
#      got 'Error: can\'t open \'t/tmp_ANGTM_B\' for reading: No such file or directory\n'
# expected /(?^:\AKey\:\
#  *Ir +Instruction +read\
#  *Dr +Data +read\
#  *Dw +Data +write\
#  *COND +conditional +branches\
#  *IND +indirect +branches\
#  *_m +branch +predict +miss\
#  *_m1 +level +1 +cache +miss\
#  *_mm +last +cache +\(e\.g\. +L3\) +miss\
#  *\- +indeterminate +percentage +\(e\.g\. +1\/0\)\
# \
# The +numbers +represent +raw +counts +per +loop +iteration\.\
# \
# call\:\:sub\:\:empty\
# function +call +with +no +args +or +body\
# \
#  *p0\
#  *-+\
#  *Ir +(\s*-?\d+\.\d|\s*-)\
#  *Dr +(\s*-?\d+\.\d|\s*-)\
# \z)/
not ok 60 - basic cachegrind percent format; 2 perls
# Failed test 60 - basic cachegrind percent format; 2 perls at t/porting/bench.t line 447
#      got 'Error: can\'t open \'t/tmp_ANGTM_C\' for reading: No such file or directory\n'
# expected /(?^:\AKey\:\
#  *Ir +Instruction +read\
#  *Dr +Data +read\
#  *Dw +Data +write\
#  *COND +conditional +branches\
#  *IND +indirect +branches\
#  *_m +branch +predict +miss\
#  *_m1 +level +1 +cache +miss\
#  *_mm +last +cache +\(e\.g\. +L3\) +miss\
#  *\- +indeterminate +percentage +\(e\.g\. +1\/0\)\
# \
# The +numbers +represent +relative +counts +per +loop +iteration\, +compared +to\
# p0 +at +100\.0\%\.\
# Higher +is +better\: +for +example\, +using +half +as +many +instructions +gives +200\%\,\
# while +using +twice +as +many +gives +50\%\.\
# \
# call\:\:sub\:\:empty\
# function +call +with +no +args +or +body\
# \
#  *p0 +p1\
#  *-+ +-+\
#  *Ir +100\.00 +(\s*-?\d+\.\d\d|\s*-)\
#  *Dr +100\.00 +(\s*-?\d+\.\d\d|\s*-)\
#  *Dw +100\.00 +(\s*-?\d+\.\d\d|\s*-)\
#  *COND +100\.00 +(\s*-?\d+\.\d\d|\s*-)\
#  *IND +100\.00 +(\s*-?\d+\.\d\d|\s*-)\
# \
# COND_m +100\.00 +(\s*-?\d+\.\d\d|\s*-)\
#  *IND_m +100\.00 +(\s*-?\d+\.\d\d|\s*-)\
# \
#  *Ir_m1 +100\.00 +(\s*-?\d+\.\d\d|\s*-)\
#  *Dr_m1 +100\.00 +(\s*-?\d+\.\d\d|\s*-)\
#  *Dw_m1 +100\.00 +(\s*-?\d+\.\d\d|\s*-)\
# \
#  *Ir_mm +100\.00 +(\s*-?\d+\.\d\d|\s*-)\
#  *Dr_mm +100\.00 +(\s*-?\d+\.\d\d|\s*-)\
#  *Dw_mm +100\.00 +(\s*-?\d+\.\d\d|\s*-)\
# \z)/
not ok 61 - basic cachegrind percent format, norm; 2 perls
# Failed test 61 - basic cachegrind percent format, norm; 2 perls at t/porting/bench.t line 452
#      got 'Error: can\'t open \'t/tmp_ANGTM_C\' for reading: No such file or directory\n'
# expected /(?^:\AKey\:\
#  *Ir +Instruction +read\
#  *Dr +Data +read\
#  *Dw +Data +write\
#  *COND +conditional +branches\
#  *IND +indirect +branches\
#  *_m +branch +predict +miss\
#  *_m1 +level +1 +cache +miss\
#  *_mm +last +cache +\(e\.g\. +L3\) +miss\
#  *\- +indeterminate +percentage +\(e\.g\. +1\/0\)\
# \
# The +numbers +represent +relative +counts +per +loop +iteration\, +compared +to\
# p0 +at +100\.0\%\.\
# Higher +is +better\: +for +example\, +using +half +as +many +instructions +gives +200\%\,\
# while +using +twice +as +many +gives +50\%\.\
# \
# call\:\:sub\:\:empty\
# function +call +with +no +args +or +body\
# \
#  *p0 +p1\
#  *-+ +-+\
#  *Ir +100\.00 +(\s*-?\d+\.\d\d|\s*-)\
#  *Dr +100\.00 +(\s*-?\d+\.\d\d|\s*-)\
#  *Dw +100\.00 +(\s*-?\d+\.\d\d|\s*-)\
#  *COND +100\.00 +(\s*-?\d+\.\d\d|\s*-)\
#  *IND +100\.00 +(\s*-?\d+\.\d\d|\s*-)\
# \
# COND_m +100\.00 +(\s*-?\d+\.\d\d|\s*-)\
#  *IND_m +100\.00 +(\s*-?\d+\.\d\d|\s*-)\
# \
#  *Ir_m1 +100\.00 +(\s*-?\d+\.\d\d|\s*-)\
#  *Dr_m1 +100\.00 +(\s*-?\d+\.\d\d|\s*-)\
#  *Dw_m1 +100\.00 +(\s*-?\d+\.\d\d|\s*-)\
# \
#  *Ir_mm +100\.00 +(\s*-?\d+\.\d\d|\s*-)\
#  *Dr_mm +100\.00 +(\s*-?\d+\.\d\d|\s*-)\
#  *Dw_mm +100\.00 +(\s*-?\d+\.\d\d|\s*-)\
# \z)/
not ok 62 - basic cachegrind percent format, norm -2; 2 perls
# Failed test 62 - basic cachegrind percent format, norm -2; 2 perls at t/porting/bench.t line 457
#      got 'Error: can\'t open \'t/tmp_ANGTM_C\' for reading: No such file or directory\n'
# expected /(?^:\AKey\:\
#  *Ir +Instruction +read\
#  *Dr +Data +read\
#  *Dw +Data +write\
#  *COND +conditional +branches\
#  *IND +indirect +branches\
#  *_m +branch +predict +miss\
#  *_m1 +level +1 +cache +miss\
#  *_mm +last +cache +\(e\.g\. +L3\) +miss\
#  *\- +indeterminate +percentage +\(e\.g\. +1\/0\)\
# \
# The +numbers +represent +relative +counts +per +loop +iteration\, +compared +to\
# p0 +at +100\.0\%\.\
# Higher +is +better\: +for +example\, +using +half +as +many +instructions +gives +200\%\,\
# while +using +twice +as +many +gives +50\%\.\
# \
# call\:\:sub\:\:empty\
# function +call +with +no +args +or +body\
# \
#  *p0 +p1\
#  *-+ +-+\
#  *Ir +100\.00 +(\s*-?\d+\.\d\d|\s*-)\
#  *Dr +100\.00 +(\s*-?\d+\.\d\d|\s*-)\
#  *Dw +100\.00 +(\s*-?\d+\.\d\d|\s*-)\
#  *COND +100\.00 +(\s*-?\d+\.\d\d|\s*-)\
#  *IND +100\.00 +(\s*-?\d+\.\d\d|\s*-)\
# \
# COND_m +100\.00 +(\s*-?\d+\.\d\d|\s*-)\
#  *IND_m +100\.00 +(\s*-?\d+\.\d\d|\s*-)\
# \
#  *Ir_m1 +100\.00 +(\s*-?\d+\.\d\d|\s*-)\
#  *Dr_m1 +100\.00 +(\s*-?\d+\.\d\d|\s*-)\
#  *Dw_m1 +100\.00 +(\s*-?\d+\.\d\d|\s*-)\
# \
#  *Ir_mm +100\.00 +(\s*-?\d+\.\d\d|\s*-)\
#  *Dr_mm +100\.00 +(\s*-?\d+\.\d\d|\s*-)\
#  *Dw_mm +100\.00 +(\s*-?\d+\.\d\d|\s*-)\
# \z)/
not ok 63 - basic cachegrind percent format, sort; 2 perls
# Failed test 63 - basic cachegrind percent format, sort; 2 perls at t/porting/bench.t line 462
#      got 'Error: can\'t open \'t/tmp_ANGTM_C\' for reading: No such file or directory\n'
# expected /(?^:\AKey\:\
#  *Ir +Instruction +read\
#  *Dr +Data +read\
#  *Dw +Data +write\
#  *COND +conditional +branches\
#  *IND +indirect +branches\
#  *_m +branch +predict +miss\
#  *_m1 +level +1 +cache +miss\
#  *_mm +last +cache +\(e\.g\. +L3\) +miss\
#  *\- +indeterminate +percentage +\(e\.g\. +1\/0\)\
# \
# The +numbers +represent +relative +counts +per +loop +iteration\, +compared +to\
# p0 +at +100\.0\%\.\
# Higher +is +better\: +for +example\, +using +half +as +many +instructions +gives +200\%\,\
# while +using +twice +as +many +gives +50\%\.\
# \
# call\:\:sub\:\:empty\
# function +call +with +no +args +or +body\
# \
#  *p0 +p1\
#  *-+ +-+\
#  *Ir +100\.00 +(\s*-?\d+\.\d\d|\s*-)\
#  *Dr +100\.00 +(\s*-?\d+\.\d\d|\s*-)\
#  *Dw +100\.00 +(\s*-?\d+\.\d\d|\s*-)\
#  *COND +100\.00 +(\s*-?\d+\.\d\d|\s*-)\
#  *IND +100\.00 +(\s*-?\d+\.\d\d|\s*-)\
# \
# COND_m +100\.00 +(\s*-?\d+\.\d\d|\s*-)\
#  *IND_m +100\.00 +(\s*-?\d+\.\d\d|\s*-)\
# \
#  *Ir_m1 +100\.00 +(\s*-?\d+\.\d\d|\s*-)\
#  *Dr_m1 +100\.00 +(\s*-?\d+\.\d\d|\s*-)\
#  *Dw_m1 +100\.00 +(\s*-?\d+\.\d\d|\s*-)\
# \
#  *Ir_mm +100\.00 +(\s*-?\d+\.\d\d|\s*-)\
#  *Dr_mm +100\.00 +(\s*-?\d+\.\d\d|\s*-)\
#  *Dw_mm +100\.00 +(\s*-?\d+\.\d\d|\s*-)\
# \z)/
not ok 64 - basic cachegrind percent format, sort, norm; 2 perls
# Failed test 64 - basic cachegrind percent format, sort, norm; 2 perls at t/porting/bench.t line 467
#      got 'Error: can\'t open \'t/tmp_ANGTM_C\' for reading: No such file or directory\n'
# expected /(?^:\AKey\:\
#  *Ir +Instruction +read\
#  *Dr +Data +read\
#  *Dw +Data +write\
#  *COND +conditional +branches\
#  *IND +indirect +branches\
#  *_m +branch +predict +miss\
#  *_m1 +level +1 +cache +miss\
#  *_mm +last +cache +\(e\.g\. +L3\) +miss\
#  *\- +indeterminate +percentage +\(e\.g\. +1\/0\)\
# \
# The +numbers +represent +relative +counts +per +loop +iteration\, +compared +to\
# p0 +at +100\.0\%\.\
# Higher +is +better\: +for +example\, +using +half +as +many +instructions +gives +200\%\,\
# while +using +twice +as +many +gives +50\%\.\
# \
# call\:\:sub\:\:empty\
# function +call +with +no +args +or +body\
# \
#  *p0 +p1\
#  *-+ +-+\
#  *Ir +100\.00 +(\s*-?\d+\.\d\d|\s*-)\
#  *Dr +100\.00 +(\s*-?\d+\.\d\d|\s*-)\
#  *Dw +100\.00 +(\s*-?\d+\.\d\d|\s*-)\
#  *COND +100\.00 +(\s*-?\d+\.\d\d|\s*-)\
#  *IND +100\.00 +(\s*-?\d+\.\d\d|\s*-)\
# \
# COND_m +100\.00 +(\s*-?\d+\.\d\d|\s*-)\
#  *IND_m +100\.00 +(\s*-?\d+\.\d\d|\s*-)\
# \
#  *Ir_m1 +100\.00 +(\s*-?\d+\.\d\d|\s*-)\
#  *Dr_m1 +100\.00 +(\s*-?\d+\.\d\d|\s*-)\
#  *Dw_m1 +100\.00 +(\s*-?\d+\.\d\d|\s*-)\
# \
#  *Ir_mm +100\.00 +(\s*-?\d+\.\d\d|\s*-)\
#  *Dr_mm +100\.00 +(\s*-?\d+\.\d\d|\s*-)\
#  *Dw_mm +100\.00 +(\s*-?\d+\.\d\d|\s*-)\
# \z)/
not ok 65 - basic cachegrind raw format; 2 perls
# Failed test 65 - basic cachegrind raw format; 2 perls at t/porting/bench.t line 472
#      got 'Error: can\'t open \'t/tmp_ANGTM_C\' for reading: No such file or directory\n'
# expected /(?^:\AKey\:\
#  *Ir +Instruction +read\
#  *Dr +Data +read\
#  *Dw +Data +write\
#  *COND +conditional +branches\
#  *IND +indirect +branches\
#  *_m +branch +predict +miss\
#  *_m1 +level +1 +cache +miss\
#  *_mm +last +cache +\(e\.g\. +L3\) +miss\
#  *\- +indeterminate +percentage +\(e\.g\. +1\/0\)\
# \
# The +numbers +represent +raw +counts +per +loop +iteration\.\
# \
# call\:\:sub\:\:empty\
# function +call +with +no +args +or +body\
# \
#  *p0 +p1\
#  *-+ +-+\
#  *Ir +(\s*-?\d+\.\d|\s*-) +(\s*-?\d+\.\d|\s*-)\
#  *Dr +(\s*-?\d+\.\d|\s*-) +(\s*-?\d+\.\d|\s*-)\
#  *Dw +(\s*-?\d+\.\d|\s*-) +(\s*-?\d+\.\d|\s*-)\
#  *COND +(\s*-?\d+\.\d|\s*-) +(\s*-?\d+\.\d|\s*-)\
#  *IND +(\s*-?\d+\.\d|\s*-) +(\s*-?\d+\.\d|\s*-)\
# \
# COND_m +(\s*-?\d+\.\d|\s*-) +(\s*-?\d+\.\d|\s*-)\
#  *IND_m +(\s*-?\d+\.\d|\s*-) +(\s*-?\d+\.\d|\s*-)\
# \
#  *Ir_m1 +(\s*-?\d+\.\d|\s*-) +(\s*-?\d+\.\d|\s*-)\
#  *Dr_m1 +(\s*-?\d+\.\d|\s*-) +(\s*-?\d+\.\d|\s*-)\
#  *Dw_m1 +(\s*-?\d+\.\d|\s*-) +(\s*-?\d+\.\d|\s*-)\
# \
#  *Ir_mm +(\s*-?\d+\.\d|\s*-) +(\s*-?\d+\.\d|\s*-)\
#  *Dr_mm +(\s*-?\d+\.\d|\s*-) +(\s*-?\d+\.\d|\s*-)\
#  *Dw_mm +(\s*-?\d+\.\d|\s*-) +(\s*-?\d+\.\d|\s*-)\
# \z)/
not ok 66 - basic cachegrind raw format, norm; 2 perls
# Failed test 66 - basic cachegrind raw format, norm; 2 perls at t/porting/bench.t line 477
#      got 'Error: can\'t open \'t/tmp_ANGTM_C\' for reading: No such file or directory\n'
# expected /(?^:\AKey\:\
#  *Ir +Instruction +read\
#  *Dr +Data +read\
#  *Dw +Data +write\
#  *COND +conditional +branches\
#  *IND +indirect +branches\
#  *_m +branch +predict +miss\
#  *_m1 +level +1 +cache +miss\
#  *_mm +last +cache +\(e\.g\. +L3\) +miss\
#  *\- +indeterminate +percentage +\(e\.g\. +1\/0\)\
# \
# The +numbers +represent +raw +counts +per +loop +iteration\.\
# \
# call\:\:sub\:\:empty\
# function +call +with +no +args +or +body\
# \
#  *p0 +p1\
#  *-+ +-+\
#  *Ir +(\s*-?\d+\.\d|\s*-) +(\s*-?\d+\.\d|\s*-)\
#  *Dr +(\s*-?\d+\.\d|\s*-) +(\s*-?\d+\.\d|\s*-)\
#  *Dw +(\s*-?\d+\.\d|\s*-) +(\s*-?\d+\.\d|\s*-)\
#  *COND +(\s*-?\d+\.\d|\s*-) +(\s*-?\d+\.\d|\s*-)\
#  *IND +(\s*-?\d+\.\d|\s*-) +(\s*-?\d+\.\d|\s*-)\
# \
# COND_m +(\s*-?\d+\.\d|\s*-) +(\s*-?\d+\.\d|\s*-)\
#  *IND_m +(\s*-?\d+\.\d|\s*-) +(\s*-?\d+\.\d|\s*-)\
# \
#  *Ir_m1 +(\s*-?\d+\.\d|\s*-) +(\s*-?\d+\.\d|\s*-)\
#  *Dr_m1 +(\s*-?\d+\.\d|\s*-) +(\s*-?\d+\.\d|\s*-)\
#  *Dw_m1 +(\s*-?\d+\.\d|\s*-) +(\s*-?\d+\.\d|\s*-)\
# \
#  *Ir_mm +(\s*-?\d+\.\d|\s*-) +(\s*-?\d+\.\d|\s*-)\
#  *Dr_mm +(\s*-?\d+\.\d|\s*-) +(\s*-?\d+\.\d|\s*-)\
#  *Dw_mm +(\s*-?\d+\.\d|\s*-) +(\s*-?\d+\.\d|\s*-)\
# \z)/
not ok 67 - basic cachegrind raw format, sort, norm; 2 perls
# Failed test 67 - basic cachegrind raw format, sort, norm; 2 perls at t/porting/bench.t line 482
#      got 'Error: can\'t open \'t/tmp_ANGTM_C\' for reading: No such file or directory\n'
# expected /(?^:\AKey\:\
#  *Ir +Instruction +read\
#  *Dr +Data +read\
#  *Dw +Data +write\
#  *COND +conditional +branches\
#  *IND +indirect +branches\
#  *_m +branch +predict +miss\
#  *_m1 +level +1 +cache +miss\
#  *_mm +last +cache +\(e\.g\. +L3\) +miss\
#  *\- +indeterminate +percentage +\(e\.g\. +1\/0\)\
# \
# The +numbers +represent +raw +counts +per +loop +iteration\.\
# \
# call\:\:sub\:\:empty\
# function +call +with +no +args +or +body\
# \
#  *p0 +p1\
#  *-+ +-+\
#  *Ir +(\s*-?\d+\.\d|\s*-) +(\s*-?\d+\.\d|\s*-)\
#  *Dr +(\s*-?\d+\.\d|\s*-) +(\s*-?\d+\.\d|\s*-)\
#  *Dw +(\s*-?\d+\.\d|\s*-) +(\s*-?\d+\.\d|\s*-)\
#  *COND +(\s*-?\d+\.\d|\s*-) +(\s*-?\d+\.\d|\s*-)\
#  *IND +(\s*-?\d+\.\d|\s*-) +(\s*-?\d+\.\d|\s*-)\
# \
# COND_m +(\s*-?\d+\.\d|\s*-) +(\s*-?\d+\.\d|\s*-)\
#  *IND_m +(\s*-?\d+\.\d|\s*-) +(\s*-?\d+\.\d|\s*-)\
# \
#  *Ir_m1 +(\s*-?\d+\.\d|\s*-) +(\s*-?\d+\.\d|\s*-)\
#  *Dr_m1 +(\s*-?\d+\.\d|\s*-) +(\s*-?\d+\.\d|\s*-)\
#  *Dw_m1 +(\s*-?\d+\.\d|\s*-) +(\s*-?\d+\.\d|\s*-)\
# \
#  *Ir_mm +(\s*-?\d+\.\d|\s*-) +(\s*-?\d+\.\d|\s*-)\
#  *Dr_mm +(\s*-?\d+\.\d|\s*-) +(\s*-?\d+\.\d|\s*-)\
#  *Dw_mm +(\s*-?\d+\.\d|\s*-) +(\s*-?\d+\.\d|\s*-)\
# \z)/
not ok 68 - basic cachegrind raw format, sort, norm; 2 perls
# Failed test 68 - basic cachegrind raw format, sort, norm; 2 perls at t/porting/bench.t line 487
#      got 'Error: can\'t open \'t/tmp_ANGTM_C\' for reading: No such file or directory\n'
# expected /(?^:\AKey\:\
#  *Ir +Instruction +read\
#  *Dr +Data +read\
#  *Dw +Data +write\
#  *COND +conditional +branches\
#  *IND +indirect +branches\
#  *_m +branch +predict +miss\
#  *_m1 +level +1 +cache +miss\
#  *_mm +last +cache +\(e\.g\. +L3\) +miss\
#  *\- +indeterminate +percentage +\(e\.g\. +1\/0\)\
# \
# The +numbers +represent +raw +counts +per +loop +iteration\.\
# \
# call\:\:sub\:\:empty\
# function +call +with +no +args +or +body\
# \
#  *p0 +p1\
#  *-+ +-+\
#  *Ir +(\s*-?\d+\.\d|\s*-) +(\s*-?\d+\.\d|\s*-)\
#  *Dr +(\s*-?\d+\.\d|\s*-) +(\s*-?\d+\.\d|\s*-)\
#  *Dw +(\s*-?\d+\.\d|\s*-) +(\s*-?\d+\.\d|\s*-)\
#  *COND +(\s*-?\d+\.\d|\s*-) +(\s*-?\d+\.\d|\s*-)\
#  *IND +(\s*-?\d+\.\d|\s*-) +(\s*-?\d+\.\d|\s*-)\
# \
# COND_m +(\s*-?\d+\.\d|\s*-) +(\s*-?\d+\.\d|\s*-)\
#  *IND_m +(\s*-?\d+\.\d|\s*-) +(\s*-?\d+\.\d|\s*-)\
# \
#  *Ir_m1 +(\s*-?\d+\.\d|\s*-) +(\s*-?\d+\.\d|\s*-)\
#  *Dr_m1 +(\s*-?\d+\.\d|\s*-) +(\s*-?\d+\.\d|\s*-)\
#  *Dw_m1 +(\s*-?\d+\.\d|\s*-) +(\s*-?\d+\.\d|\s*-)\
# \
#  *Ir_mm +(\s*-?\d+\.\d|\s*-) +(\s*-?\d+\.\d|\s*-)\
#  *Dr_mm +(\s*-?\d+\.\d|\s*-) +(\s*-?\d+\.\d|\s*-)\
#  *Dw_mm +(\s*-?\d+\.\d|\s*-) +(\s*-?\d+\.\d|\s*-)\
# \z)/
not ok 69 - basic cachegrind compact format; 2 perls
# Failed test 69 - basic cachegrind compact format; 2 perls at t/porting/bench.t line 492
#      got 'Error: can\'t open \'t/tmp_ANGTM_C\' for reading: No such file or directory\n'
# expected /(?^:\AKey\:\
#  *Ir +Instruction +read\
#  *Dr +Data +read\
#  *Dw +Data +write\
#  *COND +conditional +branches\
#  *IND +indirect +branches\
#  *_m +branch +predict +miss\
#  *_m1 +level +1 +cache +miss\
#  *_mm +last +cache +\(e\.g\. +L3\) +miss\
#  *\- +indeterminate +percentage +\(e\.g\. +1\/0\)\
# \
# The +numbers +represent +relative +counts +per +loop +iteration\, +compared +to\
# p0 +at +100\.0\%\.\
# Higher +is +better\: +for +example\, +using +half +as +many +instructions +gives +200\%\,\
# while +using +twice +as +many +gives +50\%\.\
# \
# Results +for +p1\
# \
#  *Ir +Dr +Dw +COND +IND +COND_m +IND_m +Ir_m1 +Dr_m1 +Dw_m1 +Ir_mm +Dr_mm +Dw_mm\
#  *-+ +-+ +-+ +-+ +-+ +-+ +-+ +-+ +-+ +-+ +-+ +-+ +-+\
#  *(\s*-?\d+\.\d\d|\s*-) +(\s*-?\d+\.\d\d|\s*-) +(\s*-?\d+\.\d\d|\s*-) +(\s*-?\d+\.\d\d|\s*-) +(\s*-?\d+\.\d\d|\s*-) +(\s*-?\d+\.\d\d|\s*-) +(\s*-?\d+\.\d\d|\s*-) +(\s*-?\d+\.\d\d|\s*-) +(\s*-?\d+\.\d\d|\s*-) +(\s*-?\d+\.\d\d|\s*-) +(\s*-?\d+\.\d\d|\s*-) +(\s*-?\d+\.\d\d|\s*-) +(\s*-?\d+\.\d\d|\s*-) +call\:\:sub\:\:empty +function +call +with +no +args +or +body\
# \z)/
not ok 70 - basic cachegrind average format; 2 perls
# Failed test 70 - basic cachegrind average format; 2 perls at t/porting/bench.t line 497
#      got 'Error: can\'t open \'t/tmp_ANGTM_C\' for reading: No such file or directory\n'
# expected /(?^:\AKey\:\
#  *Ir +Instruction +read\
#  *Dr +Data +read\
#  *Dw +Data +write\
#  *COND +conditional +branches\
#  *IND +indirect +branches\
#  *_m +branch +predict +miss\
#  *_m1 +level +1 +cache +miss\
#  *_mm +last +cache +\(e\.g\. +L3\) +miss\
#  *\- +indeterminate +percentage +\(e\.g\. +1\/0\)\
# \
# The +numbers +represent +relative +counts +per +loop +iteration\, +compared +to\
# p0 +at +100\.0\%\.\
# Higher +is +better\: +for +example\, +using +half +as +many +instructions +gives +200\%\,\
# while +using +twice +as +many +gives +50\%\.\
# \
# AVERAGE\
# \
#  *p0 +p1\
#  *-+ +-+\
#  *Ir +100\.00 +(\s*-?\d+\.\d\d|\s*-)\
#  *Dr +100\.00 +(\s*-?\d+\.\d\d|\s*-)\
#  *Dw +100\.00 +(\s*-?\d+\.\d\d|\s*-)\
#  *COND +100\.00 +(\s*-?\d+\.\d\d|\s*-)\
#  *IND +100\.00 +(\s*-?\d+\.\d\d|\s*-)\
# \
# COND_m +100\.00 +(\s*-?\d+\.\d\d|\s*-)\
#  *IND_m +100\.00 +(\s*-?\d+\.\d\d|\s*-)\
# \
#  *Ir_m1 +100\.00 +(\s*-?\d+\.\d\d|\s*-)\
#  *Dr_m1 +100\.00 +(\s*-?\d+\.\d\d|\s*-)\
#  *Dw_m1 +100\.00 +(\s*-?\d+\.\d\d|\s*-)\
# \
#  *Ir_mm +100\.00 +(\s*-?\d+\.\d\d|\s*-)\
#  *Dr_mm +100\.00 +(\s*-?\d+\.\d\d|\s*-)\
#  *Dw_mm +100\.00 +(\s*-?\d+\.\d\d|\s*-)\
# \z)/
not ok 71 - basic cachegrind --fields; 2 perls
# Failed test 71 - basic cachegrind --fields; 2 perls at t/porting/bench.t line 502
#      got 'Error: can\'t open \'t/tmp_ANGTM_C\' for reading: No such file or directory\n'
# expected /(?^:\AKey\:\
#  *Ir +Instruction +read\
#  *Dr +Data +read\
#  *Dw +Data +write\
#  *COND +conditional +branches\
#  *IND +indirect +branches\
#  *_m +branch +predict +miss\
#  *_m1 +level +1 +cache +miss\
#  *_mm +last +cache +\(e\.g\. +L3\) +miss\
#  *\- +indeterminate +percentage +\(e\.g\. +1\/0\)\
# \
# The +numbers +represent +relative +counts +per +loop +iteration\, +compared +to\
# p0 +at +100\.0\%\.\
# Higher +is +better\: +for +example\, +using +half +as +many +instructions +gives +200\%\,\
# while +using +twice +as +many +gives +50\%\.\
# \
# call\:\:sub\:\:empty\
# function +call +with +no +args +or +body\
# \
#  *p0 +p1\
#  *-+ +-+\
#  *Ir +100\.00 +(\s*-?\d+\.\d\d|\s*-)\
#  *Dr +100\.00 +(\s*-?\d+\.\d\d|\s*-)\
# \z)/
not ok 72 - basic cachegrind compact, fields; 2 perls
# Failed test 72 - basic cachegrind compact, fields; 2 perls at t/porting/bench.t line 507
#      got 'Error: can\'t open \'t/tmp_ANGTM_C\' for reading: No such file or directory\n'
# expected /(?^:\AKey\:\
#  *Ir +Instruction +read\
#  *Dr +Data +read\
#  *Dw +Data +write\
#  *COND +conditional +branches\
#  *IND +indirect +branches\
#  *_m +branch +predict +miss\
#  *_m1 +level +1 +cache +miss\
#  *_mm +last +cache +\(e\.g\. +L3\) +miss\
#  *\- +indeterminate +percentage +\(e\.g\. +1\/0\)\
# \
# The +numbers +represent +relative +counts +per +loop +iteration\, +compared +to\
# p0 +at +100\.0\%\.\
# Higher +is +better\: +for +example\, +using +half +as +many +instructions +gives +200\%\,\
# while +using +twice +as +many +gives +50\%\.\
# \
# Results +for +p1\
# \
#  *Ir +Dr\
#  *-+ +-+\
#  *(\s*-?\d+\.\d\d|\s*-) +(\s*-?\d+\.\d\d|\s*-) +call\:\:sub\:\:empty +function +call +with +no +args +or +body\
# \z)/
not ok 73 - basic cachegrind 1 field; 2 perls
# Failed test 73 - basic cachegrind 1 field; 2 perls at t/porting/bench.t line 512
#      got 'Error: can\'t open \'t/tmp_ANGTM_C\' for reading: No such file or directory\n'
# expected /(?^:\AKey\:\
#  *Ir +Instruction +read\
#  *Dr +Data +read\
#  *Dw +Data +write\
#  *COND +conditional +branches\
#  *IND +indirect +branches\
#  *_m +branch +predict +miss\
#  *_m1 +level +1 +cache +miss\
#  *_mm +last +cache +\(e\.g\. +L3\) +miss\
#  *\- +indeterminate +percentage +\(e\.g\. +1\/0\)\
# \
# The +numbers +represent +relative +counts +per +loop +iteration\, +compared +to\
# p0 +at +100\.0\%\.\
# Higher +is +better\: +for +example\, +using +half +as +many +instructions +gives +200\%\,\
# while +using +twice +as +many +gives +50\%\.\
# \
# Results +for +field +Ir\
# \
#  *p0 +p1\
#  *-+ +-+\
# call\:\:sub\:\:empty +(\s*-?\d+\.\d\d|\s*-) +(\s*-?\d+\.\d\d|\s*-)\
# \z)/
ok 74 - --bisect: exit result: should not match
ok 75 - --bisect: got expected output
ok 76 - 2 reads; overlapping test sets
# running cachegrind on 1 perl; may be slow...
not ok 77 - 1 read; 1 generate
# Failed test 77 - 1 read; 1 generate at t/porting/bench.t line 537
#      got 'Error: can\'t parse \'Dw_mm\' field from cachegrind output:\n==708467== Cachegrind, a cache and branch-prediction profiler\n==708467== Copyright (C) 2002-2017, and GNU GPL\'d, by Nicholas Nethercote et al.\n==708467== Using Valgrind-3.21.0 and LibVEX; rerun with -h for copyright info\n==708467== Command: /home/leont/Code/perl/perl - 10\n==708467== \n==708467== \n==708467== I refs:        1,342,958\n==708467== \n==708467== Branches:        274,720  (267,446 cond + 7,274 ind)\n==708467== Mispredicts:      17,687  ( 15,586 cond + 2,101 ind)\n==708467== Mispred rate:        6.4% (    5.8%     +  28.9%   )\n'
# expected /(?^:\AKey\:\
#  *Ir +Instruction +read\
#  *Dr +Data +read\
#  *Dw +Data +write\
#  *COND +conditional +branches\
#  *IND +indirect +branches\
#  *_m +branch +predict +miss\
#  *_m1 +level +1 +cache +miss\
#  *_mm +last +cache +\(e\.g\. +L3\) +miss\
#  *\- +indeterminate +percentage +\(e\.g\. +1\/0\)\
# \
# The +numbers +represent +relative +counts +per +loop +iteration\, +compared +to\
# p0 +at +100\.0\%\.\
# Higher +is +better\: +for +example\, +using +half +as +many +instructions +gives +200\%\,\
# while +using +twice +as +many +gives +50\%\.\
# \
# call\:\:sub\:\:empty\
# function +call +with +no +args +or +body\
# \
#  *p0 +p1\
#  *-+ +-+\
#  *Ir +100\.00 +(\s*-?\d+\.\d\d|\s*-)\
#  *Dr +100\.00 +(\s*-?\d+\.\d\d|\s*-)\
#  *Dw +100\.00 +(\s*-?\d+\.\d\d|\s*-)\
#  *COND +100\.00 +(\s*-?\d+\.\d\d|\s*-)\
#  *IND +100\.00 +(\s*-?\d+\.\d\d|\s*-)\
# \
# COND_m +100\.00 +(\s*-?\d+\.\d\d|\s*-)\
#  *IND_m +100\.00 +(\s*-?\d+\.\d\d|\s*-)\
# \
#  *Ir_m1 +100\.00 +(\s*-?\d+\.\d\d|\s*-)\
#  *Dr_m1 +100\.00 +(\s*-?\d+\.\d\d|\s*-)\
#  *Dw_m1 +100\.00 +(\s*-?\d+\.\d\d|\s*-)\
# \
#  *Ir_mm +100\.00 +(\s*-?\d+\.\d\d|\s*-)\
#  *Dr_mm +100\.00 +(\s*-?\d+\.\d\d|\s*-)\
#  *Dw_mm +100\.00 +(\s*-?\d+\.\d\d|\s*-)\
# \z)/
# running cachegrind on 2 perls; may be slow...
not ok 78 - 4 perls with autolabel and args and env
# Failed test 78 - 4 perls with autolabel and args and env at t/porting/bench.t line 563
#      got 'Error: can\'t parse \'Dr_m1\' field from cachegrind output:\n==708508== Cachegrind, a cache and branch-prediction profiler\n==708508== Copyright (C) 2002-2017, and GNU GPL\'d, by Nicholas Nethercote et al.\n==708508== Using Valgrind-3.21.0 and LibVEX; rerun with -h for copyright info\n==708508== Command: /home/leont/Code/perl/perl -Ilib -Ifoo/bar -Mstrict - 10\n==708508== \n==708508== \n==708508== I refs:        2,071,781\n==708508== \n==708508== Branches:        408,597  (395,434 cond + 13,163 ind)\n==708508== Mispredicts:      37,293  ( 32,388 cond +  4,905 ind)\n==708508== Mispred rate:        9.1% (    8.2%     +   37.3%   )\n'
# expected /(?^:\AKey\:\
#  *Ir +Instruction +read\
#  *Dr +Data +read\
#  *Dw +Data +write\
#  *COND +conditional +branches\
#  *IND +indirect +branches\
#  *_m +branch +predict +miss\
#  *_m1 +level +1 +cache +miss\
#  *_mm +last +cache +\(e\.g\. +L3\) +miss\
#  *\- +indeterminate +percentage +\(e\.g\. +1\/0\)\
# \
# The +numbers +represent +relative +counts +per +loop +iteration\, +compared +to\
# p0 +at +100\.0\%\.\
# Higher +is +better\: +for +example\, +using +half +as +many +instructions +gives +200\%\,\
# while +using +twice +as +many +gives +50\%\.\
# \
# call\:\:sub\:\:empty\
# function +call +with +no +args +or +body\
# \
#  *p0 +p1 +p2 +p3\
#  *-+ +-+ +-+ +-+\
#  *Ir +100\.00 +(\s*-?\d+\.\d\d|\s*-) +(\s*-?\d+\.\d\d|\s*-) +(\s*-?\d+\.\d\d|\s*-)\
#  *Dr +100\.00 +(\s*-?\d+\.\d\d|\s*-) +(\s*-?\d+\.\d\d|\s*-) +(\s*-?\d+\.\d\d|\s*-)\
#  *Dw +100\.00 +(\s*-?\d+\.\d\d|\s*-) +(\s*-?\d+\.\d\d|\s*-) +(\s*-?\d+\.\d\d|\s*-)\
#  *COND +100\.00 +(\s*-?\d+\.\d\d|\s*-) +(\s*-?\d+\.\d\d|\s*-) +(\s*-?\d+\.\d\d|\s*-)\
#  *IND +100\.00 +(\s*-?\d+\.\d\d|\s*-) +(\s*-?\d+\.\d\d|\s*-) +(\s*-?\d+\.\d\d|\s*-)\
# \
# COND_m +100\.00 +(\s*-?\d+\.\d\d|\s*-) +(\s*-?\d+\.\d\d|\s*-) +(\s*-?\d+\.\d\d|\s*-)\
#  *IND_m +100\.00 +(\s*-?\d+\.\d\d|\s*-) +(\s*-?\d+\.\d\d|\s*-) +(\s*-?\d+\.\d\d|\s*-)\
# \
#  *Ir_m1 +100\.00 +(\s*-?\d+\.\d\d|\s*-) +(\s*-?\d+\.\d\d|\s*-) +(\s*-?\d+\.\d\d|\s*-)\
#  *Dr_m1 +100\.00 +(\s*-?\d+\.\d\d|\s*-) +(\s*-?\d+\.\d\d|\s*-) +(\s*-?\d+\.\d\d|\s*-)\
#  *Dw_m1 +100\.00 +(\s*-?\d+\.\d\d|\s*-) +(\s*-?\d+\.\d\d|\s*-) +(\s*-?\d+\.\d\d|\s*-)\
# \
#  *Ir_mm +100\.00 +(\s*-?\d+\.\d\d|\s*-) +(\s*-?\d+\.\d\d|\s*-) +(\s*-?\d+\.\d\d|\s*-)\
#  *Dr_mm +100\.00 +(\s*-?\d+\.\d\d|\s*-) +(\s*-?\d+\.\d\d|\s*-) +(\s*-?\d+\.\d\d|\s*-)\
#  *Dw_mm +100\.00 +(\s*-?\d+\.\d\d|\s*-) +(\s*-?\d+\.\d\d|\s*-) +(\s*-?\d+\.\d\d|\s*-)\
# \z)/
1..78

jkeenan commented 1 year ago

On 5/19/23 11:54, Leon Timmermans wrote:

I can reproduce the issue on arch with gcc 13 (but I don't know if it failed previously)

Could you try with gcc-13 and g++-13 with the 5.37.11 release (or tag v5.37.11)?

I doubt any smoke tester even attempted version 13 until very recently.

Leont commented 1 year ago

Could you try with gcc-13 and g++-13 with the 5.37.11 release (or tag v5.37.11)?

Also fails. As does 5.36.

I doubt any smoke tester even attempted version 13 until very recently.

The test is skipped if one doesn't have valgrind installed or if it isn't a git checkout, I bet that combination is uncommon.

jkeenan commented 1 year ago

I guess this problem begs the question: At what point in the release cycle of major C-compilers to we begin assuring that the Perl core distribution builds and tests properly with that compiler version?

I note that GNU has had two major update releases within the past month.

GCC 12.3 released [2023-05-08] GCC 13.1 released [2023-04-26]

The release notes for GCC13 state: "The GCC developers are pleased to announce the release of GCC 13.1. This release is a major release, containing new features (as well as many other improvements) relative to GCC 12.x."

Moreover, in my experience C-compiler upgrades tend not to break the core distribution test suite. They're more likely to generate new build-time warnings, for which we create issues, label them as such, address them and move on. Here, however, it appears that we've been bitten by a very recent release of a major C-compiler.

There's obviously a policy discussion immanent here, and policy discussions are supposed to take place on the P5P list. But what do we do to clear away an obstacle to 5.38.0's release?

@rjbs @book @leonerd : Is this a release blocker?

iabyn commented 1 year ago

On Fri, May 19, 2023 at 01:41:14PM -0700, James E Keenan wrote:

@rjbs @book @leonerd : Is this a release blocker?

t/porting/bench.t just does some basic sanity checks that the Porting/bench.pl benchmarking script works. Since that tool is currently only targeted at core hackers and isn't installed anywhere, I don't think it matters if that test file, or even bench.pl itself doesn't work on a particular very bleading edge platform.

So not a blocker.

I have no opinion on whether any other problems with GCC-13 (are there any?) or whatever should be a blocker.

-- My get-up-and-go just got up and went.

Leont commented 1 year ago

I wouldn't be surprised if the real culprit was a valgrind upgrade instead of a gcc update. Given that it's failing to parse cachegrind output.

This issue may need someone who actually understands valgrind.

Leont commented 1 year ago

Is this a release blocker?

It's only really useful for core hackers, but I can imagine the test failing for distributors. It might already help a lot if we extend that -d "./.git" to -d "./.git" && -e '.mailmap' (and we may need a better solution for this once blead is open again)

iabyn commented 1 year ago

On Fri, May 19, 2023 at 02:09:44PM -0700, Leon Timmermans wrote:

Is this a release blocker?

It's only really useful for core hackers, but I can imagine the test failing for distributors. It might already help a lot if we extend that -d "./.git" to -d "./.git" && -e '.mailmap' (and we may need a better solution for this once blead is open again)

Perhaps the whole test file should skip if its not a devel (odd-numbered) perl version?

-- Spock (or Data) is fired from his high-ranking position for not being able to understand the most basic nuances of about one in three sentences that anyone says to him. -- Things That Never Happen in "Star Trek" #19

Leont commented 1 year ago

Perhaps the whole test file should skip if its not a devel (odd-numbered)

Probably. Though having an explicit opt-out may also be useful as I currently can't successfully test perl until I uninstall valgrind.

Perl / perl5

t/porting/bench.t: Failures on Fedora Linux 38 / gcc and g++ builds #21105