flang-compiler / flang

Flang is a Fortran language front-end designed for integration with LLVM.
Other
799 stars 134 forks source link

failed to run SPEC CPU2017 521.wrf_r with -march=native or -march=skylake. #1107

Open edisonchan opened 3 years ago

edisonchan commented 3 years ago

My system: Intel Hades Canyon (CPU: Intel KabyLake-G 8809G ) with 16GB memory.

I built the flang according this: https://github.com/flang-compiler/flang/wiki/Building-Flang.

It's strange, the flang bin file bulit does not show the output like in the flang page with --help:

OVERVIEW: clang LLVM compiler

USAGE: clang-12 [options] file...

OPTIONS:
  -###                   Print (but do not run) the commands to run for this compilation
  -c                     Only run preprocess, compile, and assemble steps
  -D <macro>=<value>     Define <macro> to <value> (or 1 if <value> omitted)
  -E                     Only run the preprocessor
  -fcolor-diagnostics    Enable colors in diagnostics
  -fno-color-diagnostics Disable colors in diagnostics
  -help                  Display available options
  -I <dir>               Add directory to the end of the list of include search paths
  -o <file>              Write output to <file>
  -U <macro>             Undefine macro <macro>
  --version              Print version information

now, when I use classic flang llvm 12x to build the SPEC CPU2017, -march=x86-64 & -mtune=core-avx2 is ok, but when I use -march=skylake or -march=native, 521.wrf would failed to run(I had add FPORTABILITY = -Mbyteswapio):

runs started at         1625717512, 870017000, Thu Jul  8 12:11:52 2021
run 1 started at        1625717512, 870036000, Thu Jul  8 12:11:52 2021
child started: 0,       1625717512, 870042000, pid=29023, '../run_base_refrate_                                                                               llvm_12-m64.0000/wrf_r_base.llvm_12-m64 > rsl.out.0000 2>> wrf.err'
child finished: 0,      1625717513, 062080000, sec=0, nsec=192038000, pid=29023                                                                               , rc=256
run 1 finished at:      1625717513, 062102000, Thu Jul  8 12:11:53 2021
run 1 elapsed time:              0, 192066000, 0.192066000
runs finished at        1625717513, 062118000, Thu Jul  8 12:11:53 2021
runs elapsed time:               0, 192101000, 0.192101000
error: a total of 1 children finished with errors
specinvoke exit: rc=0

****************************************
****************************************
Contents of wrf.err
****************************************
Error during math dispatch processing...
__pgmath_abort:Math dispatch table is either misconfigured or corrupted.
****************************************
Invalid run; unable to continue.
If you wish to ignore errors please use '-I' or ignore_errors
The log for this run is in /home/edison/cpu2017/result/CPU2017.004.log
The debug log for this run is in /home/edison/cpu2017/result/CPU2017.004.log.de                                                                               bug
*
* Temporary files were NOT deleted; keeping temporaries such as
* /home/edison/cpu2017/result/CPU2017.004.log.debug and
* /home/edison/cpu2017/tmp/CPU2017.004
* (These may be large!)
*
runcpu finished at 2021-07-08 12:11:53; 440 total seconds elapsed
d-parks commented 3 years ago

Can you rerun setting environment variables MTH_I_STATS=7 and MTH_I_DEBUG=1 and capture stderr?

edisonchan commented 3 years ago

I had try to run(after set MTH_I_STATS=7 and MTH_I_DEBUG=1) :

'''runcpu -c llvm --noreportable -n 1 521 2> std_err.txt'''

but the file std_err.txt is empty.

The log file output by CPU2017. cpu2017_log.zip

d-parks commented 3 years ago

In your log.debug file:

1625802351.74: Invoked as: /home/edison/cpu2017/bin/specinvoke -d /home/edison/cpu2017/benchspec/CPU/521.wrf_r/run/run_base_refrate_llvm_12-m64.0000 -f speccmds.cmd -q -e speccmds.err -o speccmds.stdout 1625802351.74: timer ticks over every 1000 ns 1625802351.76: Environment in effect for these runs:

I don't see any MTH_I_STATS or MTH_I_DEBUG.

edisonchan commented 3 years ago

My bad, I forgot use "export" to set them.

here is the output: 521.zip

d-parks commented 3 years ago

The error being reported is because the application is calling and avx512 entry point:

__pgmath_abort:Math dispatch table is either misconfigured or corrupted. log/dv8/precise Entry point not defined for CPU target=avx2.

log/dv8 implies zmm registers.

The math library detected that you're running on an AVX2 processor (not AVX512):

MTH_I_DEBUG=1 math_target: avx2(5) __mth_fast: fast(0) mth_relaxed: relaxed(1) mth_precise: precise(2) __mth_sleef: sleef(3) math_dispatch: built on Jul 7 2021 22:09:13 with clang-12.0.0

edisonchan commented 3 years ago

I have check the bin file built with flang, there is not "zmm" output from "objdump -d wrf_r_base.llvm_12-m64 |grep zmm".

d-parks commented 3 years ago

Can you look for "pow_8" in your objdump?

edisonchan commented 3 years ago
objdump -d wrf_r_base.llvm_12-m64 |grep pow_8
0000000000405890 <__ps_pow_8@plt>:
  405890:       ff 25 b2 1b 79 01       jmp    *0x1791bb2(%rip)        # 1b97448 <__ps_pow_8>
  4945a3:       e8 e8 12 f7 ff          call   405890 <__ps_pow_8@plt>
  4945bd:       e8 ce 12 f7 ff          call   405890 <__ps_pow_8@plt>
  4945d7:       e8 b4 12 f7 ff          call   405890 <__ps_pow_8@plt>
  4945f4:       e8 97 12 f7 ff          call   405890 <__ps_pow_8@plt>
  95215e:       e8 2d 37 ab ff          call   405890 <__ps_pow_8@plt>

I have used flang to buiild CPU 2017 a few months ago, its version is 7.0.1, In my memory that flang does not need "classic-flang-llvm-project" to build.

d-parks commented 3 years ago

You'll have to figure out where the call to pd_pow_8 is coming from. I'd suggest setting a break point with gdb at that symbol and when it triggers then show the backtrace of the stack. Is your executable static or dynamic? If it is not static, I'd suggest relinking it as static and again verifying that pd_pow_8 is not being called.