pLiner is a framework that helps programmers identify locations in the source of numerical code that are highly affected by floating-point compiler optimizations.
Compiler optimizations can alter significantly the numerical results of scientific computing applications. When numerical results differ significantly between compilers, optimization levels, and floating-point hardware, these numerical inconsistencies can impact programming productivity. pLiner is a framework that helps programmers identify locations in the source code that are highly affected by compiler optimizations. pLiner uses a novel approach to identify such code locations by enhancing the floating-point precision of variables and expressions. Using a guided search to locate the most significant code regions, pLiner can report to users such locations at different granularities, file, function, and line of code. pLiner is implemented as a clang tool. Currently pLiner only supports C/C++.
There are 4 options to start using pLiner:
docker pull ucdavisplse/llnl_pliner:latest
docker run -it ucdavisplse/llnl_pliner:latest /bin/bash
git clone https://github.com/llnl/pLiner.git
cd pLiner
docker build -t <tag-name> .
docker run -it <tag-name> /bin/bash
json.hpp
from https://github.com/nlohmann/json/blob/develop/single_include/nlohmann/json.hpp (version 3.5.0) and place it in the directory pLiner/clang-tool
before using pLiner.Clone pLiner and build it:
git clone https://github.com/llnl/pLiner.git
cd pLiner/clang-tool
mkdir build; cd build
cmake ..
make
Install pLiner
make install
Or, export path to pLiner (this command may differ depending on shell):
export PATH=$PATH-TO-pLiner/clang-tool/build:$PATH
Building clang/LLVM 9.0.1:
git clone https://github.com/llvm/llvm-project.git clang-llvm
git checkout llvmorg-9.0.1
cd ~/clang-llvm
mkdir build && cd build
cmake -G Ninja ../llvm -DLLVM_ENABLE_PROJECTS="clang;clang-tools-extra" -DLLVM_BUILD_TESTS=ON
ninja
ninja check # Test LLVM only.
ninja clang-test # Test Clang only.
ninja install
Note: Refer to https://clang.llvm.org/docs/LibASTMatchersTutorial.html in case you need instructions for installing cmake
and/or ninja
.
Clone pLiner in the clang-tools-extra directory and build it:
cd ../clang-tools-extra
git clone https://github.com/llnl/pLiner.git
echo "add_subdirectory(pLiner/clang-tool)" >> CMakeLists.txt
cp pLiner/clang-tool/CMakeLists.txt-insource pLiner/CMakeLists.txt
cd ../build
ninja
Export path to pLiner (this command may differ depending on shell):
export PATH=$PATH-TO-CLANG-LLVM/build/bin:$PATH
cd pLiner/tests
./test.sh
If pliner works as expected, all unit tests would pass.
We use a simple C program vtest.c
to show how to use pLiner. This program was generated by a floating-point random program generator, Varity, and it produces inconsistent results when compiled with gcc -O3 -ffast-math
compared to gcc -O0
. pLiner isolated a line of the code in the source (Line 25) as the origin of the compiler-induced inconsistency. pLiner also provided a transformed version of the program vtest_trans.c
, in which the isolated line of the code has been transformed to higher precision, and the transformed program produces consistent results between gcc -O3 -ffast-math
and gcc -O0
.
Change to the example
directory:
cd pLiner/example
Compile the original vtest.c
program with both gcc -O3 -ffast-math
and gcc -O0
, and compare the results:
make
./cmp.sh vtest
vtest_O0
corresponds to the executable generated by compiling vtest.c
with gcc -O0
, and vtest_O3
corresponds to the executable generated by compiling vtest.c
with gcc -O3 -ffast-math
.
The expected output when comparing the results should be:
./vtest_O0 +1.8768E-306 5 -1.3896E-307 +1.2460E-307 -1.4722E306 -0.0 -1.7470E-322 +1.7072E-307 -1.9009E-307 +1.6022E137 +1.3969E306 +1.8813E34 -1.9422E-99 -1.2666E305 +0.0 -1.2316E-314 +1.5805E-323 +1.7072E208 +1.9220E-307 +1.6811E-306
1.7071999999999999e+208
./vtest_O3 +1.8768E-306 5 -1.3896E-307 +1.2460E-307 -1.4722E306 -0.0 -1.7470E-322 +1.7072E-307 -1.9009E-307 +1.6022E137 +1.3969E306 +1.8813E34 -1.9422E-99 -1.2666E305 +0.0 -1.2316E-314 +1.5805E-323 +1.7072E208 +1.9220E-307 +1.6811E-306
-1.8508999968058596e-316
Note that the difference between vtest_O0
and vtest_O3
is very large. We will use pLiner to diagnose the root cause of the numerical difference.
Run pLiner:
python ../scripts/search.py vtest.c "--"
The first argument vtest.c
is the input program; the second argument "--"
indicates that to compile the input program there are no header files/librares to specify in the compilation command. Additionally, if there are any such compilation options such as "-I $PATH-TO-HEADERS", specify them following "--" in the second argument, e.g., "-- -I $PATH-TO-HEADERS". Use --help
to check for the details of the arguments.
Following is the output.
...
The following areas are transformed to high precision:
compute :
20 -> 22
23 -> 23
failed
The following areas are transformed to high precision:
compute :
25 -> 25
26 -> 26
28 -> 28
success
The following areas are transformed to high precision:
compute :
25 -> 25
success
Search for lines:
The following areas are transformed to high precision:
compute :
25 -> 25
success
Bug area:
compute :
line 25
pLiner found the root cause of the inconsistency (function compute
, line 25):
compute :
line 25
pLiner generated a transformed program vtest_trans.c
.
ls *.c
vtest.c vtest_trans.c
vtest_trans.c
program with both gcc -O3 -ffast-math
and gcc -O0
, and compare the results:
make
./cmp.sh vtest_trans
The expected output when comparing the results should be:
./vtest_trans_O0 +1.8768E-306 5 -1.3896E-307 +1.2460E-307 -1.4722E306 -0.0 -1.7470E-322 +1.7072E-307 -1.9009E-307 +1.6022E137 +1.3969E306 +1.8813E34 -1.9422E-99 -1.2666E305 +0.0 -1.2316E-314 +1.5805E-323 +1.7072E208 +1.9220E-307 +1.6811E-306
1.7071999999999999e+208
./vtest_trans_O3 +1.8768E-306 5 -1.3896E-307 +1.2460E-307 -1.4722E306 -0.0 -1.7470E-322 +1.7072E-307 -1.9009E-307 +1.6022E137 +1.3969E306 +1.8813E34 -1.9422E-99 -1.2666E305 +0.0 -1.2316E-314 +1.5805E-323 +1.7072E208 +1.9220E-307 +1.6811E-306
1.7071999999999999e+208
Both vtest_trans_O0
and vtest_trans_O3
produce 1.7071999999999999e+208
. The results are consistent with vtest_O0
.
Navigate to pLiner/benchmarks/Varity/Varity-intel
and run the below command to run pLiner on all 50 Varity programs.
python3 run_varity_programs.py
Run the below command to compare the results from pLiner with the reference Varity results and to get a summary for all 50 programs.
python3 compare_varity_results.py
The expected results summary for Varity programs
Results summary
Total Number of programs matched: 50
Total Number of programs which did not match: 0
Total Number of programs for which pLiner was unable to remove inconsistency: 4
List of programs for which pLiner was unable to remove inconsistency
[12, 32, 46, 47]
Navigate to pLiner/benchmarks/NPB/npb-debug
and run the below command to run pLiner for three NPB programs CG.B, SP.A and SP.B.
python3 run_npb_programs.py
Run the below command to compare the results from pLiner with the reference NPB results.
python3 compare_npb_programs.py
The expected results summary for NPB programs
CG.B results:
Results match
Isolated region granularity: functions
Function Name: sparse
SP.A results:
Results match
File Name: y_solve.c
Isolated region granularity: lines
Function Name: y_solve
Lines isolated: [[68, 68]]
SP.B results:
Results match
File Name: exact_solution.c
Isolated region granularity: lines
Function Name: exact_solution
Lines isolated: [[44, 44]]
For SP program with input class A, the results are different compared to what is mentioned in the pLiner paper. Please refer to results inside SP.A folder for the updated results.
You can follow the instructions as shown in the example to run pLiner for your own programs. Specifically,
Specify the compiler, compilation options that induce inconsistent results, and an error threshold in run.sh
In the example above, those are specified in run.sh
as
CC="/usr/bin/gcc"
CFLAGS=" -O0 -g -std=c99"
CFLAGS_trouble=" -O3 -ffast-math -g -std=c99"
THRESHOLD=8
CC
specifies the compiler; CFLAGS
specifies the compilation options that are used to produce ground-truth results and CFLAGS_trouble
specifies the compilation options that induce inconsistent results; lastly, THRESHOLD
specifies the number of digits that are required to be same with the ground truth for consistency check.
Specify your program file in the first argument, and the compilation options needed to compile the program in the second argument such as
python pLiner/scripts/search.py vtest.c "--"
in the example.
pLiner is distributed under the terms of the Apache-2.0 with LLVM-exception license. All new contributions must be made under this license.
See LICENSE and NOTICE for details.
LLNL-CODE-812209