Compiling with -fopenmp-targets=nvptx64-nvida-cuda generates fatal error: 'gnu/stubs-32.h' file not found

[Quuxplusone]

Bugzilla Link	PR33043
Status	NEW
Importance	P normal
Reported by	Sergio Pino (sergiop@udel.edu)
Reported on	2017-05-15 10:57:15 -0700
Last modified on	2017-12-01 07:22:33 -0800
Version	unspecified
Hardware	PC Linux
CC	a.bataev@hotmail.com, acjacob@us.ibm.com, carlo.bertolli@gmail.com, Gheorghe-Teod.Bercea@ibm.com, hahnjo@hahnjo.de, hfinkel@anl.gov, llvm-bugs@lists.llvm.org
Fixed by commit(s)
Attachments	error.txt (4591 bytes, text/plain)
Blocks
Blocked by
See also

Created attachment 18444
Verbose error

Dear All,

I'm getting the following error when compiling with triple nvptx64-nvida-cuda.
I'm trying to compile manually one of the test cases in libomptarget
(openmp/libomptarget/test/offloading/offloading_success.c).

clang -fopenmp -fopenmp-targets=nvptx64-nvida-cuda offloading_success.c

But I'm getting the following compilation error during the compilation of the
ptx code. For some reason the compiler is trying to use the gnu/stubs-32.h
header (it is using the macro __WORDSIZE == 32 rather than 64). I'm attaching
the verbose output of the compiler.

---
In file included from offloading_success.c:6:
In file included from /usr/include/stdio.h:28:
In file included from /usr/include/features.h:385:
/usr/include/gnu/stubs.h:7:11: fatal error: 'gnu/stubs-32.h' file not found
# include <gnu/stubs-32.h>
          ^~~~~~~~~~~~~~~~
1 error generated.
---

I tested this in two 64 bit systems, thus I don't understand why It wants to
use a 32 bit header:
- CentOS release 6.7 x86_64 + Intel(R) Xeon(R) CPU E5-2660 v3 @ 2.60GHz +
NVIDIA Tesla K80 GPU + gcc 6.2.0
- CentOS release 7 + IBM POWER8 + NVIDIA Tesla P100 GPU + gcc/6.2.1-20161129.

------Setup
I compiled llvm/clang/OpenMP from the master branch. LLVM/clang compiles
correctly and OpenMP compiles with LIBOMPTARGET: Building CUDA offloading
plugin.

Clang commit: 84f57d869ba15eb9b12abc57014d530647a00e01
LLVM commit: 34ad4ae9219da7d2e14b14d0eefd10515d5ac526
OpenMP commit: 1eb73292d2baee506f5dcd1584fca768c849313b

I used cmake + ninja.

1. LLVM + clang:
cmake /home/1678/Code/ornl/llvm -G Ninja -DCMAKE_BUILD_TYPE=Release -
DCMAKE_C_COMPILER=gcc -DCMAKE_CXX_COMPILER=g++ -
DCMAKE_INSTALL_PREFIX=/home/1678/Code/ornl/llvm-install

2. OpenMP as an out-of-the-tree project:
cmake /home/1678/Code/ornl/llvm-out-of-tree/openmp -G Ninja -
DCMAKE_C_COMPILER=gcc -DCMAKE_CXX_COMPILER=g++ -DCMAKE_BUILD_TYPE=Release -
DCMAKE_INSTALL_PREFIX=/home/1678/Code/ornl/llvm-oot-install -
DLIBOMP_LLVM_LIT_EXECUTABLE=/home/1678/Code/ornl/llvm-build/bin/llvm-lit -
DLIBOMPTARGET_FILECHECK_EXECUTABLE=/home/1678/Code/ornl/llvm-
build/bin/FileCheck -
DLIBOMPTARGET_LLVM_LIT_EXECUTABLE=/home/1678/Code/ornl/llvm-build/bin/llvm-lit -
DLIBOOMPTARGET_OPENMP_HEADER_FOLDER=/home/1678/Code/ornl/llvm-oot-
install/include -DLIBOMPTARGET_OPENMP_HOST_RTL_FOLDER=/home/1678/Code/ornl/llvm-
oot-install/lib

Best,

[Quuxplusone]

Attached error.txt (4591 bytes, text/plain): Verbose error

[Quuxplusone]

As I recall, the problem is that, when compiling for the target (nvptx64),
we're not defining the host-associated preprocessor macros (e.g. __X86_64__ or
__powerpc64__). As a result, when we end up in code like this:

$ cat /usr/include/bits/wordsize.h
/* Determine the wordsize from the preprocessor defines.  */

#if defined __x86_64__ && !defined __ILP32__
# define __WORDSIZE 64
#else
# define __WORDSIZE 32
#endif

we end up setting __WORDSIZE == 32 and, thus, trying to include 32-bit headers
(which are often not installed).

I suspect that the real answer here is to make sure that we pick up the CUDA
stdio.h, etc. when compiling for a nvptx target target (or something along
those lines). Alternatively, we could also define the host architecture macros
when compiling for the target.

[Quuxplusone]

Hi Hal,

To my understanding the error will go away once the outstanding OpenMP offload driver patches land. There are currently 10 accepted patches and 3 that have yet to be accepted.

Thanks,

--Doru

[Quuxplusone]

Some of the patches have landed. Can you please retry if you still see the error? Also if you are compiling for 32bit host, the correct triple is nvptx-nvidia-cuda, not nvptx64-nvidia-cuda (see https://llvm.org/docs/NVPTXUsage.html)