search

lashgar / ipmacc

IPMACC is a framework for translating OpenACC for C API to CUDA, OpenCL, and Intel ISPC.
13 stars 4 forks source link

compilation errors #3

Closed babouFomb closed 5 years ago

babouFomb commented 7 years ago

I try to install ipmacc on Tegra X1, but when I run the

./compile-all

I get following errors:

ubuntu@tegra-ubuntu:~/ipmacc$ ./compile-all ~ compiling uncrustify ...... done ~ compiling pycparser .bison oaccparser.y oaccparser.y: warning: 63 shift/reduce conflicts [-Wconflicts-sr] oaccparser.y: warning: 49 reduce/reduce conflicts [-Wconflicts-rr] oaccparser.y:76.10-94: warning: rule useless in parser due to conflicts [-Wother] | error { printf("unexpected content within cache region\n"); return -1; } ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ gcc oaccparser.tab.c -o oaccparser oaccparser.y:7:1: warning: return type defaults to ‘int’ [-Wimplicit-int] yyerror (s) / Called by yyparse on error / ^ oaccparser.tab.c: In function ‘yyparse’: oaccparser.tab.c:1217:16: warning: implicit declaration of function ‘yylex’ [-Wimplicit-function-declaration] yychar = yylex (); ^ oaccparser.y: At top level: oaccparser.y:98:1: warning: return type defaults to ‘int’ [-Wimplicit-int] yylex () ^ oaccparser.y:115:1: warning: return type defaults to ‘int’ [-Wimplicit-int] main () ^ .. done ~ compiling srcML parser .make: PureCommentLexer.g: Command not found make: [PureCommentLexerTokenTypes.txt] Error 127 . done ~ compiling OpenACC API .openacc.c: In function ‘acc_malloc’: openacc.c:704:20: warning: format ‘%llu’ expects argument of type ‘long long unsigned int’, but argument 2 has type ‘size_t {aka long unsigned int}’ [-Wformat=] printf("failed to allocate memory %16llu bytes on CUDA device by acc_malloc: error-code (%d)\n", bytes, err); ^ openacc.c:706:54: warning: format ‘%llu’ expects argument of type ‘long long unsigned int’, but argument 2 has type ‘size_t {aka long unsigned int}’ [-Wformat=] }else if(getenv("IPMACCLIB_VERBOSE")) printf("CUDA: %16llu bytes [allocated] on device by acc_malloc (ptr: %p)\n",bytes,ptr); ^ openacc.c: In function ‘acc_list_devices_spec’: openacc.c:769:20: warning: format ‘%u’ expects argument of type ‘unsigned int’, but argument 2 has type ‘size_t {aka long unsigned int}’ [-Wformat=] printf("\tTotal global memory: %u\n", devProp.totalGlobalMem); ^ openacc.c:770:20: warning: format ‘%u’ expects argument of type ‘unsigned int’, but argument 2 has type ‘size_t {aka long unsigned int}’ [-Wformat=] printf("\tTotal shared memory per block: %u\n", devProp.sharedMemPerBlock); ^ openacc.c:773:20: warning: format ‘%u’ expects argument of type ‘unsigned int’, but argument 2 has type ‘size_t {aka long unsigned int}’ [-Wformat=] printf("\tMaximum memory pitch: %u\n", devProp.memPitch); ^ openacc.c:782:20: warning: format ‘%u’ expects argument of type ‘unsigned int’, but argument 2 has type ‘size_t {aka long unsigned int}’ [-Wformat=] printf("\tTotal constant memory: %u\n", devProp.totalConstMem); ^ openacc.c:783:20: warning: format ‘%u’ expects argument of type ‘unsigned int’, but argument 2 has type ‘size_t {aka long unsigned int}’ [-Wformat=] printf("\tTexture alignment: %u\n", devProp.textureAlignment); ^ openacc.c: In function ‘acc_compress_create’: openacc.c:900:20: warning: format ‘%llu’ expects argument of type ‘long long unsigned int’, but argument 2 has type ‘size_t {aka long unsigned int}’ [-Wformat=] printf("failed to allocate memory %16llu bytes on CUDA device: %d\n", bytes, err); ^ openacc.c:901:54: warning: format ‘%llu’ expects argument of type ‘long long unsigned int’, but argument 2 has type ‘size_t {aka long unsigned int}’ [-Wformat=] }else if(getenv("IPMACCLIB_VERBOSE")) printf("CUDA: %16llu bytes [allocated] on device (ptr: %p)\n",bytes,devptr); ^ openacc.c: In function ‘acc_create’: openacc.c:975:20: warning: format ‘%llu’ expects argument of type ‘long long unsigned int’, but argument 2 has type ‘size_t {aka long unsigned int}’ [-Wformat=] printf("failed to allocate memory %16llu bytes on CUDA device: error-code (%d)\n", bytes, err); ^ openacc.c:977:54: warning: format ‘%llu’ expects argument of type ‘long long unsigned int’, but argument 2 has type ‘size_t {aka long unsigned int}’ [-Wformat=] }else if(getenv("IPMACCLIB_VERBOSE")) printf("CUDA: %16llu bytes [allocated] on device (ptr: %p)\n",bytes,devptr); ^ openacc.c: In function ‘acc_copyout_and_keep’: openacc.c:1065:49: warning: format ‘%llu’ expects argument of type ‘long long unsigned int’, but argument 2 has type ‘size_t {aka long unsigned int}’ [-Wformat=] if (getenv("IPMACCLIB_VERBOSE")) printf("CUDA: %16llu bytes [copyout] from device (ptr: %p)\n",bytes,devptr); ^ openacc.c: In function ‘acc_copyout_to_comp’: openacc.c:1103:49: warning: format ‘%llu’ expects argument of type ‘long long unsigned int’, but argument 2 has type ‘size_t {aka long unsigned int}’ [-Wformat=] if (getenv("IPMACCLIB_VERBOSE")) printf("CUDA: %16llu bytes [copyout] from device (ptr: %p)\n",bytes,devptr); ^ openacc.c: In function ‘acc_present_or_copyin’: openacc.c:1242:49: warning: format ‘%llu’ expects argument of type ‘long long unsigned int’, but argument 2 has type ‘size_t {aka long unsigned int}’ [-Wformat=] if (getenv("IPMACCLIB_VERBOSE")) printf("CUDA: %16llu bytes [copyin] to device (ptr: %p)\n",bytes,devptr); ^ openacc.c: In function ‘acc_copy_from_compressed’: openacc.c:1279:49: warning: format ‘%llu’ expects argument of type ‘long long unsigned int’, but argument 2 has type ‘size_t {aka long unsigned int}’ [-Wformat=] if (getenv("IPMACCLIB_VERBOSE")) printf("CUDA: %16llu bytes [copyin] to device (ptr: %p)\n",bytes,devptr); ^ openacc.c: In function ‘acc_memcpy_to_device’: openacc.c:1685:49: warning: format ‘%llu’ expects argument of type ‘long long unsigned int’, but argument 2 has type ‘size_t {aka long unsigned int}’ [-Wformat=] if (getenv("IPMACCLIB_VERBOSE")) printf("CUDA: %16llu bytes [copyin] to device (ptr: %p)\n",bytes,devptr); ^ openacc.c: In function ‘acc_memcpy_from_device’: openacc.c:1721:49: warning: format ‘%llu’ expects argument of type ‘long long unsigned int’, but argument 2 has type ‘size_t {aka long unsigned int}’ [-Wformat=] if (getenv("IPMACCLIB_VERBOSE")) printf("CUDA: %16llu bytes [copyout] from device (ptr: %p)\n",bytes,devptr); ^ openacc.c: In function ‘acc_profiler_dump’: openacc.c:2218:12: warning: implicit declaration of function ‘time’ [-Wimplicit-function-declaration] srand (time(NULL)); ^ /usr/bin/ld: /home/ubuntu/ipmacc/src/tasksys.o: undefined reference to symbol '__gxx_personality_v0@@CXXABI_1.3' //usr/lib/aarch64-linux-gnu/libstdc++.so.6: error adding symbols: DSO missing from command line collect2: error: ld returned 1 exit status make: [listdevices] Error 1 . done

I d'ont understand the problem, what DSO is missing? I installed all packages dependancies.

Any help is welcome. Thanks.

lashgar commented 7 years ago

Seems the error is in compiling listdevices and linking it to /home/ubuntu/ipmacc/src/tasksys.o. My guess is upon compiling listdevices, improper C++ linking flags are passed.

To solve this issue, firstly, check lib/ directory and make sure openacc.so library file is created. If it is there, in src/Makefile, replace:

listdevices: listdevices.c
    $(CC) listdevices.c -o listdevices -fPIC -g `ipmacc --cflags` `ipmacc --ldflags`

with

listdevices: listdevices.c
    $(CXX) listdevices.c -o listdevices -fPIC -g `ipmacc --cflags` `ipmacc --ldflags`

and then compile listdevices by typing make in ./src directory. This should generate listdevices in ./src/ and then runs it to list CUDA and OpenCL devices that ipmacc sees.

UPDATE: If you see the devices are listed, compilation is done successful and you may use ipmacc for compiling OpenACC codes.

Please let me know if it did not fix and I will dig in to this more to find the issue.

Thanks.

babouFomb commented 7 years ago

Thanks for your response.

First, I don't have the lib directory in my ipmacc root diretory. I have just: auxilaries codegen.py docs include INSTALL LICENSE preprocessor.py README.md setup_environment srcML TODO VERSION clean-all compile-all EXTENSIONS __init__.py ipmacc parser pycparser scanner.py src test-case uncrustify

I search with find command the openacc.so file, and I did'n found it. So I think it was not created. I replaced $CC$ by $CXX$ in src/Makefile for listdevices line and I run again make in src directory, I got the same error:

g++ /home/rokiatou/Documents/Thèse/other-openacc-implementation/ispc-v1.8.2knl-linux/examples/tasksys.cpp -O2 -c -o /home/rokiatou/Documents/Thèse/other-openacc-implementation/ipmacc-maste/src/tasksys.o Assembler messages: Fatal error: can't create /home/rokiatou/Documents/Thèse/other-openacc-implementation/ipmacc-maste/src/tasksys.o: No such file or directory make: *** [tasksys.o] Error 1

lashgar commented 7 years ago

Try two things:

1) Seems it cannot access the directory to create tasksys.o. Make sure IPMACCROOT is properly set (do echo $IPMACCROOT and check if this is the root of ipmacc) and /home/rokiatou/Documents/Thèse/other-openacc-implementation/ipmacc-maste/src/ exists. I also noticed that the root folder where IPMACC is stored is different in your first and second messages. In first message it is at ~/ipmacc/ and on the second it is at /home/rokiatou/Documents/Thèse/other-openacc-implementation/ipmacc-maste/. Make sure IPMACCROOT environment variable in the setup_environment file is set to point to the proper directory. or 2) Disable ISPC backend in setup_environment file by setting ISPCSUPPORT to 0 and see if compilation succeeds. Make sure to update environment (source /path/to/ipmacc/setup_environment) followed by ./clean-all and ./compile-all.

babouFomb commented 7 years ago

In fact I want to install ipmacc on a tegra X1 and an intel x86 with nvidia gpu.

For both, I replaced $CC$ by $CXX$ in src/Makefile for listdevices and disabled ISPC backend in setup_environnement file.

All compiled with many warnings but without errors.

I verified my installation on both platform with commands ipmacc --version and ipmacc --list-devices CUDA and these commands work fine.

But when, I tried to compile and run vectoradd.c sample in test-case/ directory, compilation failed on both platforms with different messages. I followed the same installation procedure on both platforms. The only difference is that on the x86, I enabled OpenCL backend.

On the Tegra X1, the output of ipmacc vectoradd.c -o vectoradd was:

Traceback (most recent call last): File "/home/ubuntu/ipmacc//scanner.py", line 1, in from lxml.builder import E ImportError: No module named lxml.builder

Before installation, I installed all packages dependencies as it is requested in INSTALL, and I verified with dpkg -l | grep libxml2 to be sure that xml package is installed. This is the output of dpkg:

ubuntu@tegra-ubuntu:~/ipmacc/test-case$ dpkg -l | grep libxml2 ii libxml2:arm64 2.9.3+dfsg1-1ubuntu0.2 arm64 GNOME XML library ii libxml2-dev:arm64 2.9.3+dfsg1-1ubuntu0.2 arm64 Development files for the GNOME XML library ii python3-lxml 3.5.0-1build1 arm64 pythonic binding for the libxml2 and libxslt libraries

On my PC x86, the output of ipmacc vectoradd.c -o vectoradd was:

rokiatou@rokiatou:~/Documents/Thèse/other-openacc-implementation/ipmacc/test-case$ ipmacc vectorAdd.c -o vectorAdd warning: storing the translated code in (target: ) unable to open src2srcml binary

I looked in the subdirectories bin and obj, on both platforms, these subdirectories are empty.

lashgar commented 7 years ago

From the output of dpkg on your system, it seems srcML and python-lxml are not installed. Follow these: 1) Make sure Python 2.x is installed on the system (python2 should be on the PATH) 2) Install python-lxml. python-lxml is not included in IPMACC package. It should be installed using apt-get or yum. (Notice python3-lxml is not the same as python-lxml. python3-lxml provides lxml in Python 3. But ipmacc uses Python 2 and lxml in Python 2 is provided by python-lxml package. See here: http://lxml.de/installation.html) 3) Make sure ./compile-all successfully extracts and compiles ./srcML/srcml.tar.gz. (If you see an error, please send the error so I'll be able to find the reason.)

babouFomb commented 7 years ago

On x86 platform, the following packages are installed:

bison (3.0.2); gcc/g++ 4.8; python-2.7 and python-3.4; libxml2 (2.9.1); libxml2-dev (2.9.1); antlr (2.7.7); libantlr-dev (2.7.7); libarchive-dev (3.1.2); libxslt1-dev (1.1.28); libboost-dev (1.54.0); python-libxml2 (2.9.1); python-lxml (3.3.3); python3-lxml (3.3.3)

PATH variable contain is:

/opt/pgi/linux86-64/16.10/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/home/rokiatou/forge/openmp4/llvm/install/bin

Python is installed and is on my PATH. With sudo dpkg -L python2.7 I see that it is located in /usr/bin, /usr/lib, /usr/share.

I have downloaded ipmacc in new directory. So the path is different from that in my previous post. But, I keep same configuration: disabled ISPC backend, enabled CUDA and OpenCL backends and replace $CC$ by $CXX$ in src/Makefile for listdevices. Plus a little modification in compile_all: replacing all make > /dev/null and make install > /dev/null by make > output.txt 2>&1 and make install > output.txt 2>&1 respectively, in order to save complete make output.

There some errors during the compilation of srcML. I attached a txt file there I put all output of compile_all.

I looked on srcML directory, there are: bin/ doc/ INSTALL obj/ src/ srcml.tar.gz wrapper/:

Thanks.

ipmacc-output.txt

lashgar commented 7 years ago

Thanks for attaching the log file. I looked at the log and I believe the source of the problem is already listed in the known issues in ./INSTALL file: srcML, the parser which is used in IPMACC, has a compatibility issue with latest version of libxml2. To resolve this issue, use libxml2-2.7.6 which can be downloaded from here: ftp://xmlsoft.org/libxml2/libxml2-2.7.6.tar.gz

babouFomb commented 7 years ago

Thanks for your response.

On x86, I still can not installlibxml2-2.7.6 from sources with make install or checkinstall, and this version is not provided by apt repositories (so I can't downgrade libxml2-2.9.1 to 2.7.6with apt-get).

On Tegra X1, there is the same problem due to others packages dependencies of libxml2. When I installed libxml2-2.7.6 with checkinstall, I can see whith dpkg -l grep libxml2:

ubuntu@tegra-ubuntu:~$ dpkg -l | grep libxml ii libxml-parser-perl 2.44-1build1 arm64 Perl module for parsing XML files ii libxml-twig-perl 1:3.48-1 all Perl module for processing huge XML documents in tree mode ii libxml-xpathengine-perl 0.13-1 all re-usable XPath engine for DOM-like trees iU libxml2 2.7.6-1 arm64 Package created with checkinstall 1.6.2 ii libxml2-dev:arm64 2.9.3+dfsg1-1ubuntu0.2 arm64 Development files for the GNOME XML library ii python-libxml2 2.9.3+dfsg1-1ubuntu0.2 arm64 Python bindings for the GNOME XML library ii python-lxml 3.5.0-1build1 arm64 pythonic binding for the libxml2 and libxslt libraries ii python3-lxml 3.5.0-1build1 arm64 pythonic binding for the libxml2 and libxslt libraries

libxml2 is installed but un-packed. So the compilation of srcML failed.

Tipically, it is a configuration problem, but for now, I have no solution. I will continue to look for other alternatives to install same version 2.7.6 of libxml, libxml-dev and python-lxml.

Thanks.

lashgar commented 7 years ago

What is the error you get during compiling libxml2-2.7.6?

Notice that make install and checkinstall might not be enough for installing libxml2-2.7.6 when an older version exists. Follow these:

wget ftp://xmlsoft.org/libxml2/libxml2-2.7.6.tar.gz
tar xvzf libxml2-2.7.6.tar.gz
cd libxml2-2.7.6/
./configure --prefix=/path/to/store/libxml2/ --disable-static --with-python=/usr/bin/python2 
make
make install
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/path/to/store/libxml2/lib/

and then put the following command in your bashrc (~/.bashrc): export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/path/to/store/libxml2/lib/

babouFomb commented 7 years ago

I followed the steps of your last post to compile and locally install libxml2, then add the path to the lib in my bachrc.

Now the errors are different from the previous errors,

In the two attached files, I get all output when installing libxml2 and compiling ipmacc on both platforms. tegra-ipmacc.txt x86-ipmacc.txt

lashgar commented 7 years ago

The error is in compiling srcML where -minline-all-stringops is not supported by GCC 5.x.

IPMACC has been tested with GCC 4.x (4.4.6 specifically) and I have not tested ipmacc under GCC 5.x. My guess is ipmacc cannot be compiled with GCC 5.x since it relies on nvcc (nvcc is certainly not compatible with GCC 5.x.)

babouFomb commented 7 years ago

Hi,

I removed all tar *in compile_all, and do extraction manually.

I fixed in all Makefile, CC=gcc-4.8 and CXX=g++-4.8 in order to solve the problem with gcc/g++ version.

In srcML/srcml/src/Makefile:

Line 94, I fixed LIBXML_INC_DIR to my actual libxml2-2.7.6 install/include location. This solves the problem of libxml2 version.

I comment -lgcrypt in line 99 (in order to solve ld error about lgcrypt, lthough the lib gcrypt11 is installed)

Now, srcML is compiled, and in my srcML/srcml/bin/ I have libsrcml.a src2srcml srcml2src srcmldirect

But, when I tried to compile ipmacc stdvector.cpp -o test, I get an error:

ImportError: /home/rokiatou/libxml2-2.7.6/install/lib/libxml2.so: version LIBXML2_2.9.0 not found (required by /usr/lib/python2.7/dist-packages/lxml/etree.so

lashgar commented 7 years ago

From the error I would say the original lxml/etree.so from python 2.7 is compiled and linked against the original libxml2.so and requires that version. The solution might be recompiling python lxml. The instruction can be found here: http://lxml.de/installation.html#requirements

Notice: Make sure python lxml installation does not overwrite the current lxml/etree.so.