search

ICLDisco / parsec

PaRSEC is a generic framework for architecture aware scheduling and management of micro-tasks on distributed, GPU accelerated, many-core heterogeneous architectures. PaRSEC assigns computation threads to the cores, GPU accelerators, overlaps communications and computations and uses a dynamic, fully-distributed scheduler based on architectural features such as NUMA nodes and algorithmic features such as data reuse.
Other
47 stars 17 forks source link

Building code with both gcc OpenMP and Parsec fails #123

Closed abouteiller closed 7 years ago

abouteiller commented 7 years ago

Original report by Benjamin Lorendeau (Bitbucket: Agyar, GitHub: Agyar).

Hi,

parsec version: (latest) commit b858c4aacbfc00d532caef2b7d9e773f32945c11 (bb) gcc version: gcc (GCC) 6.3.1 20170109 (bb)

while building Code_Saturne with latest PaRSEC enabled + OpenMP enabled, I encountered this issue:

#!bash

libtool: compile:  gcc -DHAVE_CONFIG_H -I. -I/home/ben/Work/Repos/saturne/src/base -I../.. -I/home/ben/Work/Repos/saturne/src/alge -I/home/ben/Work/Repos/saturne/src/cdo -I/home/ben/Work/Repos/saturne/src/ctwr -I/home/ben/Work/Repos/saturne/src/lagr -I/home/ben/Work/Repos/saturne/src/mesh -I/home/ben/Work/Repos/saturne/src/bft -I/home/ben/Work/Repos/saturne/src/fvm -I/home/ben/Work/Repos/saturne/src/mei -I/home/ben/Work/Repos/saturne/src/gui -I/home/ben/Work/Repos/saturne/src/turb -I/home/ben/Work/Repos/saturne/libple/src -I../../libple -I/home/ben/Work/Install/openmpi/include -DOMPI_SKIP_MPICXX -D_POSIX_SOURCE -DNDEBUG -funroll-loops -O2 -Wuninitialized -fexcess-precision=fast -std=c99 -fms-extensions -funsigned-char -pedantic -W -Wall -Wshadow -Wpointer-arith -Wcast-qual -Wcast-align -Wwrite-strings -Wstrict-prototypes -Wmissing-prototypes -Wmissing-declarations -Wnested-externs -Wunused -Wfloat-equal -Wmisleading-indentation -Wduplicated-cond -fdiagnostics-color=auto -fPIC -m64 -std=c1x -mcx16 -I/home/ben/Work/Install/parsec/include -I/home/ben/Work/Install/openmpi/include -fopenmp -MT cs_boundary_conditions.lo -MD -MP -MF .deps/cs_boundary_conditions.Tpo -c /home/ben/Work/Repos/saturne/src/base/cs_boundary_conditions.c  -fPIC -DPIC -o .libs/cs_boundary_conditions.o
In file included from /home/ben/Work/Install/parsec/include/parsec/sys/atomic.h:17:0,
                 from /home/ben/Work/Install/parsec/include/parsec/class/parsec_object.h:123,
                 from /home/ben/Work/Install/parsec/include/parsec/class/list_item.h:11,
                 from /home/ben/Work/Install/parsec/include/parsec/parsec_internal.h:13,
                 from /home/ben/Work/Install/parsec/include/parsec/data_internal.h:10,
                 from /home/ben/Work/Repos/saturne/src/alge/halo_desc.h:11,
                 from /home/ben/Work/Repos/saturne/src/alge/cs_parsec_gradient.h:36,
                 from /home/ben/Work/Repos/saturne/src/base/cs_boundary_conditions.c:56:
/usr/lib/gcc/x86_64-pc-linux-gnu/6.3.1/include/stdatomic.h:40:1: désolé, pas implanté : « _Atomic » with OpenMP
 typedef _Atomic _Bool atomic_bool;

as I remember, I use to get through this issue by linking using -lgcc -lgcc_s -latomic. However the issue now is at compile time. I remember having seen some related CMAKE options but I do not see them in the latest versions of PaRSEC.

Removing OpenMP from the build made it build and work succesfully.

abouteiller commented 7 years ago

Original comment by Thomas Herault (Bitbucket: herault, GitHub: therault).

This is a limitation of Open MP, that assumes only C-99, so does not allow the compiler to properly define C-11 Atomics. See https://gcc.gnu.org/bugzilla/show_bug.cgi?id=65467

We are working on a better set of .h files that will expose less intrinsics of PaRSEC, hopefully removing this error.

In the meanwhile, can you try to disable C-11 despite the fact that your compiler supports it.

To do so, try to set -fopenmp into your configuration CFLAGS before redoing the CMake from a clean directory (remove at least CMakeFiles/ and CMakeCache.txt)? This should make the CMake test to use C-11 Atomics fail, and thus make PaRSEC fall back on x86 or the old gcc-atomics.

abouteiller commented 7 years ago

Original comment by Benjamin Lorendeau (Bitbucket: Agyar, GitHub: Agyar).

Passing -fopenmp before ccmake does not change its choice over the version of C and cmake still puts -std=c1x in its CFLAGS. I tried to add -std=c99 in addition with no success, cmake seems to add the -std=c1x post to that when running the configure. 

#!bash

Build/parsec_git » CFLAGS="-fopenmp -m64 -mcx16 -std=c99" cmake .
-- Building for target x86_64
-- Found target X86_64
-- Performing Test PARSEC_HAVE_COMPARE_AND_SWAP_128
-- Performing Test PARSEC_HAVE_COMPARE_AND_SWAP_128 - Failed
-- Performing Test PARSEC_HAVE_COMPARE_AND_SWAP_128
-- Performing Test PARSEC_HAVE_COMPARE_AND_SWAP_128 - Failed
--       support for 32 bits atomics - found
--       support for 64 bits atomics - found
-- Could NOT find GTG (missing:  GTG_LIBRARY GTG_INCLUDE_DIR)
-- Cython version 0.25.2 found
-- Could NOT find Omega; Options depending on Omega will be disabled (missing:  OMEGA_INCLUDE_DIR OMEGA_LIBRARY)
-- PARSEC Modular Component Architecture (MCA) discovery:
-- -- Found Component `pins'
-- ---- Module `iterators_checker' is ON
-- The PAPI Library is found at /usr/lib64/libpapi.so
-- Module papi not selectable: PARSEC_PROF_TRACE disabled.
-- ---- Module `print_steals' is ON
-- ---- Module `ptg_to_dtd' is ON
-- Module task_profiler not selectable: PARSEC_PROF_TRACE disabled.
-- Component pins sources: mca/pins/pins.c;mca/pins/pins_init.c
-- -- Found Component `sched'
-- ---- Module `ap' is ON
-- ---- Module `gd' is ON
-- ---- Module `ip' is ON
-- ---- Module `lfq' is ON
-- ---- Module `lhq' is ON
-- ---- Module `ltq' is ON
-- ---- Module `pbq' is ON
-- ---- Module `rnd' is ON
-- Component sched sources:
-- PARSEC Modular Component Architecture (MCA) discovery done.
-- Generate precision dependencies in /home/ben/Work/Repos/parsec/data_dist/matrix
  File "/home/ben/Work/Repos/parsec/tools/precision_generator/genDependencies.py", line 79
    if new_file <> filename or self.prefix is not None:
                 ^
SyntaxError: invalid syntax
-- Generate precision dependencies in /home/ben/Work/Repos/parsec/data_dist/matrix - Done
-- Looking for COREBLAS - pkgconfig not used
-- pkg-config not available. You need to provide COREBLAS_DIR or COREBLAS_PKG_DIR
-- Could NOT find COREBLAS (missing:  COREBLAS_INCLUDE_DIRS COREBLAS_LIBRARIES COREBLAS_LIBRARY_DIRS)
-- A library with COREBLAS API not found. Please specify library location.
    COREBLAS_INCLUDE_DIRS = []
    COREBLAS_LIBRARY_DIRS = []
    COREBLAS_LIBRARIES    = []
    COREBLAS_LDFLAGS      = []
-- Could NOT find Doxygen (missing:  DOXYGEN_EXECUTABLE)

Configuration flags:
  CMAKE_C_FLAGS          =  -fopenmp -std=c99 -m64 -std=c1x
  CMAKE_C_LDFLAGS        =  -m64
  CMAKE_EXE_LINKER_FLAGS =

-- Configuring done
-- Generating done
-- Build files have been written to: /home/ben/Work/Build/parsec_git

The action does make the building of PaRSEC to fail though.

abouteiller commented 7 years ago

Original comment by Thomas Herault (Bitbucket: herault, GitHub: therault).

There should be a fix in https://bitbucket.org/herault/parsec/branch/hotfix

We're reviewing it. A test on my mac shows it should do what we want, which is not changing the compiler flags, but use an alternative set of atomics. In the CMake output, you should see

-- Looking for include file stdatomic.h - not found

Instead of

-- Looking for include file stdatomic.h - found
-- Performing Test PARSEC_HAVE_C11_ATOMICS
-- Performing Test PARSEC_HAVE_C11_ATOMICS - Success

You can try the branch if you want, or wait that this is merged into the trunk master.

abouteiller commented 7 years ago

Original comment by Benjamin Lorendeau (Bitbucket: Agyar, GitHub: Agyar).

Tried your patch yesterday, it's working. Thank you for the quick fix.

abouteiller commented 7 years ago

Original comment by Thomas Herault (Bitbucket: herault, GitHub: therault).

Let's keep the issue open until the fix is merged.

abouteiller commented 7 years ago

Original comment by Thomas Herault (Bitbucket: herault, GitHub: therault).

Addresses issue #123

The test STDC_VERSION >= 201112L && !STDC_NO_ATOMICS is not sufficient: there are cases (e.g. gcc-6.3.1 with flag -fopenmp) where the compiler version is bigger than 201112 (bb), and STDC_NO_ATOMICS__ is not defined, but yet, including stdatomic.h makes fail. This is due to the fact that Open MP is not defined for memory models above C99.

This patch propagates the PARSEC_HAVE_C11_ATOMICS to the PaRSEC config headers

→ <<cset f3902a713fb3 (bb)>>

abouteiller commented 7 years ago

Original comment by Thomas Herault (Bitbucket: herault, GitHub: therault).

Merged in herault/parsec/hotfix (pull request #84)

Addresses issue #123

Approved-by: Thomas Herault Approved-by: George Bosilca

→ <<cset 7ae5963f04e3 (bb)>>

abouteiller commented 7 years ago

Original comment by Thomas Herault (Bitbucket: herault, GitHub: therault).

Fixed by Pulled Request #84