lightsighter
commented
4 years ago Fixing this issue should also address this warning that we're currently getting on GASNet builds:
mpicxx -o /home/mebauer/legion/runtime//realm/activemsg.cc.o -c /home/mebauer/legion/runtime//realm/activemsg.cc -DENABLE_REMOTE_TRACING -DCTRL_REPL_TRACING -DUSE_DISK -march=native -DUSE_ZLIB -O0 -ggdb -Wall -Wno-strict-overflow -I/home/mebauer/legion/runtime/ -I/home/mebauer/legion/runtime//mappers -I/home/mebauer/legion/examples/circuit -I/usr/local/gasnet-1.32.0-openmpi/include -I/usr/local/gasnet-1.32.0-openmpi/include/ibv-conduit In file included from /usr/local/gasnet-1.32.0-openmpi/include/gasnet.h:177:0, from /home/mebauer/legion/runtime//realm/activemsg.cc:31: /usr/local/gasnet-1.32.0-openmpi/include/ibv-conduit/gasnet_core_fwd.h:24:0: warning: "GASNET_CONDUIT_IBV" redefined [enabled by default]
define GASNET_CONDUIT_IBV 1
^ In file included from /home/mebauer/legion/runtime/realm/realm_config.h:23:0, from /home/mebauer/legion/runtime//realm/activemsg.cc:16: /home/mebauer/legion/examples/circuit/realm_defines.h:16:0: note: this is the location of the previous definition
define GASNET_CONDUIT_IBV
^
It appears that Legion is not currently using the GASNet-provided compiler flags for compiling modules that #include, or for linking libgasnet.
With a debug-mode GASNet, the lack of GASNet's compiler flags leads to a large number of warnings that we intentionally suppress (mostly notably -Wunused) when compiling with gcc < v6.0. The GASNet makefile fragments and pkgconfig files provide $(GASNET_CXXCPPFLAGS) $(GASNET_CXXFLAGS) that include these warning suppression flags, as well as other flags that are needed on some platforms (eg to enable inline assembly support, C99 support or POSIX support in some exotic compilers). There are likely cases that will fail to compile at all in the absence of these configure-detected flags.
GASNet also provides linker flags (in GASNET_LDFLAGS and GASNET_LIBS) that include everything (notably -l and -L options) needed to correctly link the current conduit and related network libraries. These often include things like custom pathnames to network driver installs provided by the user when he configured GASNet (eg when building against an IB verbs or OFI library that was installed in a custom location, ie not /usr/lib). These variables also do not currently appear to be used by Realm links. Currently runtime/runtime.mk appears to be hard-coding values for several conduits (ibv, gemini, aries, psm, mpi and udp), however this list excludes several conduits (pami, ofi, mxm) and in general is a fragile approach as it discards any settings detected or provided by the user when configuring GASNet. This is very likely to cause link failures on many configurations.
These flags are documented in the section "Basic Usage Information" of the GASNet README. We provide the settings via Makefile fragments and equivalent pkgconfig data files.
I see some code in cmake/FindGASNet.cmake that appears to be querying these flags, but they don't currently seem to make it to the compile line.
We'd like to encourage the Realm team to find a way to use our provided flags, and are willing to help if there's anything we can do on our side to make this integration work as intended.
CC: @phhargrove