search

charmplusplus / charm

The Charm++ parallel programming system. Visit https://charmplusplus.org/ for more information.
Apache License 2.0
203 stars 49 forks source link

Build target(s) for Intel MIC #190

Closed PhilMiller closed 6 years ago

PhilMiller commented 11 years ago

Original issue: https://charm.cs.illinois.edu/redmine/issues/190

We should incorporate the work done to build on MIC with the main codebase. If it's easily backported, it should go on the stable branch as well.

ericjbohm commented 5 years ago

Original date: 2013-06-20 22:14:35

The initial issues here stem from subtle interactions between the Intel cross compiler scheme for MIC (aka Intel Xeon Phi, KNC, Knight's Corner, plus whatever the, arguably overly fecund, brand name developers from Intel come up with next as a name for these things) and Charm++ compiler wrapper settings.

All libraries which a MIC application binary must link to must go through the cross compiler (icc -mmic, or a more gnarly explicit path call to x86_64-k1om-linux-g++, which should probably be avoided as we have no guarantee about name stability for that thing) and its linker.

All binaries which would be executed on the host (usually charmc and various configure tests) should usually not go through the cross compiler. In the past, we have generally used CMK_NATIVE (aka charmc -seq) for these. However, -seq has crept into a bunch of things which must be linked to by application binaries, such as libckqt.a. So we now have some semantic confusion over what -seq means in this context and how it should be configured.

In the general case, on machines like Stampede, the mic0 host is not always directly accessible, so various tools like charmrun, should not be compiled with -mmic. On experimental machines, like NCSA's Xeon Phi, root access to mic0 is available and they might wish to use charmrun natively compiled for the MIC. IMHO, anyone doing that should be expected to solve their own wacky compilation and linking issues. Our main audience is people trying to leverage Phi on Stampede.

On a related note, the MIC environment has its own issues. Notably, there is no MIC ibverbs library to link to on stampede, so you cannot even attempt to use their verbs proxy within the MIC context.

One can build using intel MPI with one binary for host and one for MIC and use ibrun.symm to have PEs running on both host and MIC. The process currently requires some somewhat arduous hackery. That process will be covered in a different issue.

A proper native layer should be able to use the dual binary launch scheme of ibrun.symm, but the necessary underpinning is not yet developed.

mprobson commented 5 years ago

Original date: 2015-08-18 16:12:51

Just saving some useful changes from knc3 which is being decommissioned today:


1.       renamed:    src/arch/mpi-linux-x86_64/conv-mach-mic.h -> src/arch/common/conv-mach-mic.h
1.       renamed:    src/arch/mpi-linux-x86_64/conv-mach-mic.sh -> src/arch/common/conv-mach-mic.sh
stwhite91 commented 5 years ago

Original date: 2016-12-13 23:27:36

is this still relevant or should it be closed?

mprobson commented 5 years ago

Original date: 2016-12-14 21:14:30

Sam White wrote:

is this still relevant or should it be closed?

I'm not sure. Kavitha or Seonmyeong can comment further as they've been using new KNL hardware recently.

stwhite91 commented 5 years ago

Original date: 2017-02-02 04:38:45

KNL is self-hosted and doesn't require the complicated dual binary launch scheme. Since this is off the radar for now, but something similar could appear in the future, reassigning to PPL for now.