search

charmplusplus / charm

The Charm++ parallel programming system. Visit https://charmplusplus.org/ for more information.
Apache License 2.0
207 stars 50 forks source link

Make AMPI's within-process messaging visible to Projections #1818

Open stwhite91 opened 6 years ago

stwhite91 commented 6 years ago

Original issue: https://charm.cs.illinois.edu/redmine/issues/1818

Currently, AMPI does it's own messaging within a process. By that I mean AMPI invokes an entry method on the destination chare array element (AMPI rank), then once the message is matched it will do a direct memcpy() of the source buffer to the destination buffer, before invoking a Charm++ callback on the sender to notify it of completion of the send request. This means that only the rendezvous message and the completion message are visible in Projections, and all of the real message payload's transfer is invisible.

I'd like to add explicit tracing to AMPI to make the memcpy() visible in all Projections views and to have the number of bytes copied be visible in the appropriate views such as Communication over Time and Communication across PEs.

PhilMiller commented 4 years ago

Is that communication volume between co-located ranks also thus hidden from the LB instrumentation?