search

charmplusplus / charm

The Charm++ parallel programming system. Visit https://charmplusplus.org/ for more information.
Apache License 2.0
207 stars 50 forks source link

Support XPMEM interprocess communication within node when SMP is used within process #3794

Open ericjbohm opened 8 months ago

ericjbohm commented 8 months ago

We have basic support for XPMEM, but it is limited to nonsmp mode and has some rust from disuse that needs to be addressed.

There are some routing and thread safety issues that need to be addressed to support both XPMEM and SMP paths at the same time. A similar approach could be applied to CMA, or PXSHM, but we would expect XPMEM to be performant and featureful enough to address the performance issues.

ericjbohm commented 7 months ago

The CXI branch supports use of XPMEM, but I haven't found it, or CMA, to enhance performance in benchmarks like commbench on crusher. For example, I've noticed CMA can show an improvement in pingpong, but then has lower aggregate bandwidth in flood.

ericjbohm commented 7 months ago

Also, enabling these is not generally safe, as some tests (like large_bcast) will fail with CMA actively enabled.