search

charmplusplus / charm

The Charm++ parallel programming system. Visit https://charmplusplus.org/ for more information.
Apache License 2.0
207 stars 50 forks source link

Update AMPI's version of ROMIO #952

Open stwhite91 opened 8 years ago

stwhite91 commented 8 years ago

Original issue: https://charm.cs.illinois.edu/redmine/issues/952

ROMIO is no longer released/packaged separately from MPICH, but it still has its own separate build system and its structure is still very similar to the version (ROMIO-1.2.4) that we package with AMPI.

ROMIO-1.2.4 was released in May, 2002, while the latest version of ROMIO that ships with MPICH-3.2 is from March 2008. It makes use of some MPI-2.2 features (mostly datatype related ones) and should have improved performance.

stwhite91 commented 5 years ago

Original date: 2017-01-12 05:23:39

Need to implement generalized requests, datatypes, and anything else it uses that we don't currently support.

matthiasdiener commented 5 years ago

Original date: 2017-06-10 22:06:19

Note that the version that is currently integrated in AMPI is based (mostly) on MPICH2 0.91. Integrating ROMIO 1.2.5 from MPICH2 0.92 is already not trivial, since the configure format changed.

For reference, old mpich versions with ROMIO are here: ftp://ftp.mcs.anl.gov/pub/mpi/old

The closest match I have found to our current ROMIO is: ftp://ftp.mcs.anl.gov/pub/mpi/old/mpich-1.2.5-1a.tar.gz

Official MPICH git repository (includes ROMIO) is here: https://github.com/pmodels/mpich

Despite the last change note in the README file being from 2008, ROMIO still receives regular patches in the git repo.

matthiasdiener commented 5 years ago

Original date: 2017-06-12 21:44:32

I'm preparing a series of patches to update our ROMIO to 1.2.5-1a (first patch here: https://charm.cs.illinois.edu/gerrit/#/c/2665/ https://github.com/UIUC-PPL/charm/commit/36e9e76eedfff68b23bed72452530181aab1e3f3 ).

Currently, 15 out of 16 ROMIO tests pass with AMPI (with all patches applied):

ROMIO tests working:

ROMIO tests NOT working (1):

matthiasdiener commented 5 years ago

Original date: 2017-06-19 04:58:48

With the MPI_Comm_dup patch (https://charm.cs.illinois.edu/gerrit/#/c/2706/ https://github.com/UIUC-PPL/charm/commit/33836785127d8b8b7cfd582d9b573370afa54d2a), all ROMIO tests (except the one requiring PMPI) pass.

stwhite91 commented 5 years ago

Original date: 2017-07-13 03:03:15

Marking "In Progress" to reflect the fact that we've updated to a more recent version, though not the most recent version.

stwhite91 commented 5 years ago

Original date: 2017-08-29 17:27:26

Update our ROMIO to v1.2.6, which can be built as a shared object / dynamic library: https://charm.cs.illinois.edu/gerrit/#/c/2940/

matthiasdiener commented 5 years ago

Original date: 2017-08-29 17:28:55

Note that the patch mentioned above (https://charm.cs.illinois.edu/gerrit/#/c/2940/) does not enable compilation as a shared library yet. Some manual changes to the Makefile are necessary.

matthiasdiener commented 5 years ago

Original date: 2017-11-01 19:46:32

Pushing to 6.9.1

stwhite91 commented 5 years ago

Original date: 2017-12-19 20:48:26

We'd still like this to be in 6.9.0 if possible. I don't expect 6.9.0 to be released in the next 3-4 weeks.

evan-charmworks commented 5 years ago

Original date: 2019-02-12 17:39:45

https://charm.cs.illinois.edu/gerrit/c/charm/+/4933 https://github.com/UIUC-PPL/charm/commit/b25d36ce2fb19985ba0494481c2a1e4afe2c7d53

matthiasdiener commented 5 years ago

Original date: 2019-02-12 18:00:42

The patch above updates ROMIO to the version shipped with mpich2-1.4.1p1. For newer versions of ROMIO, we would need support for the external32 data representation, including an implementation of MPI_Pack_external and MPI_Unpack_external). With such support, we could run all ROMIO versions up to 3.3 at least.

stwhite91 commented 5 years ago

Original date: 2019-02-12 18:12:29

I think we can implement the MPI_Pack_external() and MPI_Unpack_external() routines as calls to MPI_Pack() and MPI_Unpack(), since we don't insert any kind of header inside our MPI_Pack and MPI_Unpack routines (it just copies the bytes as they are). On little-endian systems though we should convert to big-endian for external32.

matthiasdiener commented 5 years ago

Original date: 2019-02-12 18:23:10

Apart from the endianness conversion, wouldn't we also need to adjust the size of data types to the external32 format?

stwhite91 commented 5 years ago

Original date: 2019-02-12 18:33:26

Yeah, that too. Looking into it more, it might be worth us checking to see if HDF5 uses MPI_File_set_view with external32 or something. If not, we could implement the external32 routines as runtime aborts or something

matthiasdiener commented 5 years ago

Original date: 2019-02-12 18:47:35

From what I could see, hdf5-1.10.1 only calls MPI_File_set_view with the "native" data format, which should be fine. Independent of that, having fake external32 routines can be useful in any case. For the long term, we could look at copying mpich's external32 implementation (which is a few thousand lines of code :-S).

evan-charmworks commented 5 years ago

Original date: 2019-02-15 20:29:58

Gerrit patch 4933 to upgrade us to mpich2-1.4.1p1 is merged. I'll leave this issue open since it's not the newest version (see above discussion).

matthiasdiener commented 5 years ago

WIP patch to update ROMIO to 3.2.1: https://charm.cs.illinois.edu/gerrit/c/charm/+/5151

matthiasdiener commented 5 years ago

ROMIO has been upgraded to 3.2.1 in PR #2141. Newer versions of ROMIO (3.3+) consist of two libraries, libmpio and libmpl, and will require more changes to our build system.

evan-charmworks commented 1 year ago

Since we now embed ROMIO's objects into the AMPI library and no longer use its own .a/.so etc, are the two separate libraries still a major blocker on updating ROMIO further?