Datatype aliases

[dalcinl]

Looking at the current proposal for Datatype handles, there is a comment suggesting that language-specific types (e.g. MPI_LONG) may not be not necessary, and that they could be be just aliases to fixed-size types (e.g. MPI_INT32_T or MPI_INT64_T).

I believe this suggestion should be explored further, although this may be seen as a non strictly-ABI change to the standard and may trigger unnecessary friction upstream this working group.

The NumPy package user the aliasing approach for datatypes. NumPy datatype (numpy.dtype) instances ultimately refer to a fixed-size C type.

>>> import numpy as np
>>> np.dtype('long')
dtype('int64')

There are however some drawbacks to aliasing. Once MPI has an stable ABI, people will start abusing of it. Imagine that a user wants to serialize some C/C++ data structure to disk (or network), and this person uses the integral value of an MPI_Datatype for serialization. Now suppose the user write (or send) the value of MPI_LONG from Linux 32 bits (thus MPI_LONG is an alias for MPI_INT32_T) . Later, that value is read (or recv) from Linux 64 bits to deserialize or reconstruct the datatype. The deserialized datatype value will map to MPI_INT32_T, but in the new architecture MPI_LONG alias to MPI_INT64_T, and this may not be what the rocky user intended. Long story short: Aliasing predefined language-specific datatypes to predefined fixed-size datatypes prevent easy serialization via the handle value.

MPI should have an API similar to the Fortran-specific MPI_Type_match_size to query for size-specific datatypes. For example:

int MPI_Type_generic_match_size(MPI_Datatype template, int size, MPI_Datatype *outtype);

that given a predefined "template" datatype (e.g. MPI_SHORT or MPI_INT or MPI_LONG) and a byte size (e.g. 8) would return the closest datatype with requested size (e.g. MPI_INT64_T) in the same language and type class category (boolean/logical, signed/unsigned integral, real/complex floating). This routine would work well for the official MPI languages C/C++/Fortran, and would render the Fortran-specific MPI_Type_match_size obsolete.

[jeffhammond]

Note that external32 representation exists to solve this problem already.

[dalcinl]

@jeffhammond You are not getting my point. This is not about serializing data. It is about serializing predefined MPI handles.

[jeffhammond]

Okay but I think it works with aliasing to fixed size types. If user sends LONG and it turns into INT32, the receiver will know it's INT32 no matter what they think LONG is. I don't see a problem here. In fact, type aliasing seems to make heterogenous cases more straightforward.

[dalcinl]

If user sends LONG and it turns into INT32, the receiver will know it's INT32 no matter what they think LONG is. I don't see a problem here.

What if the actual intention is for the MPI_LONG datatype, after a serialization and deserialization round, to ALWAYS match the platform's C long type? That's the case I'm seeing as problematic if we use aliasing.

[jeffhammond]

I'm perfectly willing to throw out heterogeneous support to get an ABI. Does anyone care about heterogeneous usage anymore?

[dalcinl]

Maybe we should discuss this in a meeting. My only observation for now is the following: Aliasing datatypes like LONG to either INT32_T or INT64_T maybe have drawbacks and unintended consequences. So far, MPI implementations have not done that, LONG is always different than (in the equality comparison == sense ) either INT32_T or INT64_T . Therefore, I would argue that introducing datatypes aliases is a move beyond the scope of getting ABI.

[jeffhammond]

Okay that's fair. We can discuss. I have not committed to aliasing anyways. I just seemed to make sense since an ABI in the traditional sense involves fixing the size of LONG, eg LP64.

[qkoziol]

We have similar issues in HDF5. From the beginning we chose to alias H5T_NATIVE_INT to whatever the "actual" type is for the platform, e.g. H5T_INT32_LE, etc.

[qkoziol]

There is literally no way in HDF5 to "store" H5T_NATIVE_INT - a user can ask for it, but we always return to them the actual type.

There's also a routine that's similar to what @dalcinl suggested: H5Tget_native_type(), which takes an HDF5 datatype and returns the equivalent "actual" type for the particular platform.

[qkoziol]

There's also a utility function, H5Tequal(type1, type2), which returns TRUE when two HDF5 datatypes are the same actual type, even if one or both are aliases.

[dalcinl]

Another observation: without aliasing, the mpi.h header will be mostly platform-agnostic, with very little reliance on the C preprocessor (maybe only to define MPI_VERSION/SUBVERSION, which we could still expose as enum constants besides the #define ones).

@qkoziol I understand and agree with the HDF5 choice. If things were not done that way, that would hinder platform portability of .h5 files. But MPI is not an IO library (except for MPI I/O of course), and even in the heterogeneous case, the user never has to deal with handling the on-wire data, and MPI can ultimately handle heterogeneity on the fly. IMHO, making native types different than size-specific types would be of little benefit to MPI, even for the heterogeneous case. 30 years of MPI history prove my point, up to MPI 4.1, native datatypes are not aliases.

I'm only raising this issue as a reaction a casual comment that Jeff wrote in the proposal for predefined datatype handle values. I want everyone to be aware of the issues. And even if my opinion is ignored (I'm quite used to it) and native types become aliases, then at least everyone has to understand that MPI would a new routine modelled exactly like H5Tget_native_type().