datatype_builder for std::vector

[tkonolige]

How can I construct a datatype_builder for std::vector? I'm trying to use all2all with a std::vector<std::vector<int>>.

[patflick]

Hi,

unfortunately, that doesn't work atm. The datatype_builder can create types only for fixed size data types. However, std::vector is dynamically sized. This also wouldn't be easy to implement, because this is also a limitation of the underlying MPI. The memory segment used for alltoall should be contiguous, or if they are gapped, the gaps need to be a multiple of the vector's data type.

I suggest for now you linearize your vector into a single vector, then using all2allv is rather straight forward:

std::vector<std::vector<T>> my_data = .. ;

// create a size vector
std::vector<size_t> sizes(comm.size());
size_t total_size = 0;
for (int i = 0; i < comm.size(); ++i) {
    sizes[i] = my_data.size();
    total_size += my_data.size();
}

// linearize
std::vector<T> data_linear(total_size);
std::vector<T>::iterator it = data_linear.begin();
for (int i = 0; i < comm.size(); ++i) {
    it = std::copy(my_data[i].begin(), my_data[i].end(), it):
}

// now do the all2all:
std::vector<size_t> recv_sizes = mxx::all2all(data_sizes, comm);
std::vector<T> data_recv = mxx::all2allv(data_linear, sizes, recv_sizes, comm);

Then you'll have the received data in a single vector, but the recv_sizes will help to disentangle the messages from each of the origin processors.

Maybe I should just add a function that does all that to mxx. Although, this function would not be low-overhead and would hide a lot of complexity. For MPI (and mxx) it should be important to think about data layout.

Thoughts?

[tkonolige]

That seems like a reasonable approach.

It might make sense to allow for automatic serialization if the object has a defined serialization function. I believe Boost.MPI does this.