friedmud
commented
6 years ago No. @permcody and I were just discussing that nothing in parallel.h should ever assume the user has pre-sized the vector. The few cases where that is true are infuriating and should be changed.
Further, for true non-blocking with unknown size messages this is nearly impossible.
Even further: for complex types you can't predict the size because the size is dependent on the way the packing is done. For instance, the new std::vector<std::vector<T>> packs into a std::vector<char> and encodes the sizes of each vector within there. How is a user going to know how to size the receive buffer there?
No: the right way to do this is already built into the MPI library itself: Statuses. The stat tells you the size. It works. Every time.
There is nothing wrong with stat based code - it doesn't look dumb.
I propose that nonblocking receives should internally do an iprobe() to get a status, then resize the temporary buffer. If there is no message they will return false - returning true if there was a message.
So for receive() the signature would look like:
bool receive (const unsigned int dest_processor_id,
T & buf,
Request & req,
const MessageTag & tag=any_tag) const;Internally it would do a non-blocking probe, get the status, resize the buffer, attach post-wait work and return true (if there is a message) or return false if there's no message.
To use this you would do something like (with a few details left out):
while(true)
{
if (receive (any_source, buff_to_fill, req, tag))
current_receives.push_back(std::make_pair(buff_to_fill, req));
for (auto & rec : current_receives)
{
if (rec.first.test())
{
rec.first.wait(); // Required to to post-wait work
// deal with the data within rec.second (the filled in buffer
}
}
}This basic structure is fully asynchronous and allows overlapping of dealing with the buffers and pulling them into memory. It will work with any type of object and any size of buffer without needing to know anything about the sizes of them beforehand.
roystgnr
I'd thought that "how do you figure out the proper size for a temporary buffer" was an impossible issue to solve, but one solution is simple: "the same way you figure out the proper size for a non-blocking vector receive: make the user do it".
Granted, making the user preallocate with std::vector::resize() leads to code that doesn't look as dumb as making the user initialize std::set with a counting_iterator or making them fill it with a loop, but either way the library would be happy.
On the other hand, in the set case (and in the map case, w.r.t. keys) unlike in the vector case, any preallocation is wasted unless it happens to preallocate an element which will be received, because otherwise it'll just have to be deleted. The waste here is on the order of having to use a temporary buffer in the first place, but if we ever really want to do a non-blocking set or map receive maybe we should just bite the bullet and make them use a non-standard argument list with an extra size_t to specify incoming size.