search

ComputationalRadiationPhysics / graybat

Graph Approach for Highly Generic Communication Schemes Based on Adaptive Topologies :satellite:
Other
8 stars 4 forks source link

Added BufferAdaptor and some STL implementations of it #126

Closed fabian-jung closed 3 months ago

fabian-jung commented 7 years ago

This is not the final solution, but i dont know, where to start to hook things up in graybat. Where i am at is, that i can convert a Type to a void* and size and back. I know, that you are using memcpy in the background, to copy the messages to the send buffers and from the receive buffer. Since there are specilised implementations for non trivial copyable types, these specialisations take care of resizing the input types if needed.

Now all I need to know, is all places in the communication policies, where this is done to make a clean transition. Maybe we can tackle this on the cage layer but i am not sure about this.

fabian-jung commented 7 years ago

Here are some test cases: https://gist.github.com/fabian-jung/1f07b57891db091ba48185c675c8a589

fabian-jung commented 7 years ago

I added the transitive version of the buffer adapter for std::vector. For the moment it is only in the cracen repo, but i will add it to the pull request if you want.

fabian-jung commented 7 years ago

push

erikzenker commented 7 years ago

I am currently working a little more on the serialization topic / serialization policy which has something in common with your approach.Your buffer need to be used in all p2p and collective operations. Give me some time to stabilize this in my mind. Do you have some time constraints when this needs to be merged?

fabian-jung commented 7 years ago

If i understand it correctly, you suggest to alter the cage, rather than the communication policy?

erikzenker commented 7 years ago

I would totally do this in the cage methods and extend the communication policy interface by a probing method.

erikzenker commented 7 years ago

Currently, i experimenting with your code and try to make a buffer adapter for std::array<std::tuple<unsigned, std::string>, 1> (because one example is using it. Would this be a use case for the transitiv buffer adapter ?

erikzenker commented 7 years ago

What is the status of this pull request?

fabian-jung commented 7 years ago

The BufferAdapter is working for non transitive use cases. I found the transitive extension to be too complex for the little benefit it offers.

The integration in graybat is not done, but should be a very easy, now the probe() is implemented. After i am back from the hackathon, i can do that. The api of graybat is not affected.

If std::vector is supported, then serialisation can be done by e.g. boost/serialize by the user.

fabian-jung commented 7 years ago

Does the BytePack Seriallization policy resize a vector, if received via the cage?

erikzenker commented 7 years ago

It does not resize the vector, since it is a serialization policy which serializes data. We had an offline discussion about resizing a container. My idea was to provide a function which does the resizing instead of injecting cage logic into the serialization policy. We should continue the discussion here.