Open kuberry opened 4 years ago
In order to speed up parallel computation, it was necessary to modularize the source and target coordinates as well as neighbor lists into a struct called PointConnections. To keep the functor size light, this struct does not manage host/device data relations, and instead is entirely on the device.
Future work can still be done in allowing a PointConnections object to be provided to the GMLS instance, but the host to device copy should be done in the PointConnections object prior to setting it.
Implementation of this class will be used as a template for extracting other view access from the toolkit and replacing it with objects.
This issue deals with both the PointData class as well as its use in the GMLS class.
To date,
a view is passed by the user to the GMLS class. If template arguments match, the view is shallow-copied. If they do not, even if data lives on the same device, then a deep copy is made.
If point information were requested from the GMLS class, then there are two questions that need to be addressed regarding the implementation.
Should the data inside of the PointData class be able to be changed by the PointData instance given to GMLS? It continues to persist outside of it being sent to GMLS. The user could change it, and if so, what should happen to the instance inside of the class?
How should the information in the PointData class be made available through an accessor?
Issues with current approach
User manipulates Kokkos views directly
Kokkos view setting of data is required to be handled by GMLS interface
It is not obvious if view being used results in shallow or deep copy
If it results in a shallow copy, the view could be changed outside of the GMLS class, which would leave the GMLS state inconsistent with the coordinates
Solution should:
Only allow shallow copy of data (user can convert objects prior to providing to GMLS if they want a deep copy)
Prevent changing data inside GMLS class after it is provided by outside PointData instance
Allow instance inside of GMLS to be accessed externally in a way that does not allow for modification
Path forward:
Eliminate the interface for setting coordinates with views
Only accept setting a PointData view is of the same template type (prevents a deep copy where unintended)
Pass by value the PointData class to GMLS, which will shallow copy the underlying data. This allows the user to modify their PointData instance in the future without modifying the GMLS class, with the exception that modifying entries in the underlying view will still affect the instance in the GMLS class (shallow copy, afterall).
Add a marker for
shared_underlying_data
or something similar to mark when a copy has been made. Then, if an attempt is made to modify the data, a check can be made ofshared_underlying_data
to determine whether to reallocate, copy, and then modify.This gives the best of both worlds in the sense that by default, no deep copies will be made and only one copy of the coordinates will exist (unless someone modifies the original PointData instance). However, if the original instance of PointData is changed after sharing, then it resizes, reallocating memory and causing a second instance to exist.