Open lerandc opened 4 years ago
Has anyone taken a look at this yet? It would be nice to have implemented, even if it just means saving png's of the approach taken by the prismatic-gui!
so I've been recently looking into some options for the implementation of this (and generalized modularity of pyprismatic, where we could ideally run the subroutines of prismatic freely through python). one of the challenges is minimizing memory costs when we transfer data from the python interface to the c++ compiled backend. we could just copy the data over manually from the numpy memory addresses and have a lot of data passing and call it a day, this wouldn't be that tricky, though it would require the re-initialization of the parameter object in both python and the backend libraries every single time an action is taken.
instead, ideally, we could use the same physical memory for everything and never actually move data around. for example, to my understanding the numpy libraries operate directly on C-arrays, and retrieve pointers to the numpy array data to do so. in Prismatic, however, we have an array class abstraction (defined in ArrayND.h) which wraps around C-style arrays and provides an interface. all of the constructors of the class create copies of data currently, like so:
test code:
std::vector<PRISMATIC_FLOAT_PRECISION> data({0,1,2,3,4,5,6,7});
std::array<size_t,2> dims = {2,4};
ArrayND<2, std::vector<PRISMATIC_FLOAT_PRECISION>> testArray(data, dims);
std::cout << &data[0] << std::endl;
std::cout << &testArray[0] << std::endl;
sample output (memory addresses for the start of arrays):
0x556eb330d470
0x556eb3303730
the best, sustainable way to accomplish S-matrix (generally, all data) reusabiltiy and things like the probe analyzer would be to do the following: 1) define a new constructor for the ArrayND class which accepts pointers 2) define a parameter object/class with all the same fields as the parameter object in the backend, and setters/getters that control the rank and data type of the arrays (i.e., forcing rank 4 and single float)
if done right, this should be safe, but of course, would require a bit of testing- especially when working off of HPC systems over several nodes.
for small sims, the data copying isn't necesarilly bad-- of course it would be much slower than optimal-- but for large sims, you would quickly run into speed and memory issues if we just copy the arrays.
Considering that this kind of things would be useful mostly for case where the S-matrix is large, copying to numpy array is not very attractive (worth it?) even if this is the easiest route. Having a python wrapper to the ArrayND
object would be definitely better!
As discussed on the gitter chat by @erh3cq, it would be useful to run the probe analyzer and to reuse the S-matrix calculation in PyPrismatic. Implementation could be similar to the Qt slots used on the GUI as mentioned by @apryor6