Right now DArray is both the notion of where and the data. I propose to factor out the where into it's own Partitons type, so that we can have a type-hierarchy of partitions, from generic n-dim hypercubes, to partitionings that are only distributed across one axes, to block cyclic or cartesian partitioning.
One important improvement would be that a process can hold more than one chunk and that the chunk array type can be heterogeneous across the DArray allowing us to mix something like a FillArray in and represent
block sparsity more efficiently.
Partitions should support reasoning about moving data from A to B, including this data is local (even though it is a different chunk).
Right now
DArrayis both the notion of where and the data. I propose to factor out the where into it's own Partitons type, so that we can have a type-hierarchy of partitions, from generic n-dim hypercubes, to partitionings that are only distributed across one axes, to block cyclic or cartesian partitioning.One important improvement would be that a process can hold more than one chunk and that the chunk array type can be heterogeneous across the
DArrayallowing us to mix something like a FillArray in and represent block sparsity more efficiently.Partitions should support reasoning about moving data from A to B, including this data is local (even though it is a different chunk).