search

enthought / distarray

Default Repo description from terraform module
BSD 3-Clause "New" or "Revised" License
5 stars 1 forks source link

O-8: distarray global API -- array slicing #188

Closed kwmsmith closed 10 years ago

kwmsmith commented 10 years ago

Objective: To provide array slicing functionality from the global address space, and to determine the semantics of the result of distributed array slicing, that is, whether to make the result of a slice a distributed array view, or a copy, or user-specifiable.

Relevance: Array slicing and array views are a key feature of existing NumPy, and providing their distributed analogue will be useful and familiar to existing NumPy users. Array slices will result in a new array, but whether they share the underlying data buffer with the original array is complicated by the distributed nature of ODIN. Shared distributed array views are non-trivial, and this task will explore the feasibility of distributed array views with shared-memory semantics.

Description: Distributed array slicing has ambiguities that will require exploration and prototyping to determine the best strategy for different usecases: should distributed views be supported for all slicing operations? If an array slice results in a sub-optimally distributed array, should the sliced array be redistributed for performance reasons? How can the end-user control different array slicing options?

bgrant commented 10 years ago

So far, PRs #384, #385, #387, #395.

kwmsmith commented 10 years ago

@bgrant what in this milestone is still to be implemented? We can separate those parts out; I'd like to mark what we can as complete to move this along.

bgrant commented 10 years ago

Hm- this task is pretty high level. What isn't supported yet are slices on anything but Block or NoDist arrays. The explicit view method on DistArrays also is unimplemented.

kwmsmith commented 10 years ago

OK, moving this issue to 0.5 to be updated at that release.

kwmsmith commented 10 years ago

@bgrant What can be updated for this SBIR objective in 0.5?

bgrant commented 10 years ago

Hm- view is now implemented. Otherwise, I don't think there has been any work on slicing in this release.

kwmsmith commented 10 years ago

From what I can tell, with block-based slicing in place, there is a clear path forward to implementing slicing for the other distribution types. In the interest of spending time implementing other critical features, I'm closing this milestone as complete for the purposes of the SBIR.