achirkin / easytensor

Many-dimensional type-safe numeric ops
https://hackage.haskell.org/package/easytensor
BSD 3-Clause "New" or "Revised" License
46 stars 2 forks source link

Slicing a DataFrame #14

Closed Rotaerk closed 5 years ago

Rotaerk commented 6 years ago

Is there some way to take, say, a Vector _ 10 and drop the last 5 elements to get a Vector _ 5? In Vulkan, there are cases where the size of the array that Vulkan returned is much larger than the number of actual valid elements.

For example, within VkPhysicalDeviceMemoryProperties, the memoryHeaps field is an array that always has length VK_MAX_MEMORY_HEAPS, but only the first memoryHeapCount elements are actually valid.

I would like to be able to, say, fold over these elements. As it stands, the only way I can see to do it is to first convert it to a list, and then use the standard list functions for that.

achirkin commented 6 years ago

Hm, probably, there is no such function in Numeric.DataFrame.SubSpace right now. I definitely need to implement it.

On the good side, you still can do it via unsafeThawDataFrame + copyMutableDataFrame + unsafeFreezeDataFrame. It is ok to use unsafe thaw/freeze as long as you don't really modify them.

Implementing such a function is a little tricky, because sometimes it requires copying data, sometimes doesn't.

Another thing is that since you use the vectors in Vulkan, you always have to check if the underlying arrays are pinned. Normally, easytensor-vulkan helpers do that for you.

If you want to fold over the part of the vector, you can use indexed folds, such as iwfoldl or iwfoldMap and discard elements based on their indices. Not super efficient, but you don't need to copy elements at least.

achirkin commented 5 years ago

This function is implemented now, available in Numeric.DataFrame.SubSpace.slice.

I am closing the issue, but feel free to reopen it if there are any problems.