search

static-frame / arraykit

Python C Extensions for StaticFrame
Other
8 stars 2 forks source link

22/add array2d to 1d #160

Closed gettodaze closed 4 months ago

gettodaze commented 5 months ago

From https://github.com/static-frame/arraykit/issues/22

flexatone commented 4 months ago

Interestingly, out-performance degrades with scale. No material difference seen due to the size of the tuple. Might be able un-roll the loop, or special case a contiguous input array. array2d_to_array1d

flexatone commented 4 months ago

There was a flaw in the preparation of the fixture arrays used in the performance tests (row / column sizes were swapped). Fixing this error, we now see excellent out-performance maintained with scale; perhaps more intuitively, we see out-performance diminish with tuple size. array2d_to_array1d

flexatone commented 4 months ago

I have created an alternate iterator interface, array2d_tuple_iter, that iterates lazily over tuples rather than creating a 1D array. This too shows excellent out-performance to native Python alternatives, both in creating a list and in just calling next(). array2d_tuple_iter

gettodaze commented 4 months ago

Yay! Thanks @flexatone!