fslaborg / Deedle

Easy to use .NET library for data and time series manipulation and for scientific programming
http://fslab.org/Deedle/
BSD 2-Clause "Simplified" License
924 stars 196 forks source link

Optimize Frame.AggregateRowsBy #469

Closed zyzhu closed 5 years ago

zyzhu commented 5 years ago

Address #269 The current performance bottleneck is in getting labels by rows.

Typical cases in using Frame.AggregateRowsBy has data frame with way more rows than columns. Switching to getting labels by columns yields a 3x speed improvement in a test case of 100000 rows.

Didn't change the row index to group keys as it will be a breaking change. Will consider it next time as it's more in line with what pandas does.

Needs to fix the breaking performance test to document speed different among different versions in the future.