search

fslaborg / Deedle

Easy to use .NET library for data and time series manipulation and for scientific programming
http://fslab.org/Deedle/
BSD 2-Clause "Simplified" License
933 stars 195 forks source link

BigDeedle improvements (replacing columns, sorting, merging ordinally indexed) #285

Closed tpetricek closed 9 years ago

tpetricek commented 9 years ago

Support adding/replacing columns with same-indexed series in BigDeedle

In general, we do not (yet?) support joins on BigDeedle frames and series. But if the newly added (joined) frame or series has the same index, then this should work. But - the index might not even support "GetRange" (which is normally called before testing equality), so we first check if they are the same object (which is a useful optimization anyway).

In the future, it would make sense to have some mechanism for testing which operations are supported (so that we do not try calling "restrict" if it is not supported). But for now, equality test should do.

Allow sorting of an already sorted virtual frame/series (#284)

Currently, the abstraction for virtual source does not provide a mechanism for sorting things (and this sounds tricky to do), but if the frame/series is already sorted, we do not need to throw. Better error reporting (or deciding what is not supported) is still TODO.

Support merging on ordinally indexed virtual frames & series

To implement this, we need to represent ordinal index not as int64 * int64 (single range) but as (int64 * int64) list (a sequence of non-overlapping ranges). The commit implements this as the Ranges module. There was also a bug in how missing values were generated in the test suite for BigDeedle, so this is fixed (with a few test fixes).

adamklein commented 9 years ago

Looks great, feel free to merge!