Closed adamklein closed 10 years ago
After some blood, sweat, tears - and looking into internal representation of stuff - I think this is working! I'm going to create a PR (#140) for review.
For a benchmark - it used to take 1m7s to unnest 173 frames with 35 columns to 7274 rows, now takes closer to 35s. This is still abominable... need to get this faster!
The stuff with dispatchers is nasty :-( I feel guilty for adding that to the codebase! I think there might be a better way of dealing with it - but did not have much time to look into this yet. I'll see if I can have a look when 1) I get back home 2) I get my laptop fixed (currently does not work much..) Though that might not really help with the fancier dispatchers like the one needed here.
@tpetricek @hmansell
There is a bug in
createVectorListDispatcher
(which was created forappendN
functionality), and I’m struggling to fix it. I believe I need to build a Linq expression to dynamically convert anIVector list
to anIVector<T> list
.You can see the problem by running the test
AppendN works on non-primitives
in my branch https://github.com/BlueMountainCapital/Deedle/tree/ak.fixAppendNI had used the function
createVectorDispatcher
fromVectorHelper.fs
as a template, and I tried to follow the pattern and simply add use of list type constructor, but it yields the error message:The problem is
Expression.Convert
. What I believe I need to do is construct a different linq expression to map the dynamically generatedIVector -> IVector<’T>
conversion expression over theIVector list
argument, to yield aIVector<’T> list
. Any ideas how to do this?This is critical for unnest to work better, as right now it appends all the nested frames via a pairwise reduce – as right now, it’s O(N^2).