search

fsdonks / proc

MARATHON post proccessor
0 stars 0 forks source link

Integrate incanter dataset with spork.util.table operations (and vice versa). #16

Closed fs-tom closed 7 years ago

fs-tom commented 7 years ago

We have a somewhat artificial distinction between operations in spork.util.table and incanter datasets. spork tables can use schemas - which allow us much much faster parsing and more efficient storage. Unfortunately, at the type-level, table operations don't currently extend to tables and incanter dataset ops are somewhat locked off in this branch (1.5.6, ~1.9 has a dataset implementation in core.matrix...but that's another can of worms).

Look into patching over these disparities with some shims, possibly runtime patching the internals to allow tables and datasets to be used synonymously. One current shim is the as-dataset multimethod in proc.util, which could be leveraged more.

fs-tom commented 7 years ago

Integrated via proc.dataset ns. Tables can be passed to anything expecting an incanter dataset and they should "just work." Debating migrating this to spork for general re-use...