techascent / tech.ml.dataset

A Clojure high performance data processing system
Eclipse Public License 1.0
680 stars 35 forks source link

tranduce-compatible rf functions for parquet ds-seq->parquet and arrow/ds-seq->arrow pathways. #354

Open cnuernber opened 1 year ago

cnuernber commented 1 year ago

Users should not have to provide a seq or seq-like things to write sequences of datasets to either arrow or parquet. A transduce-compatible rf function is more ideal for these sort of 'sink' use cases.