fstpackage / fsttable

An interface to fast on-disk data tables stored with the fst format
GNU Affero General Public License v3.0
27 stars 4 forks source link

Rough implementation of row_filter & slice_map #28

Closed martinblostein closed 6 years ago

martinblostein commented 6 years ago

A naive implementation of the filter & slicemap, entirely in R. Nothing here has been optimized for grace or efficiency, but I think it more or less captures the concepts in #22 and #25.

Here's a demo/test:

tmp <- tempfile()
fst::write_fst(data.table::data.table(a = 1:100), tmp)
ft <- fsttable::fst_table(tmp)

ft[c(1,9,2,5,10)] # prints original rows 1,9,2,5,10

ft[10:30] # prints original rows 10-30
ft[10:30][c(1,3)] # prints original rows 10 & 12
ft[10:30][20:10] # prints original rows 29-19
ft[10:30][20:10][c(2,4)] # prints original rows 28 & 26

ft # prints original rows 1-5 -- 96-100
ft[40:100] # print original rows 40-44 -- 96-100
ft[100:20] # prints original rows 100-96 -- 24-20
ft[c(100:20, 1)] # prints original rows 100-96 -- 23-20,1
MarcusKlik commented 6 years ago

Hi @martinblostein, great, thanks, we can do the further optimization later!