search

tk3369 / SASLib.jl

Julia library for reading SAS7BDAT data sets
Other
34 stars 7 forks source link

Is it possible to jump ahead and not read from the beginning? #35

Open xiaodaigh opened 6 years ago

xiaodaigh commented 6 years ago

I note the README has this passage

Note that there is no facility at the moment to jump and read a subset of rows. Currently, SASLib always read from the beginning.

But is it technically possible to jump ahead and read from a certain row? I think it's related to #5

tk3369 commented 6 years ago

That's right. I think it's possible but it hasn't been a priority so far. Do you have a use case other than just making read-performance awesome?

xiaodaigh commented 6 years ago

I want to utilise multi-processing so that each process can process one chunk of data.

tbeason commented 3 years ago

Any new thoughts on this? Could the logic from CSV.jl, which also has native multi-threading now, be helpful here?

calebwin commented 2 years ago

Even if the reading logic could be modified so that it still scans the whole file but it only allocates memory for the given range, that would be really awesome!