search

rethomics / damr

Read TriKinetics' DAM data in R
http://rethomics.github.io
6 stars 7 forks source link

Possible to use fread for load_dam? #14

Closed brelwill closed 6 years ago

brelwill commented 6 years ago

I'm just wondering if it's possible to modify load_dam to use the fread function? With large raw data files like the ones were using, it seems to only use one core when processing the data. Not sure if this is compatible with what you've written, but I thought it was worth an ask.

qgeissmann commented 6 years ago

It is definitely possible and probably good. Indeed, we avoid dependency on readr. At first, I implemented as much as possible using the resources from the tidyverse, mainly to learn. In the end, I reimplemented most of it in data.table for consistency. It would definitely be good to make a last push for it. (https://github.com/rethomics/damr/blob/master/R/read-dam-file.R, line 47). Are you confident the bottleneck is here (see profiling http://adv-r.had.co.nz/Profiling.html#measure-perf)?

qgeissmann commented 6 years ago

@brelwill finally got a bit of time to ake this commit. Let me know if it helps causes any unforeseen trouble

brelwill commented 6 years ago

@qgeissmann thank you so much for doing this despite not having the profile! Hopefully this solves the issue!