search

OHDSI / Andromeda

AsynchroNous Disk-based Representation of MassivE DAta: An R package aimed at replacing ff for storing large data objects.
https://ohdsi.github.io/Andromeda/
11 stars 10 forks source link

Issue 53 #56

Closed schuemie closed 1 year ago

schuemie commented 1 year ago

This uses as_record_batch_reader(), thus avoiding the need to copy a dplyr query to an Andromeda object. Addresses issue #53 .

Note: this PR also includes the changes of PR https://github.com/OHDSI/Andromeda/pull/55 (allowing me to test both locally).

codecov[bot] commented 1 year ago

Codecov Report

Merging #56 (20d5799) into develop (09d73a0) will decrease coverage by 0.18%. The diff coverage is 100.00%.

:exclamation: Current head 20d5799 differs from pull request most recent head 68f9dbe. Consider uploading reports for the commit 68f9dbe to get more accurate results

@@             Coverage Diff             @@
##           develop      #56      +/-   ##
===========================================
- Coverage    82.44%   82.26%   -0.18%     
===========================================
  Files            3        3              
  Lines          262      265       +3     
===========================================
+ Hits           216      218       +2     
- Misses          46       47       +1
Impacted Files Coverage Δ
R/LoadingSaving.R 92.53% <100.00%> (ø)
R/Operations.R 88.09% <100.00%> (-0.80%) :arrow_down:

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

schuemie commented 1 year ago

Maybe not such a good idea. I'm running out of memory for large models

schuemie commented 1 year ago

Damn, also running out of memory with the original code...

ablack3 commented 1 year ago

@schuemie Still interested in merging this PR?

schuemie commented 1 year ago

Since we're putting the arrow implementation on ice for now this PR doesn't make much sense. I'll close it