search

rstudio / promises

A promise library for R
https://rstudio.github.io/promises
Other
198 stars 19 forks source link

Memory usage in promises #56

Closed LuShuYangMing closed 4 years ago

LuShuYangMing commented 4 years ago

Sorry this is more of a question rather than an issue, but I could not get a definitive answer on other channels.

I am major in R and shiny, but new to the world of async programming, I have two question about promises package:

  1. Does the shiny-server free support promises package or just shiny-server Pro support promises package?
  2. Under the premise of loading a large file about 2.5GB for calculating, the final result size is very small. Suppose my server memory is 4GB, there are two website (or many) users access my shiny app, does the promises package load the file in queue or parallel? It's extremely fortunate if loading file in queue, but what should I do when it loading in parallel?

Thank you very much!

wch commented 4 years ago

For (1), promises work on both Shiny Server Open Source and Pro versions.

For (2), the answer is that whether it loads in serial or parallel really depends on how you write your code.

jcheng5 commented 4 years ago

If the calculating you're doing on the 2.5GB is the same every time (i.e. not affected by user input) then it'd be a huge performance/efficiency win to do that calculation beforehand, outside of Shiny, and persist the result to disk. Or even if a portion of the calculation is affected by user input, maybe you can still do precalculation/aggregation/summarization that slims down your input ahead of time (can you drop columns/rows that you know you'll never use?).

If you really need to load a ton of data, and you have two/many users accessing your site, and your server memory is barely larger than your data, I think those are three things that don't go together--you'll have to choose one of those three to change.