search

subugoe / hoad

Deprecated: Please check https://github.com/subugoe/hoaddash
https://github.com/subugoe/hoaddash
GNU Affero General Public License v3.0
15 stars 4 forks source link

reduce RAM utilisation in shiny app #230

Open maxheld83 opened 4 years ago

maxheld83 commented 4 years ago

now that we're on GCP (instead of shinyapps.io) as per #25, we're getting a clearer view into the resource utilisation of our shiny app.

It ain't pretty.

It (apparently, pending further testing) won't be stable below 4GiB of RAM.

I am guessing this is mostly the pretty huge data object being loaded into RAM, which is just R design.

Still, I should profile this to be sure that's what's going on.

Long-term, to make the app easier on the RAM, it would be good to offload most of this work to Google BigQuery or a separate R (plumber) API.

(We're not using much CPU, as expected).

maxheld83 commented 4 years ago

nope it's not the data object, or at least not just them.

pryr::object_size(hoad::hybrid_publications(), hoad::unpaywall_df())

yields 56.9 MB which is not nothing, but doesn't eat up 4GiB of RAM.

Will need to investigate further, later.