albertosantini / node-rio

Integration with Rserve, a TCP/IP server for R framework
https://github.com/albertosantini/node-conpa
MIT License
176 stars 35 forks source link

Guidance on how to use the package #34

Closed debsush closed 8 years ago

debsush commented 8 years ago

Hi,

This is not an issue but rather a query on how to use the package to call R functions from within Javascript

I have a shinyapp where I am using a JS charting library. I am computing the data in R and passing it to the Javascript using shinyjs package in R. This works great. Now as my data size increase or users from remote geographies try to access my app, I am having to transfer huge datasets from my server.R to the client-side javascript library in ui.R and network bandwidth is slowing my app down. I was wondering if I could transfer the entire computation to the client side by enabling the execution of the Rscript functions within JS. Then I can pass the computed data (which I think would now reside on the client-side) directly to the charting library and enable me to bypass the transfer of heavy data between server and the client. Let me know if I am making sense so far or I am conceptually inaccurate (which can well be the case given my limited knowledge in this field).

So, if the above theory makes sense, I would like to run the R functions from a particular package within JS.

Current Scenario:

  1. When app loads, the R script pulls an initial data set and passes it to the JS charting library using shinyJS. This is when the initial transfer of data between server and client takes place and this cannot be avoided.

Say the initial data is a data frame DF. DFJson <- jsonlite::toJSON(DF) Now pass this json to the JS charting library

For subsequent calculations based on users input, all I need to do is use the same json which I have passed onto the JS and run some R functions on it and then pass it back to the charting library. In the current scenario, I am computing the revised data within R and passing it back to JS DF<-newDF(DF) DFJson <- jsonlite::toJSON(DF) newDF<-function(data){

convert the data frame to xts

data<-as.xts(data) SMA<-quantmod::SMA(data,20) data<-merge(data,SMA)

convert it back to data frame

data<-as.data.frame(Date=index(data),coredata(data)) return(data) } Now this new data with an additional column is being transferred to the JS library. This time again the transfer between server and client is subject to network bandwidth. I would like to bypass these subsequent transfers by computing the new data within JS using the R function newDF that I have defined. Please note that DF has dependency on the R library quantmod.

Do I make sense at all. If yes, can I perform the above strategy using your package. The final objective is to use the strategy within a shinyapp Appreciate a small example.
Regards SD

albertosantini commented 8 years ago

Thanks for considering rio.

TLDR; rio doesn't resolve your use case.

rio is a Node.js module, intending it is a module running on server side, allowing to Node.js apps to communicate though a Rserve instance without any native bindings, because Rserve protocol is has been ported to js.

I don't see how opening a second channel of communication, adding complexity to the architecture, might reduce data size transfer: it would remain the same passed in shiny approach. rio, as I said, runs on server side.

Computational load is not on front end, but it is executed via Rserve on the back end.

To reduce data size transfer you need to focus on data model slicing: for instance, general approaches like memoizing or a sort of "pagination" may help (and all this is viable inside shiny).

About the distribution of computation on front end, the only solution is reducing the complexity and porting the algorithm to js, as I did with quadratic programming (quadprog module). Generally speaking, when there is an IO resource, intending a database or a cpu, it is not recommended a direct access to that resource using a two layer approach (front end communicating directly with layer data or math engine), in terms of maintanance and scalability.

Indeed it would be better to move that task to server side, but we would return to the square number one.

Hope that helps.

As you said it is not an issue: closing.

debsush commented 8 years ago

Thank you for the insight

On Wed, Jul 20, 2016 at 6:34 PM, Alberto Santini notifications@github.com wrote:

Closed #34 https://github.com/albertosantini/node-rio/issues/34.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/albertosantini/node-rio/issues/34#event-729238941, or mute the thread https://github.com/notifications/unsubscribe-auth/AOYQSUubUEe4RrEBHlqupnAJP4uCHBQ7ks5qXk4IgaJpZM4JQmN2 .