Defining handler functions for control elements

[sa-lee]

I've been playing around with shiny to try and get some more low level functions to work - which is proving surprisingly difficult, the official way of getting these extensions in is via the shinyjs package via the onevent function but these give you surprisingly little control over the event but can't assign the output to an object on the server side. The alternative is to use Shiny.onInputChange with javascript, here's a working demo with a proper drag (rather than a brush):

library(shiny)
library(ggplot2)

click_handler_js <- function() {
  "$(function(){ 
    $(pl).click(function(e) {
      var output = {};
      output.coord_x = e.clientX;
      output.coord_y = e.clientY;
      output.width = $(pl).width();
      output.height = $(pl).height();
      Shiny.onInputChange('click', output)
    });
  }) 
 "
}

drag_handler_js <- function() {
  "$(function(){
    var is_drawing = false;
    var output = {};
    output.width = $(pl).width();
    output.height = $(pl).height();
    $(pl).on('dragstart', function(e) { e.preventDefault()});
    $(pl).mousedown(function(e) {
      is_drawing = true;
      output.start_x = e.clientX;
      output.start_y = e.clientY;
    }).mousemove(function(e) {
      if (is_drawing) {
        console.log('moving')
      }
    }).mouseup(function(e) {
      is_drawing = false;
      output.end_x = e.clientX;
      output.end_y = e.clientY
      console.log('not moving');
      console.log(output);
      Shiny.onInputChange('drag', output);
    });
  })"
}

ui <- basicPage(
  tags$script(HTML(click_handler_js())),
  tags$script(HTML(drag_handler_js())),
  plotOutput("pl"),
  tableOutput("clicked"),
  verbatimTextOutput("dragged")
)

# an example of projecting onto data coordinates, would only work for cts vars
click_handler_r <- function(input) {
  coord_x <- input$click$coord_x
  coord_y <- input$click$coord_y
  width <- input$click$width
  height <- input$click$height
  # this doesnt take into account plot margins but will do for now
  data.frame(hp = coord_x*(diff(range(mtcars$hp))/width), 
             wt = (height - coord_y)*(diff(range(mtcars$wt))/height))
}

server <- function(input, output, session) {
  plot <- ggplot(mtcars, aes(x = hp, y = wt)) + geom_point()
  output$pl <- renderPlot(plot)
  output$clicked <- renderTable(click_handler_r(input))
  output$dragged <- renderPrint(input$drag)
}

shinyApp(ui, server)

Again these handlers are specific to how shiny does things - it also seems kinda clunky to write out javascript as inline text and having the user write javascript is something we should avoid.

[lawremi]

As long as we can give the user all the important events, then this will be good enough for now. Obviously, none of this should even be dependent on Shiny, much less Javascript.

[dicook]

What about crosstalk? Carson went that route with plotly, I think.

Sent from my iPhone

On 11 Jul 2018, at 2:50 pm, Michael Lawrence notifications@github.com wrote:

As long as we can give the user all the important events, then this will be good enough for now. Obviously, none of this should even be dependent on Shiny, much less Javascript.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or mute the thread.

[sa-lee]

I think crosstalk will still require some javascript at the end of the day - as we want lower-level handling of events than it provides (which I think is a selection brush and filter).

Since the event handling happens at the device level and I think it would be nice to have a solution that works with both native graphics devices or a browser

[lawremi]

If we can get something working then it won't be hard to convince them to modify Shiny to make it cleaner. We need a good proof of concept though first.

[lawremi]

Btw, if we're now hacking our own brush, are we sure it is going to be performant enough, even for this prototype? Do we need to draw the brush using e.g. SVG and JS (presumably this is what Shiny does)? ggplot2/grid might be too slow. Maybe we could plot the rectangle in a separate graphics device but overlay it on the plot using CSS? Just throwing some stuff out there.

There's also the option of using Deepayan's Qt-based graphics device. It would be relatively easy to overlay a Qt scene with a brush or whatever. Depending on Qt is not ideal but it's just a prototype.

Let's wait and see but I'm guessing we'll need to do something about performance.

[sa-lee]

Yep that is what shiny does - I've got this kinda working with grid but it is indeed very slow.

I guess in my mind there's two ways that this is currently done -

1. have the data processed in R handle everything else with JS (including the drawing of the graphic with javascript), i.e. this is how ggvis/plotly approaches things

2. have R process the data and draw the graphic, render it to an SVG or HTML5 canvas and have JS handle interactivity. For example, the gridSVG package, and really how shiny does it too.

In my mind we are going for something like 2?

[lawremi]

Yea, something like 2, for now. Reimplementing ggplot2 would be too much work. Plotting the cues on the plot, like the brush rectangle, will be relatively easy in JS. But updates in response to brushing will still be slow, so we can't expect very good performance from the prototype.

We'll have to move to 1 fairly quickly after the proof of concept.

[sa-lee]

Ok so here's how shiny creates a brush (via modifying the DOM):

create a div corresponding to the plot (this has attributes containing the brush id):

• within that div place an img tag which is just the rendered ggplot object as a png
• if a brush event happens a new div with id = brush id, and the drawing is done by modifying the div to have top and left , height and width attributes - resulting in a rectangle being drawn
• there's a bit of manipulation done with the image so that offsets are in place (see https://github.com/rstudio/shiny/blob/389463aea512518ede85ed14b1cb05437fb3ffc5/srcjs/output_binding_image.js#L291)

[lawremi]

Ok, interesting. I think we'll want something more than a brush though, so maybe <canvas> or <svg>.

[sa-lee]

Turns out the canvas package (https://www.rforge.net/canvas/index.html) doesn't work :(