Integrate JS Capture into R Shiny

[bastianilso]

Now that we have a prototype implementation of a JS event capture system, we should investigate how to integrate it into a basic R shiny application, for example the R shiny application found in the brush_whack folder. There are a few things to consider:

• Loading the JS implementation. This should be possible inside the ui function of the R Shiny app.
• Currently the JS implementation requires a "Download CSV" button, which generates a CSV file of the captured events. We can continue to have this, but it might need implementation from R Shiny UI itself, within the ui function. In the future, we may want a more sophisticated way (invisible to the user in the experiment), in which we can access the data.
• The other elements from the HTML page should not be necessarily, but if they are please let me know and we can discuss the technical design.

[bilal-62210]

@bastianilso I am still working on this, currently i can capture all events for all Elements except for the plotlyoutput. Can you help me to find the solution for this element please ?

[bastianilso]

@bilal-62210 what did you try?

[bilal-62210]

@bastianilso i tried to change 'plotlyoutput' to 'plotoutput', in this case i managed to capture the id but the chart is not showing on the app, so this it didn't work. I tried also to give to the plotlyoutput an 'outputid' instead of the basic 'id' and this didn't work too. I tried also to include the plotlyoutput in a "

to call the div id when the mouse hovers over the plotlyoutput and it didn't work. So i think the issue is that the 'plotlyouput' is reactive because for a basic 'plotouput' it works.

[bastianilso]

@bilal-62210 did you try to use a jQuery style of binding your log function instead?

See my suggestion here: https://github.com/med-material/r-shiny-js-data-capture/issues/1#issuecomment-1257627352

Think about this: How do you know when the HTML DOM changes? If you are subscribing all elements at the beginning to your logging, but then something in the HTML changes, how do you ensure the new elements are also subscribed to your logging? Maybe this is the problem?

take a look at: https://stackoverflow.com/questions/15657686/jquery-event-detect-changes-to-the-html-text-of-a-div

[bilal-62210]

Hi @bastianilso This is what i did exactly for the plotlyoutput id :

• I tried to put the plotlyoutput in a Div to get the Div id
• I tried to use jquery event to detect changes on the plotlyoutput
• I tried to do a function which give a static id to the plotlyoutput
• I tried to use hover event to get the id

all these tests did not work. As i said yesterday, if we can't find the solution to get the plotlyoutput id we can put the value to null for the "element" field on the csv file when there is a mouse event on a plotlyoutput(but we still have the screen-x and y values).

[bastianilso]

Hi @bilal-62210 thanks for the report and for trying these.

• it is difficult for me to follow what you did, if you dont provide more details. for example, what does "use jquery event" refer to? A code example would be great. The reason I would like you to be more elaborate with how you did your tests, is that we need to eliminate all possible problems, until we can identify the one problem.

Here are my suggestions to you to try next to solve this:

1. Plotly uses SVG for its output in many cases, and in other cases WebGL. I manipulated your HTML file to include a some basic SVG elements. Please try if your JS capture system can detect hovering over the SVG element as a whole and the individual SVG elements (ellipses). web_page2-with-svg.zip

2. Try and see whether you can integrate this Codepen into your HTML file and use the JS capture system on the plot (this uses Plotly.js outside R Shiny, this way we can test it in isolation): https://codepen.io/mmakrzem/pen/mdOpWLd

3. Try to see if you can use your JS data capture system on the visualization produced by Aldryck.

[bastianilso]

@bilal-62210 I posted a question in the plotly community forum. We have one reply already, please have a look.

https://community.plotly.com/t/detecting-mouse-events-within-plotly-plots-from-external-js-script/68676

[bastianilso]

Other suggestions to try out:

4. Try loading the data capture directly from the browser inspector, by manipulating the HTML code and see if your function can be called this way.

[bastianilso]

Now that we have elementFromPoint, we don't have to rely on this other issue solely, so I'll resolve this.