Event Handler Metrics Collection

[michaelhogersnplm]

When running R Shiny applications in production it would be extremely valuable if it is possible to easily collect the performance of all event handlers. When applications start using dozens (or perhaps hundreds) of observe(Events) and (event)Reactives it becomes hard to track how all these different elements perform.

I propose that, if event handlers are assigned to a variable as below:

example <- observeEvent(input$trigger, { interestingFunction() })

example may then be assigned performance information on every request by input$trigger. Perhaps the simplest and most interesting piece of information would be the time it takes to finish handlerExpr. I understand that when an event handler is triggered often that performance information collection may introduce a performance decrease, therefore we could make this feature opt-in only.

Furthermore, the performance data does not have to be stored in the example variable, but might instead be written to a csv file (for example). Writing performance data to a csv would easily allow developers to write to another (persistent) data storage solution of their choice using session$onSessionEnded().

If we structure this data collection well, it may be quite trivial to create a library or Shiny function that structures the collected data and shows a performance overview as you typically see when monitoring APIs.

Perhaps this feature already exists, if not, how much work would be involved in creating such performance collection? I would be happy giving it a first shot if I could get some pointers on crucial pieces of the Shiny code base.

Best regards,

Michael Hogers

[schloerke]

Hi @michaelhogersnplm ,

Thank you for your patience. Sorry I did not see this sooner.

I believe reactlog would solve this situation. https://rstudio.github.io/reactlog/

reactlog allows insight into every reactive calculation that shiny executes. While it is typically used for reactive dependency debugging, it could also be used for measuring the execution time. (Currently no memory usage like how profviz does.)

In the gif on the landing page, you can see the performance time of each reactive and output. This shows that the information is in the data.

To enable reactlog, set options(shiny.reactlog = TRUE) before running your app. As you stated above, it should be considered a memory leak and should not be enabled on long performing R sessions.

To access the raw reactlog data, call shiny::reactlog(). This provides all the information that is processed and displayed within the reactlog UI. If you're looking to build a performance style dashboard, it would be great if it could ingest this style of data. (I am open to the idea of having reactlog eventually serve multiple dashboards!)

To display the reactlog UI, call shiny::reactlogShow(). This can also be done while running a shiny application with cmd + fn + F3 on macOS or ctrl + fn + F3 on windows.

[michaelhogersnplm]

Hi @schloerke,

Thank you for the extra information. That is exactly what I was looking for! It has the labels and timing data, great! I would need to see how much time is needed to build a useful performance style dashboard.

I assume it would be most beneficial for Shiny developers to add a single statement to their apps and write these logs into a database or perhaps some binary file storage solution. A second step would be to create a package that picks up that data structure from the relevant database and turn that data source into the basis of an entire performance dashboard. I may come back on a solution for this later

[schloerke]

Going to update this issue with relevant information from an email thread.

---

Blue print: Make all rLog events in shiny's graph.R code be registrable.

Currently there are three hard coded systems:

• Rlog
• MessageLogger
• domain$reactlog() in appendEntry function

The rLog object could be created from a method dispatcher. Then the RLog and MessageLogger classes could register all of their corresponding events to the reactlog method dispatcher

Browser showcase mode information is unfortunately mislabeled as 'domain$reactlog()': https://github.com/rstudio/shiny/blob/7aa3a243ba7ee33c89060a52cb7518e892421ecf/R/graph.R#L157 . This very small function should also register to the dispatched appendEntry event.

Motivation: By doing this registration change, we could expose this registration system and you would be able to tap into the stream of events. This would also allow for you to organize / store your information how you would like (ex: forget when session is closed or redirect to a database)