search

rstudio / learnr

Interactive Tutorials with R Markdown
https://pkgs.rstudio.com/learnr
Apache License 2.0
713 stars 240 forks source link

interactive maps with tmap & mapview display earlier chunks on shinyapps #529

Open andysouth opened 3 years ago

andysouth commented 3 years ago

System details

Output of sessioninfo::session_info()():

- Session info -------------------------------------------------------------------------------------
 setting  value                       
 version  R version 4.0.3 (2020-10-10)
 os       Windows 10 x64              
 system   x86_64, mingw32             
 ui       RStudio                     
 language (EN)                        
 collate  English_United Kingdom.1252 
 ctype    English_United Kingdom.1252 
 tz       Europe/London               
 date     2021-05-24                  

- Packages -----------------------------------------------------------------------------------------
 ! package           * version     date       lib source                            
   abind               1.4-5       2016-07-21 [1] CRAN (R 4.0.0)                    
   afrilearnr        * 0.0.0.9005  2021-05-24 [1] local                             
   askpass             1.1         2019-01-13 [1] CRAN (R 4.0.2)                    
   assertthat          0.2.1       2019-03-21 [1] CRAN (R 4.0.2)                    
   backports           1.2.0       2020-11-02 [1] CRAN (R 4.0.3)                    
   base64enc           0.1-3       2015-07-28 [1] CRAN (R 4.0.0)                    
   cachem              1.0.1       2021-01-21 [1] CRAN (R 4.0.3)                    
   callr               3.7.0       2021-04-20 [1] CRAN (R 4.0.5)                    
   checkmate           2.0.0       2020-02-06 [1] CRAN (R 4.0.2)                    
   class               7.3-17      2020-04-26 [1] CRAN (R 4.0.3)                    
   classInt            0.4-3       2020-04-07 [1] CRAN (R 4.0.2)                    
   cli                 2.5.0       2021-04-26 [1] CRAN (R 4.0.3)                    
   codetools           0.2-18      2020-11-04 [1] CRAN (R 4.0.3)                    
   colorspace          2.0-0       2020-11-11 [1] CRAN (R 4.0.3)                    
   crayon              1.4.1       2021-02-08 [1] CRAN (R 4.0.5)                    
   crosstalk           1.1.0.1     2020-03-13 [1] CRAN (R 4.0.2)                    
   curl                4.3         2019-12-02 [1] CRAN (R 4.0.2)                    
   DBI                 1.1.0       2019-12-15 [1] CRAN (R 4.0.2)                    
   desc                1.3.0       2021-03-05 [1] CRAN (R 4.0.5)                    
   devtools          * 2.4.0       2021-04-07 [1] CRAN (R 4.0.5)                    
   dichromat           2.0-0       2013-01-24 [1] CRAN (R 4.0.0)                    
   digest              0.6.27      2020-10-24 [1] CRAN (R 4.0.3)                    
   dplyr               1.0.5       2021-03-05 [1] CRAN (R 4.0.5)                    
   e1071               1.7-4       2020-10-14 [1] CRAN (R 4.0.3)                    
   ellipsis            0.3.1       2020-05-15 [1] CRAN (R 4.0.2)                    
   evaluate            0.14        2019-05-28 [1] CRAN (R 4.0.2)                    
   fansi               0.4.1       2020-01-08 [1] CRAN (R 4.0.2)                    
   fastmap             1.1.0       2021-01-25 [1] CRAN (R 4.0.3)                    
   fs                  1.5.0       2020-07-31 [1] CRAN (R 4.0.3)                    
   generics            0.1.0       2020-10-31 [1] CRAN (R 4.0.3)                    
   glue                1.4.2       2020-08-27 [1] CRAN (R 4.0.3)                    
   htmltools           0.5.1.9000  2021-02-02 [1] Github (rstudio/htmltools@e7f0393)
   htmlwidgets         1.5.3       2020-12-10 [1] CRAN (R 4.0.3)                    
   httpuv              1.5.5       2021-01-13 [1] CRAN (R 4.0.3)                    
   jsonlite            1.7.2       2020-12-09 [1] CRAN (R 4.0.3)                    
   KernSmooth          2.23-18     2020-10-29 [1] CRAN (R 4.0.3)                    
   knitr               1.31        2021-01-27 [1] CRAN (R 4.0.3)                    
   later               1.1.0.1     2020-06-05 [1] CRAN (R 4.0.2)                    
   lattice             0.20-41     2020-04-02 [1] CRAN (R 4.0.3)                    
   leafem              0.1.3       2020-07-26 [1] CRAN (R 4.0.3)                    
   leaflet             2.0.3       2019-11-16 [1] CRAN (R 4.0.2)                    
   leaflet.providers   1.9.0       2019-11-09 [1] CRAN (R 4.0.2)                    
   leafsync            0.1.0       2019-03-05 [1] CRAN (R 4.0.2)                    
 V learnr              0.10.1.9008 2021-05-24 [1] Github (rstudio/learnr@03a6db6)   
   lifecycle           1.0.0       2021-02-15 [1] CRAN (R 4.0.5)                    
   lwgeom              0.2-5       2020-06-12 [1] CRAN (R 4.0.2)                    
   magrittr            2.0.1       2020-11-17 [1] CRAN (R 4.0.3)                    
   mapview           * 2.9.3       2020-09-04 [1] Github (r-spatial/mapview@7cb18c3)
   markdown            1.1         2019-08-07 [1] CRAN (R 4.0.2)                    
   memoise             2.0.0       2021-01-26 [1] CRAN (R 4.0.5)                    
   mime                0.9         2020-02-04 [1] CRAN (R 4.0.0)                    
   munsell             0.5.0       2018-06-12 [1] CRAN (R 4.0.2)                    
   openssl             1.4.3       2020-09-18 [1] CRAN (R 4.0.3)                    
   pillar              1.6.0       2021-04-13 [1] CRAN (R 4.0.5)                    
   pkgbuild            1.2.0       2020-12-15 [1] CRAN (R 4.0.5)                    
   pkgconfig           2.0.3       2019-09-22 [1] CRAN (R 4.0.2)                    
   pkgload             1.2.1       2021-04-06 [1] CRAN (R 4.0.5)                    
   png                 0.1-7       2013-12-03 [1] CRAN (R 4.0.0)                    
   prettyunits         1.1.1       2020-01-24 [1] CRAN (R 4.0.2)                    
   processx            3.5.1       2021-04-04 [1] CRAN (R 4.0.5)                    
   promises            1.1.1       2020-06-09 [1] CRAN (R 4.0.2)                    
   ps                  1.4.0       2020-10-07 [1] CRAN (R 4.0.3)                    
   purrr               0.3.4       2020-04-17 [1] CRAN (R 4.0.2)                    
   R6                  2.5.0       2020-10-28 [1] CRAN (R 4.0.3)                    
   raster              3.4-5       2020-11-14 [1] CRAN (R 4.0.3)                    
   RColorBrewer        1.1-2       2014-12-07 [1] CRAN (R 4.0.0)                    
   Rcpp                1.0.6       2021-01-15 [1] CRAN (R 4.0.3)                    
   remotes             2.3.0       2021-04-01 [1] CRAN (R 4.0.5)                    
   renv                0.12.5      2021-01-09 [1] CRAN (R 4.0.3)                    
   rlang               0.4.10      2020-12-30 [1] CRAN (R 4.0.3)                    
   rmarkdown           2.6         2020-12-14 [1] CRAN (R 4.0.3)                    
   rprojroot           2.0.2       2020-11-15 [1] CRAN (R 4.0.3)                    
   rsconnect           0.8.16      2019-12-13 [1] CRAN (R 4.0.2)                    
   rstudioapi          0.13        2020-11-12 [1] CRAN (R 4.0.3)                    
   satellite           1.0.2       2019-12-09 [1] CRAN (R 4.0.2)                    
   scales              1.1.1       2020-05-11 [1] CRAN (R 4.0.2)                    
   sessioninfo         1.1.1       2018-11-05 [1] CRAN (R 4.0.2)                    
   sf                  0.9-8       2021-03-17 [1] CRAN (R 4.0.5)                    
   shiny               1.6.0       2021-01-25 [1] CRAN (R 4.0.3)                    
   sp                  1.4-4       2020-10-07 [1] CRAN (R 4.0.3)                    
   stars               0.5-2       2021-03-17 [1] CRAN (R 4.0.5)                    
   testthat            3.0.2       2021-02-14 [1] CRAN (R 4.0.5)                    
   tibble              3.1.1       2021-04-18 [1] CRAN (R 4.0.5)                    
   tidyselect          1.1.0       2020-05-11 [1] CRAN (R 4.0.2)                    
   tmap              * 3.3-1       2021-03-15 [1] CRAN (R 4.0.5)                    
   tmaptools           3.1-1       2021-01-19 [1] CRAN (R 4.0.5)                    
   units               0.6-7       2020-06-13 [1] CRAN (R 4.0.2)                    
   usethis           * 2.0.1       2021-02-10 [1] CRAN (R 4.0.5)                    
   utf8                1.1.4       2018-05-24 [1] CRAN (R 4.0.2)                    
   vctrs               0.3.5       2020-11-17 [1] CRAN (R 4.0.3)                    
   viridisLite         0.3.0       2018-02-01 [1] CRAN (R 4.0.2)                    
   webshot             0.5.2       2019-11-22 [1] CRAN (R 4.0.2)                    
   withr               2.4.1       2021-01-26 [1] CRAN (R 4.0.3)                    
   xfun                0.22        2021-03-11 [1] CRAN (R 4.0.5)                    
   XML                 3.99-0.5    2020-07-23 [1] CRAN (R 4.0.3)                    
   xtable              1.8-4       2019-04-21 [1] CRAN (R 4.0.2)                    
   yaml                2.2.1       2020-02-01 [1] CRAN (R 4.0.0)

Example application or steps to reproduce the problem

On shinyapps learnr chunks display interactive maps from earlier chunks that have been run.

Minimal example app Reproducible example code

Chunk1 makes an orange world map. Chunk2 makes a blue world map. Chunk3 makes an orange dot map. Chunk4 makes a blue dot map.

If user Runs chunks 1-4 in that order, they all display map1. If user runs chunk 3 then 4, both chunks display map3.

Happens on shinyapps but not locally.

Possibly related to issue fixed by RStudio in 2021-02 https://github.com/rstudio/learnr/issues/484 Causing us problems with our tutorials here : https://github.com/afrimapr/afrilearnr/issues/16

Thankyou!

gadenbuie commented 3 years ago

Thank you for posting the example app with session info. Unfortunately, I cannot reproduce the error you're seeing. In Chrome, Firefox and Safari on MacOS (Big Sur) the app returns the correct map regardless of the order in which the examples are run. What browser are you using? Have you tried reproducing the problem in other browsers on your machine? Does clicking "Start Over" to reset your progress in the tutorial resolve the issue?

anelda commented 3 years ago

Good morning @gadenbuie, I'm a collaborator of @andysouth and gets the same. I'm on Ubuntu 20.04.2 and got the same in Firefox and Chrome.

On closer inspection of https://andysouth.shinyapps.io/intro-to-spatial-r/ I wonder if it isn't because somewhere the tmap_mode("view") gets cached. That's why @gadenbuie couldn't replicate it on a first try, but I bet if you ran it again, you'll see the same thing we get.

The first maps are supposed to be displayed with tmap_mode("plot") and then later in the code (https://andysouth.shinyapps.io/intro-to-spatial-r/#section-g.-interactive-maps) we set tmap_mode("view") and it seems as if once you've run that block of code, it will repeatedly show the last map that was produced in an interactive mode. Even when you click on "Start over".

I added tmap_mode("plot") to all the code chunks in https://andysouth.shinyapps.io/intro-to-spatial-r/#section-e.-first-maps-with-tmap where I first start experiencing the problem and that fixes it. Now each code chunk produces the correct map (not interactive as we wanted).

Strangely though https://andysouth.shinyapps.io/intro-to-spatial-r/#section-f.-mapping-multiple-layers always renders the correct map without having to add tmap_mode("plot")

andysouth commented 3 years ago

Thanks @gadenbuie @anelda

I see this problem on Windows Chrome, and Firefox. It seems to be intermittent. Yesterday it was working on Firefox.

From the minimal example the shinyapps log messages associated with the problem are :

2021-05-25T08:14:24.672358+00:00 shinyapps[4177430]: Warning in selectChildren(jobs, timeout) : 2021-05-25T08:14:24.672360+00:00 shinyapps[4177430]: cannot wait for child 651 as it does not exist 2021-05-25T08:14:28.823369+00:00 shinyapps[4177430]: tmap mode set to interactive viewing 2021-05-25T08:14:29.536338+00:00 shinyapps[4177430]: Warning in selectChildren(jobs, timeout) : 2021-05-25T08:14:29.536356+00:00 shinyapps[4177430]: cannot wait for child 667 as it does not exist

Clicking 'Start over' for the minimal example does allow the latter plots to run correctly, but only if the earlier plots are not run beforehand.

andysouth commented 3 years ago

To note @gadenbuie that I've also just seen the same behaviour in a RStudio cloud project running one of our tutorials.

It similarly gave the message :

tmap mode set to interactive viewing Warning in selectChildren(jobs, timeout) : cannot wait for child 661 as it does not exist

gadenbuie commented 3 years ago

It turns out that the warning about non-existent children looks bad but is generally harmless, except that in this case it pointed me in the direction of the solution. That warning only shows up when using a forked evaluator and usually means that the exercise result returned cleanly and learnr didn't have to kill the process.

The forked evaluator is the default evaluator on Linux, i.e. shinyapps.io. You can use a forked evaluator setup locally by adding the following to your setup chunk:

options(tutorial.exercise.evaluator = learnr:::forked_evaluator)

Once I turn on the forked evaluator, I can reproduce the issue described here reliably, even locally on MacOS. It took me a bit to track down the root cause, but it ultimately comes down to the forked evaluator model: with a forked evaluator, each exercise submission is run in a fork of the current R process.

The forked evaluator uses parallel::mcparallel() to spin off a new R process, and this function by default sets the random seed of the new R process to the same initial random number generator (RNG) state as the current R session. This means that every exercise is evaluated in an R process with the same initial RNG seed.

How does this cause problems for tmap? With tmap_mode("view"), the interactive map is provided in as an htmlwidget. The htmlwidget involves some HTML and some data which are connected via an id argument that is randomly generated in R, e.g. <div id="htmlwidget-ba32e0541469b5ed4906" ... >. When a second id is generated, it also runs in a forked R process with the same seed as the previous evaluation, so the same id is created again. The data changes between runs, but ids are expected to be unique in HTML so the JavaScript that hydrates the interactive map uses the data from a previously created map.

Here's the final solution: for each interactive map, set the seed of the htmlwidget ids to a different number in an exercise setup chunk:

```{r tmap-test-orange-setup}
htmlwidgets::setWidgetIdSeed(1)
library(tmap)
tmap_mode("view")
data("World")

tm_shape(World) +
    tm_polygons("HPI")
htmlwidgets::setWidgetIdSeed(2)
library(tmap)
tmap_mode("view")
data("World")

tm_shape(World) +
    tm_polygons("HPI", palette='Blues')



The `-setup` chunk is always executed before the user code whenever the exercise is run by the user. You can also use this chunk to ensure that a particular plot always render in `plot` or `view` mode, by adding `tmap_mode("plot")` or `tmap_mode("view")` in the exercise's `-setup` chunk.

Please give this idea a try and let me know if it works for you!
andysouth commented 3 years ago

Many thanks @gadenbuie for the detective work - a tricky operating system dependent one - and comprehensive explanation. I'll try out the solution and report back.

andysouth commented 3 years ago

Thanks @gadenbuie this solution does work for us.

Implemented here : https://andysouth.shinyapps.io/intro-to-spatial-r/#section-e.-first-maps-with-tmap

(One side note that setting the forked_evaluator to generate the issue locally did not work for me locally on Windows, with that pressing Run on a chunk gave : Error in : 'mcparallel' is not an exported object from 'namespace:parallel')

kcarnold commented 2 years ago

This issue would apply to anything that uses htmlwidgets in learnr then, not just maps, right? If so, then manual patches like this will be needed for other types of displays too.

A few potential solutions come to mind:

  1. Turn off RNG seed management in mcparallel. But the behavior of mc.set.seed = FALSE in mcparallel might depend on whether the setup chunk calls set.seed...
  2. Set the RNG seed based on hashing the chunk name (guaranteed to be unique if it knits successfully)
  3. Generate n random numbers before starting the evaluation where n is the number of cells above the current one.
  4. Add a special hook for htmlwidgets::setWidgetIdSeed in the evaluator.

More research is needed for option 1. Option 2 seems most straightforward, if it's possible to get the name of the currently executing chunk.

gadenbuie commented 2 years ago

Thanks @kcarnold for bumping this issue and for brainstorming some solutions!

2. Set the RNG seed based on hashing the chunk name (guaranteed to be unique if it knits successfully)

I like this idea and I think it'd be reasonable for learnr to do this in general, e.g. the exercise evaluator could easily hash the exercise label (i.e. the chunk name) into an integer value before executing the exercise. Randomness is already a bit tricky in learnr and this would make outputs consistent between both runs and across environments. We might want to add an "escape hatch" (an exercise-level option) for authors who really really really want random values to be generated between exercise runs.

I'm envisioning an exercise.set_seed option that's TRUE by default and could be set to FALSE (don't set seed) or to a specific integer value (exercise.set_seed = 42).

gadenbuie commented 2 years ago

Unfortunately, we can't fix this with seed setting/management alone and we'll need a change in htmlwidgets or in our forked evaluator to fix this. The crux of the issue looks more or less like this:

  1. At some point early in the tutorial lifecycle, we create the first htmlwidget for the app. This calls htmlwidget:::createWidgetId() to create an ID for this widget. Unfortunately, internally createWidgetId() forks the global RNG state and starts using it internally.
  2. When we evaluate the user's submitted exercise code, by default on Linux (like shinyapps), we use a forked R process that forks the current R process into a new session. The current internal state of htmlwidgets' widget id RNG goes along for the ride, but changes to this state aren't returned back to the parent process.
  3. The next submission evaluation restarts the same process and the only way out is to call htmlwidgets::setWidgetIdSeed() before each evaluation.

I submitted a PR upstream in htmlwidgets that would change this behavior so that htmlwidgets would only manage it's own ID seed when the user calls setWidgetIdSeed(). I think this is reasonable but might have broader implications I'm not aware of.

The other option would be for learnr to switch from forked processes to using callr, futures, or promises, but for that to be an effective solution for this particular problem, we'd need to completely evaluate the exercise start-to-finish in a background process, which would come with at least few drawbacks.