quarto-dev / quarto-cli

Open-source scientific and technical publishing system built on Pandoc.
https://quarto.org
Other
3.76k stars 309 forks source link

Interactive Plotly-R Plot Not Rendering in HTML Book #6128

Closed IULibScholComm closed 1 year ago

IULibScholComm commented 1 year ago

Bug description

Dear Quarto Team, I am writing on behalf of an author whose book I am publishing in Quarto. His R-Plotly plot--two line plots with an interactive slider--is not being rendered in the HTML version of the Quarto book (though it is rendered in the Quarto book's PDF, DOCX, and EPUB versions); instead, the HTML just stops--the plot and all the text after it do not get rendered.

May I ask for your help with solving this? (Please note: I am a beginner with software, programming, etc.)

I believe the R code is error-free (it runs fine in RStudio); my hunch however is that the data produced by the r code may be too large. If so, then I would also welcome advice on how to fix this. Thank you! Adam

Steps to reproduce

https://iulibscholcomm.github.io/through-the-looking-glass/why-sex.html https://github.com/IULibScholComm/through-the-looking-glass

```{r}
#| label: fig-clonal
#| fig-cap: "Results from a simulation study in which a single clonal individual was introduced into a sexual population at generation 1000 (Lively 2009). The sexual population was initiated at carrying capacity: $K_{sex} = 10000$. Note that the asexual lineage replaces the sexual population in about 25 generations. The asexual population then attains a new carrying capacity at $K_{asex} = 20000$ individuals. The frequency of males in the sexual population was assumed to be 1/2. Annual reproduction, with non-overlapping generations, was also assumed."
#| fig-alt: "Line plot of clonal invasion dynamics"
#| results: markup
#| code-summary: R code for clonal invasion dynamics simulation visualization
#| tidy: true
#| cache: true
#| cache-lazy: false

library(plotly)
library(ggthemes)
library(dplyr)

a = 0.0001  #a is a constant that gives the sensitivity to total population density
d = 1.0 # d is the death rate.  Here I set d=1, meaning an annual species.
b = 3.0 # b is the number of offspring prouduced by a single female (sexual or asexual)
c = 0  # c is a constant that gives the sensitivity of the death rate to density.  Here set = 0.
#s = 1/2 # s is the frequency of males in the sexual subpopulation.  (1-s) gives the freq of females in sexual popl
#f is freq of females

Fig12 <- data.frame("f" = 0, "Generation" = 0, "Asexuals" = 0, "Sexuals" = 0)

for(f in c(0.5, 0.55)){
# anayltical solutions: carrying capacities for sexual and asexuals are set by the parameters given above,
# following Lively (2009) J Evol Biol. doi: 10.1111/j.1420-9101.2009.01824.x
Ksex = (f*b - d)/((f)*a + c) #solution for carrying capacity of sexual population
Kasex = (b - d)/(a + c) #solution for carrying capacity of asexual population

# intitial conditions.  Sex initiated at Ksex.  Asex at 0.
Sex = Ksex 
Asex = 0

Gasex = 100 #generation at which a single asexual female is introduced

T = 200 # T is the number of time steps in addition to time step 0
time = c(1:T)  #sets up do loop for i = 1 to T

outSex = vector()  # outSex vector
outSex[1]=Sex[1]

outAsex = vector()  # outAsex  vector
outAsex[1]=Asex[1]

for(i in 1:T)

{
  outSex[i] = Sex[i]
  Sex[i+1] = Sex[i] - Sex[i]*(d + c*(Sex[i]+Asex[i])) + Sex[i]*(f)*(b - a*(Sex[i]+Asex[i]))

  if(i == Gasex)
    {
    Asex[i] = Asex[i] + 1
    }

  outAsex[i] = Asex[i]
  Asex[i+1] = Asex[i] - Asex[i]*d + Asex[i]*(b - a*(Sex[i]+Asex[i]))
Fig12 <- add_row(Fig12, "f" = f, "Generation" = i, "Asexuals" = Asex[i], "Sexuals" = Sex[i])
}}

Fig12 <- filter(Fig12, Generation > 0)
library(tidyr)
#Fig12 <- pivot_longer(Fig12, cols = Asexuals:Sexuals, names_to = "Mode", values_to = "Number")
Fig12$visible = rep(FALSE, length(Fig12$Generation))

curtstheme <- theme_tufte() + theme(
  axis.title.x = element_text(size=14,face="bold", vjust = -1),
  axis.title.y = element_text(size=14,face="bold", vjust = -1), 
  axis.text.x = element_text(face="bold", size=14), 
  axis.text.y = element_text(face="bold", size=14), 
  plot.title = element_text(face = "bold", size = 24, hjust = 0.5),
  plot.subtitle= element_text(size = 14, hjust = 0.5),
  plot.caption = element_text(size = 12, hjust = 0.5),
  axis.line = element_line(size = 1),
  axis.ticks = element_line(size = 1),
  axis.ticks.length = unit(0.25, "cm"))

points <- ggplot(Fig12) + geom_point(aes(x = Generation, y = Sexuals, frame = f), color = "#FF4242") + geom_point(aes(x = Generation, y = Asexuals, frame = f), color = "#235FA4") +
  scale_x_continuous(name="GENERATION") + 
  scale_y_continuous(name="NUMBER\n", limits = c(0, 21500)) +
  scale_color_manual(values = c(Asexuals = "#235FA4", Sexuals = "#FF4242"))  +
  annotate(geom = "text", x= 25, y = 1000, label = "Asexuals", color = "#235FA4", size = 6) +
  annotate(geom = "text", x= 190, y = 1000, label = "Sexuals", color = "#FF4242", size = 6) +
  theme(aspect.ratio = 1/1.5) + curtstheme +  theme(legend.position = "none")

ggplotly(points)


### Expected behavior

A line plot with an interactive slider + the remaining content in the qmd file should be rendered in the HTML book

### Actual behavior

Neither the line plot + slider, nor the content that follows it, are rendered in the HTML book

### Your environment

- IDE: VS Code 1.79.2
- OS: Windows 10 Enterprise

### Quarto check output

[>] Checking versions of quarto binary dependencies...
      Pandoc version 3.1.1: OK
      Dart Sass version 1.55.0: OK
[>] Checking versions of quarto dependencies......OK
[>] Checking Quarto installation......OK
      Version: 1.3.361
      Path: C:\Users\amazel\AppData\Local\Programs\Quarto\bin
      CodePage: 1252

[>] Checking basic markdown render....OK

ERROR: The file cannot be accessed by the system. (os error 1920), stat 'C:\Users\amazel\AppData\Local\Microsoft\WindowsApps\python3.10.exe'

Error: The file cannot be accessed by the system. (os error 1920), stat 'C:\Users\amazel\AppData\Local\Microsoft\WindowsApps\python3.10.exe'
    at Object.statSync (deno:runtime/js/30_fs.js:322:9)
    at getQuartoJupyterCapabilities (file:///C:/Users/amazel/AppData/Local/Programs/Quarto/bin/quarto.js:51852:22)
    at jupyterCapabilities (file:///C:/Users/amazel/AppData/Local/Programs/Quarto/bin/quarto.js:51771:34)
    at async file:///C:/Users/amazel/AppData/Local/Programs/Quarto/bin/quarto.js:104084:16
    at async withSpinner (file:///C:/Users/amazel/AppData/Local/Programs/Quarto/bin/quarto.js:56613:9)
    at async checkJupyterInstallation (file:///C:/Users/amazel/AppData/Local/Programs/Quarto/bin/quarto.js:104080:5)
    at async check (file:///C:/Users/amazel/AppData/Local/Programs/Quarto/bin/quarto.js:103995:13)
    at async Command.fn (file:///C:/Users/amazel/AppData/Local/Programs/Quarto/bin/quarto.js:104212:5)
    at async Command.execute (file:///C:/Users/amazel/AppData/Local/Programs/Quarto/bin/quarto.js:8437:13)
    at async quarto (file:///C:/Users/amazel/AppData/Local/Programs/Quarto/bin/quarto.js:127540:5)
cderv commented 1 year ago

Thanks for sharing the repository. I can reproduce on your project.

I see this browser console error on your website

VM1504:1 Uncaught SyntaxError: Unterminated string in JSON at position 10000000
    at JSON.parse (<anonymous>)
    at htmlwidgets.js:646:27
    at Array.forEach (<anonymous>)
    at forEach (htmlwidgets.js:55:14)
    at htmlwidgets.js:574:7
    at Array.forEach (<anonymous>)
    at forEach (htmlwidgets.js:55:14)
    at window.HTMLWidgets.staticRender (htmlwidgets.js:572:5)
    at maybeStaticRenderLater (htmlwidgets.js:697:26)
    at HTMLDocument.<anonymous> (htmlwidgets.js:704:7)

So I believe there is indeed something related to the number of point. If I reduce T= value, we see that the plotly graph is working correctly.

If we look at the HTML source for the plotly element we can see the data, it starts like this

<figure class="figure"><div class="plotly html-widget html-fill-item-overflow-hidden html-fill-item" id="htmlwidget-29617411b776018e6e4e" style="width:100%;height:464px;"></div>
<script type="application/json" data-for="htmlwidget-29617411b776018e6e4e">{"x":{"data":[{"

and indeed at the end we can see

17500,17500,17500,17500,17500,17500,17500,17500,17500,17500,17500,17500,17500,17500,17500,17500,17500,17500,17500,17500,17500,17500,17500,17500,17500,17500,17500,17500,17500,17500,17500,17500,17500,17500,17500,17500,17500,17500,</script></figure>
</div>
</div></section></section></main>
</div>

<script src="site_libs/quarto-html/zenscroll-min.js"></script>
</body></html>

So two conclusion:

If I reduce the number of points, then the data is complete and the document contains the end of the plotly part + the rest of content

HTML subset for correct end ````html "Generation: 44
Asexuals: 0
f: 0.80","Generation: 45
Asexuals: 0
f: 0.80","Generation: 46
Asexuals: 0
f: 0.80","Generation: 47
Asexuals: 0
f: 0.80","Generation: 48
Asexuals: 0
f: 0.80","Generation: 49
Asexuals: 0
f: 0.80","Generation: 50
Asexuals: 0
f: 0.80"],"frame":"0.8","type":"scatter","mode":"markers","marker":{"autocolorscale":false,"color":"rgba(35,95,164,1)","opacity":1,"size":5.6692913385826778,"symbol":"circle","line":{"width":1.8897637795275593,"color":"rgba(35,95,164,1)"}},"hoveron":"points","showlegend":false,"xaxis":"x","yaxis":"y","hoverinfo":"text","visible":true}],"traces":[0,1]}],"shinyEvents":["plotly_hover","plotly_click","plotly_selected","plotly_relayout","plotly_brushed","plotly_brushing","plotly_clickannotation","plotly_doubleclick","plotly_deselect","plotly_afterplot","plotly_sunburstclick"],"base_url":"https://plot.ly"},"evals":[],"jsHooks":[]}
Figure 1.2: Results from a simulation study in which a single clonal individual was introduced into a sexual population at generation 1000 (Lively 2009). The sexual population was initiated at carrying capacity: \(K_{sex} = 10000\). Note that the asexual lineage replaces the sexual population in about 25 generations. The asexual population then attains a new carrying capacity at \(K_{asex} = 20000\) individuals. The frequency of males in the sexual population was assumed to be 1/2. Annual reproduction, with non-overlapping generations, was also assumed.

````

There is indeed an error somewhere in the processing. I am looking into this. It does not seem to happen in a single doc, or simpler project... 🤔

cderv commented 1 year ago

Ok so investigation leads to

code-link: true

Set it to false it should solves this.

code-link: true will trigger a processing of the HTML file by downlit R package. And the issue is there as the returned HTML is truncated. Something is not working as expected.

I'll open an issue in downlit and will link here.

Thanks a lot for the report ! We would not have found this without your report.

cderv commented 1 year ago

See the rest of investigation at https://github.com/r-lib/downlit/issues/172

The issue is definitely with how downlit is working. The HTML produced with this plotly graph is creating a HTML <script> node with a very long JSON data in it. It is 12950084 characters ( bytes here) long, which is more than the hardcoded limit of libxml2 parser used by downlit through xml2 R package. Setting an option for xml2 can remove this constraint and will make things work, but there is no way currently to pass such option to xml2 from your code.

I don't know what Quarto could / should do here. We could try to check for such content and not run downlit on this file.

But it seems to me something than downlit should deal with 🤔