Error in accentuation in Rmarkdown with self_contained: FALSE and dygraphs

[diegocgaona]

I'm trying to make a html Rmarkdown with a dygraphs and using a source file. When I use self_contained: TRUE I don't have a problem, but when I use self_contained: FALSE , the legend accentuation goes wrong.

If I use without the source, that's OK, but I need to use sources in my case.

Here I put a reproducible code:

the source (tilemaker.R)

ptcolsumm <-  c("Usuários", "Sessões", "Pageview Único", "Pageviews",  "Tempo Médio", "Rejeição", "Taxa de Rejeição", "Taxa de Saída", "não sei", "ACENTUAÇÃO", "Teste")

organic <- data.frame(organic = as.numeric(c(186,640,278,11,267,117,41,17,38)))
organic <- as.ts(organic)

# Facebook impressions by type
teste <- dygraph(organic) %>%
  dySeries("organic", label = "Orgânico") %>%
  dyLegend(width = 400, show = "always") %>%
  dyOptions(fillGraph = TRUE, fillAlpha = 0.2, drawGrid = FALSE, drawPoints = TRUE, pointSize = 2)

The Rmd:

---
title: "Habits"
output:
  html_document:
    theme: flatly
    highlight: tango
    self_contained: FALSE

---
```{r source}
require(magrittr)
require(dygraphs)
source("tilemaker.R", encoding = "UTF-8")

Quarterly Results {.tabset}

By Region

teste

BLOH BLOH 3

TESTE DE 3 DIVISÕES

If I don't use the tabs, it's works too, but I need it too.

[jjallaire]

A couple of things:

1. You might want to delete any existing _files directory (as it could have stale versions of dygraphs in it preventing painting from occurring correctly).
2. You should install the very latest versions of dygraphs and htmlwidgets from CRAN (as both have newish code to deal with visibility/sizing problems).

On Fri, Apr 29, 2016 at 9:37 AM, Diego Gaona notifications@github.com wrote:

I'm trying to make a html Rmarkdown with a dygraphs and using a source file. When I use self_contained: TRUE I don't have a problem, but when I use self_contained: FALSE , the legend accentuation goes wrong.

[image: dygraph with self_contained: FALSE. The correct would be "Orgânico"] http://i.stack.imgur.com/U9EPC.png

If I use without the source, that's OK, but I need to use sources in my case.

Here I put a reproducible code:

the source (tilemaker.R)

ptcolsumm <- c("Usuários", "Sessões", "Pageview Único", "Pageviews", "Tempo Médio", "Rejeição", "Taxa de Rejeição", "Taxa de Saída", "não sei", "ACENTUAÇÃO", "Teste")

organic <- data.frame(organic = as.numeric(c(186,640,278,11,267,117,41,17,38))) organic <- as.ts(organic)

Facebook impressions by type

teste <- dygraph(organic) %>% dySeries("organic", label = "Orgânico") %>% dyLegend(width = 400, show = "always") %>% dyOptions(fillGraph = TRUE, fillAlpha = 0.2, drawGrid = FALSE, drawPoints = TRUE, pointSize = 2)

The Rmd:

---

title: "Habits" output: html_document: theme: flatly highlight: tango

self_contained: FALSE

require(magrittr)
require(dygraphs)
source("tilemaker.R", encoding = "UTF-8")

Quarterly Results {.tabset}

By Region

teste

BLOH BLOH 3

TESTE DE 3 DIVISÕES

If I don't use the tabs, it's works too, but I need it too.

— You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub https://github.com/rstudio/rmarkdown/issues/686

[diegocgaona]

I'm using htmlwidgets 0.6 and dygraphs 0.8 (I tried the 0.9, but I have some errors related to vectors, and in the update, doesn't appears to have any new feature related to text or encoding). I deleted the folder, but I had the same result.

I noted that this happens with a DT table too, as you can see in the image bellow:

In this case, I'm using source("data_transform.R", encoding = "UTF-8"), which works fine when I use self_contained: TRUE

[jjallaire]

Are there JavaScript errors in the console?

On Fri, Apr 29, 2016 at 10:47 AM, Diego Gaona notifications@github.com wrote:

I'm using htmlwidgets 0.6 and dygraphs 0.8 (I tried the 0.9, but I have some errors related to vectors, and in the update, doesn't appears to have any new feature related to text or encoding). I deleted the folder, but I had the same result.

I noted that this happens with a DT table too, as you can see in the image bellow:

In this case, I'm using source("data_transform.R", encoding = "UTF-8"), which works fine when I use self_contained: TRUE [image: download] https://cloud.githubusercontent.com/assets/9142126/14919441/3dda7500-0dff-11e6-9134-533d16956d14.png

— You are receiving this because you commented. Reply to this email directly or view it on GitHub https://github.com/rstudio/rmarkdown/issues/686#issuecomment-215741677

[diegocgaona]

No. I think it's a error in the exporting, the characters came wrong in the HTML. You can see the complete report here: gaona.work/1U99k1P

[jjallaire]

We may be over my head in terms of encoding issues here. @yihui Any thoughts?

[yihui]

This is not reproducible to me. Although you didn't mention your sessionInfo(), I guess you are using Windows. If your locale is Portuguese and the file tilemaker.R is indeed encoded in UTF-8, I cannot think of a possible reason why this could fail. Anyway, we need library(rmarkdown); sessionInfo() if you have done update.packages() and it still does not work.

[diegocgaona]

My info:
`R version 3.2.3 (2015-12-10)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 7 x64 (build 7601) Service Pack 1

locale:
[1] LC_COLLATE=Portuguese_Brazil.1252  LC_CTYPE=Portuguese_Brazil.1252    LC_MONETARY=Portuguese_Brazil.1252
[4] LC_NUMERIC=C                       LC_TIME=Portuguese_Brazil.1252    

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] rmarkdown_0.9.6     RevoUtilsMath_3.2.3

loaded via a namespace (and not attached):
 [1] Rcpp_0.12.4         knitr_1.12.3        magrittr_1.5        bit_1.1-12          xtable_1.8-2       
 [6] rjson_0.2.15        R6_2.1.2            httr_1.1.0          tools_3.2.3         DT_0.1             
[11] DBI_0.4             htmltools_0.3.5     yaml_2.1.13         bit64_0.9-5         digest_0.6.9       
[16] shiny_0.13.2        twitteR_1.1.9       htmlwidgets_0.6     rsconnect_0.4.3     mime_0.4           
[21] formattable_0.1.7.1 httpuv_1.3.3        markdown_0.7.7`

Yes, I'm using Windows in Brazilian Portuguese. The tilemaker.R is in UTF-8 But I did a test without use the tilemaker.R, and the error persists:

---
title: "Habits"
output:
  html_document:
    theme: flatly
    highlight: tango
    self_contained: FALSE
---
```{r source}
require(magrittr)
require(dygraphs)
ptcolsumm <-  c("Usuários", "Sessões", "Pageview Único", "Pageviews",  "Tempo Médio", "Rejeição", "Taxa de Rejeição", "Taxa de Saída", "não sei", "ACENTUAÇÃO", "Teste")

organic <- data.frame(organic = as.numeric(c(186,640,278,11,267,117,41,17,38)))
organic <- as.ts(organic)

# Facebook impressions by type
teste <- dygraph(organic) %>%
  dySeries("organic", label = "Orgânico") %>%
  dyLegend(width = 400, show = "always") %>%
  dyOptions(fillGraph = TRUE, fillAlpha = 0.2, drawGrid = FALSE, drawPoints = TRUE, pointSize = 2)
``
<style> /* CSS style to solve the problem with the sparkline tooltip */
.jqstooltip {
  -webkit-box-sizing: content-box;
  -moz-box-sizing: content-box;
  box-sizing: content-box;
}
</style>

## Quarterly Results {.tabset}

### By Region

```{r3}

teste

``

### BLOH BLOH 3
TESTE DE 3 DIVISÕES

[yihui]

I still cannot think of a possible reason. How exactly did you render Rmd to HTML? Using the Knit button in RStudio or just the command line rmarkdown::render()?

[diegocgaona]

I use the Knit button in RStudio. Is a very odd problem... works fine with self_contained: TRUE , that's is strange.

[yihui]

Can you send the Rmd file, the self-contained HTML file, and a zip archive of the HTML file with all dependencies to me by email? yihui [at] rstudio.com

[diegocgaona]

Thanks @yihui! I sent the email for you.

[yihui]

Okay, I can reproduce the issue with your example. I think I have tracked down the problem to https://github.com/rstudio/rmarkdown/blob/a566648/R/html_document_base.R#L167 but it is so very weird -- I have no idea why the characters can be possibly mangled simply by paste(collapse = '\n'). I'll keep on investigating. Thanks!

[hadley]

Try including <meta charset="utf-8"> in the first line of your md document - it's something that has fixed encoding problems for me in the past (but I have no idea if it will work here)

[diegocgaona]

Thanks @yihui , I hope you can solve this :)

Thanks for your help @hadley, I tried your solution. Well, it works, but, when I do this way, the document works like when I use self_contained: TRUE , the characters come OK, but the HTML have all the dependencies in it.

I tried this way:

<meta charset="utf-8">

---
title: "Habits"
output:
  html_document:
    theme: flatly
    highlight: tango
    self_contained: FALSE
---

I think that way, the knit ignore the options.

And anyway, thanks both of you for your incredible packages.

[paulofelipe]

I'm facing the same problem. I noticed the problem occurs when there is acentuation in markdown part and in javascript part.

Here is my sessioninfo:

R version 3.2.5 (2016-04-14)
Platform: i386-w64-mingw32/i386 (32-bit)
Running under: Windows 7 (build 7601) Service Pack 1

locale:
[1] LC_COLLATE=Portuguese_Brazil.1252  LC_CTYPE=Portuguese_Brazil.1252   
[3] LC_MONETARY=Portuguese_Brazil.1252 LC_NUMERIC=C                      
[5] LC_TIME=Portuguese_Brazil.1252    

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] dygraphs_0.9

loaded via a namespace (and not attached):
 [1] htmlwidgets_0.6 zoo_1.7-12      magrittr_1.5    htmltools_0.3.5 tools_3.2.5    
 [6] xts_0.9-7       yaml_2.1.13     Rcpp_0.12.4     rmarkdown_0.9.6 grid_3.2.5     
[11] jsonlite_0.9.19 digest_0.6.9    lattice_0.20-33

This example has accentuation in both:

---
title: "Teste - Dygraphs"
output: 
  html_document:
    self_contained: false
---

# Ação
```{r, echo = FALSE, warning=FALSE, message=FALSE}
library(dygraphs)
lungDeaths <- cbind(mdeaths, fdeaths)
dygraph(lungDeaths, main = "Ação")
``

This one has no accentuation in markdown. It works:

---
title: "Teste - Dygraphs"
output: 
  html_document:
    self_contained: false
---

# Acao
```{r, echo = FALSE, warning=FALSE, message=FALSE}
library(dygraphs)
lungDeaths <- cbind(mdeaths, fdeaths)
dygraph(lungDeaths, main = "Ação")
``

With self_contained: true everything is fine.

Thanks in advance!

[diegocgaona]

@paulofelipe Yeah, I tested your code and the same happened. Nice discovery!! It is a very weird error... I hope that this information can help @yihui @jjallaire @hadley to find a solution. Thanks!

[yihui]

Yes I have already discovered that during my debugging process two weeks ago, and I just need to figure out why that happens. It could be a base R bug on Windows, or something beyond my understanding. I'll let you know when I make more progress on this issue.

[paulofelipe]

@diegocgaona, I don't know if you have figured out a solution, but I'm doing the follow:

• Create the htmlwidget and save it
• After that, read the html file with readLines() using UTF-8 as encoding
• cat the relevant lines to generate the chart in final html with results='asis' option

It is important to include the dependencies manually or create the same chart in a chunk with include=FALSE. In this case, rmarkdown will include the dependency but will not render the chart with errors in encoding.

```{r, include=FALSE}
library(dygraphs)
library(htmlwidgets)
lungDeaths <- cbind(mdeaths, fdeaths)
dygraph(lungDeaths, main = "Ação")

``

```{r, result='asis'}
d<-dygraph(lungDeaths, main = "Ação", width = "100%")
saveWidget(d, file = 'd_widget.html', selfcontained = FALSE)
cat(readLines('d_widget.html', encoding = "UTF-8")[16])
cat(readLines('d_widget.html', encoding = "UTF-8")[18])

``

[diegocgaona]

Hi @paulofelipe,

I didn't figured out yet, thanks for your help.

Sorry for my lack of knowledge, when you say

include the dependencies manually

It's like you did in the first chunk? Put the library in every chunk with codification problem?

[paulofelipe]

Hi @diegocgaona, as I said there are two options. I'm using the second: "create the same chart in a chunk with include=FALSE". We don't need do this for every chunk, just once for each function that you are using. For example, if you repeat the second chunk multiple times it should work. If you use another function of the same package or another package you should do the same thing as the first chunk. When using the cat() it is important to get the relevant lines, I don't know if it changes when one uses another function/package (highcharts for example). To be more organized, you could separate the library()in another chunk.

I don't know if I'm making myself clear, but if you have other questions I can try to give a better answer.

[diegocgaona]

Thanks @paulofelipe

I tried and it worked! I hope that in the future, it will be more simple, but it's a solution right now.

The code from my test is: (if someone try to run, please, modify the double ` to three)

---
title: "Habits"
output:
  html_document:
    theme: flatly
    highlight: tango
    self_contained: FALSE
---

``{r source, echo=FALSE}
require(magrittr)
require(dygraphs)
require(htmlwidgets)
ptcolsumm <-  c("Usuários", "Sessões", "Pageview Único", "Pageviews",  "Tempo Médio", "Rejeição", "Taxa de Rejeição", "Taxa de Saída", "não sei", "ACENTUAÇÃO", "Teste")

organic <- data.frame(organic = as.numeric(c(186,640,278,11,267,117,41,17,38)))
organic <- as.ts(organic)

``

``{r, include=FALSE, echo=FALSE}
# Facebook impressions by type
teste <- dygraph(organic) %>%
  dySeries("organic", label = "Orgânico") %>%
  dyLegend(width = 400, show = "always") %>%
  dyOptions(fillGraph = TRUE, fillAlpha = 0.2, drawGrid = FALSE, drawPoints = TRUE, pointSize = 2)
teste
``

``{r, results='asis', echo=FALSE}
saveWidget(teste, file = 'teste_widget.html', selfcontained = FALSE)
cat(readLines('teste_widget.html', encoding = "UTF-8")[16])
cat(readLines('teste_widget.html', encoding = "UTF-8")[18])
``

[diegocgaona]

Hi @paulofelipe , a stupid question... how I can save the widgets in a subfolder of my working folder? I tried, but I get errors.... I tried this way:

saveWidget(teste, file = './widgets/teste_widget.html', selfcontained = FALSE)

Thanks!

[paulofelipe]

@diegocgaona try this:

wd <- getwd()
saveWidget(teste, file = file.path(wd, 'widgets/teste_widget.html'), selfcontained = FALSE)

[yihui]

@diegocgaona @paulofelipe No need to hack it like that. I think I have found the root cause of this issue. It took me quite a long time but eventually the fix was simple. The problem came from the htmltools package, and you may test the fix with:

devtools::install_github('rstudio/htmltools')

Please let me know if it fixes the problem for you. You'd better restart the R session before installing the fix.

[diegocgaona]

@yihui Great work! In my tests worked fine without the hacks, (with different types of widgets, dygraphs, highcharts, sparklines)

You will put this on the next release?

Thanks for your time and help.

[paulofelipe]

Thanks @yihui! It worked in my tests as well.

[yihui]

I have submitted a pull request at https://github.com/rstudio/htmltools/pull/64 Not sure when it will be merged and released, though.

[LilianaPacheco]

Thank you all for this thread, the tests worked very good!

[github-actions[bot]]

This old thread has been automatically locked. If you think you have found something related to this, please open a new issue by following the issue guide (https://yihui.org/issue/), and link to this old issue if necessary.