Add book chapter section on rendering to `github_document`?

[rempsyc]

Related to: #216

---

I understand that this would be relevant for a smaller subset of users, but I wonder if it would make sense to add a new section in the book chapter 4.3 (https://ardata-fr.github.io/flextable-book/rendering.html#r-markdown-documents) about an official way of rendering flextables to a GitHub README file. I also understand that so far there is no official solution for this and that this is why the flextable README does not include rendered flextables.

Given that GitHub does not support HTML, I have so far relied on a workaround using images saved in "man/figures" with the fig.path argument of knitr::opts_chunk, which allowed me to refer to the images using regular markdown or html syntax. Using this strategy, it seems that simply running the flextable automatically converted them or saved them as images. That has always worked until now.

I recently updated all my packages, and then tried to reknit my README file. But this time, without changing the code, I got this error:

Error: Functions that produce HTML output found in document targeting gfm+tex_math_dollars-yaml_metadata_block output.
Please change the output type of this document to HTML. Alternatively, you can allow
HTML output in non-HTML formats by adding this option to the YAML front-matter of
your rmarkdown file:

  always_allow_html: true

Note however that the HTML output will not be visible in non-HTML formats.

I've tried what the error suggests, setting always_allow_html to true, but this creates two problems: (1) the table does not look as it should, but (2) GitHub does not support HTML anyway.

So it seems my old workaround of having the images saved automatically does not work anymore. Here is a small reprex.

---
output: 
  github_document
---

```{r setup, include = FALSE}
knitr::opts_chunk$set(
  collapse = TRUE,
  comment = "#>",
  fig.path = "man/figures/README2-",
  out.width = "100%"
)

flextable::set_flextable_defaults(background.color = "white")
library(flextable)

flextable(mtcars)

Session Info:

sessionInfo() R version 4.2.2 (2022-10-31 ucrt) Platform: x86_64-w64-mingw32/x64 (64-bit) Running under: Windows 10 x64 (build 19045)

Matrix products: default

locale: [1] LC_COLLATE=English_Canada.utf8 LC_CTYPE=English_Canada.utf8 LC_MONETARY=English_Canada.utf8 [4] LC_NUMERIC=C LC_TIME=English_Canada.utf8

attached base packages: [1] stats graphics grDevices utils datasets methods base

loaded via a namespace (and not attached): [1] zip_2.2.2 Rcpp_1.0.10 fontBitstreamVera_0.1.1 compiler_4.2.2
[5] later_1.3.0 gfonts_0.2.0 tools_4.2.2 uuid_1.1-0
[9] digest_0.6.31 evaluate_0.20 jsonlite_1.8.4 lifecycle_1.0.3
[13] rlang_1.1.0 shiny_1.7.4 cli_3.6.1 rstudioapi_0.14
[17] yaml_2.3.7 crul_1.3 curl_5.0.0 fontLiberation_0.1.0
[21] xfun_0.38 fastmap_1.1.1 xml2_1.3.3 officer_0.6.2.002
[25] knitr_1.42 gdtools_0.3.3 systemfonts_1.0.4 askpass_1.1
[29] grid_4.2.2 glue_1.6.2 httpcode_0.3.0 data.table_1.14.8
[33] R6_2.5.1 textshaping_0.3.6 rmarkdown_2.21 magrittr_2.0.3
[37] fontquiver_0.2.1 promises_1.2.0.1 htmltools_0.5.5 ellipsis_0.3.2
[41] rsconnect_0.8.29 mime_0.12 flextable_0.9.1 xtable_1.8-4
[45] httpuv_1.6.9 ragg_1.2.5 openssl_2.0.6 crayon_1.5.2

Furthermore, here is the rendered GitHub README for the actual package, https://github.com/rempsyc/rempsyc/ and here is the associated code, https://github.com/rempsyc/rempsyc/blob/main/README.Rmd

Pinging @DanChaltiel in case you may have any insight into that. Thanks.

[rempsyc]

Note: flextable v0.9.0 does not produce this error: the file renders as if always_allow_html was already true, so prints it as html instead of image (setting it to false in that case changes nothing, it still renders). However, the image is not saved automatically, so does not appear correctly on the rendered file. That could be due to an update from a different package most likely; I am trying to figure out which one.

Also, flextable::save_as_image() does not produce satisfying output unfortunately.

Other packages using this workaround to render flextables in the README file: autoreg, lavaanExtra.

[davidgohel]

Also, flextable::save_as_image() does not produce satisfying output unfortunately.

show me an example.

[rempsyc]

Good idea, thanks. Here is a screenshot of the first flextable of the readme, as it appears on the current pkgdown site:

Here is the new version using save_as_image():

The three minor differences I observe with save_as_image() are the following:

1. Lines are soft wrapping, whereas I would like them to stay on the same line. This one is probably an easy fix code-wise.
2. The black horizontal lines are greyed out, and the text is also paler.
3. The image appears slightly blurrier.

Would you like a reprex for this?

That said, it certainly is a workaround (along with not updating the README). However, compared to the previous workaround, it also requires an additional code chunk per figure.

[rempsyc]

ftExtra for example, uses the following workaround to include flextables in the GitHub README:

render_html <- function(x, options, ...) {
  to <- tempfile(fileext = '.png')
  knit_print(structure(
    flextable::save_as_image(x, to, webshot = 'webshot2'),
    class = 'webshot'
  ))
}

Using webshot2 is also the suggested fix in #216, but I understand from #522 that this workaround will not work anymore since flextable has stopped using webshot2.

[DanChaltiel]

As explained in #216, I indeed have a similar workflow to generate my readme (see https://github.com/DanChaltiel/crosstable/blob/main/README.Rmd).

I agree that save_as_image()` is still a bit unpredictable today, as each time I render the file, the resulting image has a slightly different resolution. The difference is not very significant though, so I haven't worked on this problem and cannot really help, sorry!

[davidgohel]

Hello

About save_as_image()

save_as_image() now uses ragg to produce a PNG, an R graphical device, that mean it has a resolution (by default 200).

The result of the example provided here is presented below: https://davidgohel.github.io/flextable/reference/save_as_image.html

You can check the resultion, it is 200.

@DanChaltiel Based on that example, can you post two runs so that we can check if the resolution vary? I'd be happy to investigate.

@rempsyc the example you provided is not a png generated by save_as_image but a screenshot made with a software named Greenshot. can you post the original generated file please?

[rempsyc]

Here is the raw image of the flextable on the current pkgdown site:

And here is using save_as_image(). Thanks to your suggestion, I was able to make it less blurry (point 3) and more similar to the first one using expand = 0 and res = 300. However, points 1 and 2 remain.

[davidgohel]

OK so we get the same result.

The version with resolution 200 is the following:

The version with resolution 300 is the following:

To me the difference is not obvious between the two and not blurry

[davidgohel]

For the question 1, you should call autofit()

[rempsyc]

Thanks David. For the blurriness, I'm not sure, maybe it's an interaction with pkgdown, but the higher resolution definitely helps. I think it's hard to see it when we zoom on the picture, but in the pkgdown site where the pictures are small, it is more obvious. There's definitely a difference though, as you can see from my previous website screenshots.

Using autofit() in the hidden chunk fixes the soft wrap, thanks. Internally, it already uses set_table_properties(layout = "autofit"), and that seemed to be enough for HTML/Word, but apparently not for png.

For question 2, do you see what I mean for the colour difference on the horizontal lines though (black vs. grey)? Not a huge deal, but mentioning just in case.

[DanChaltiel]

@DanChaltiel Based on that example, can you post two runs so that we can check if the resolution vary? I'd be happy to investigate.

Sorry, false alert, the output is now reproducible. I remember there was one in the past though, maybe when you switched from webshot.

[rempsyc]

Relevant to this issue: html tables automatic conversion to images through webshot and phantomjs, https://bookdown.org/yihui/rmarkdown-cookbook/html-widgets.html. I had forgotten but this is what I have been using, and the issue is thus probably with either webshot or phantomjs. Perhaps the book/docs could refer to this documentation in Chapter 4: https://ardata-fr.github.io/flextable-book/rendering.html

HTML widgets (https://www.htmlwidgets.org) are typically interactive JavaScript applications, which only work in HTML output. If you knit an Rmd document containing HTML widgets to a non-HTML format such as PDF or Word, you may get an error message like this:

Error: Functions that produce HTML output found in document targeting X output. Please change the output type of this document to HTML. Alternatively, you can allow HTML output in non-HTML formats by adding this option to the YAML front-matter of your rmarkdown file:

always_allow_html: yes

Note however that the HTML output will not be visible in non-HTML formats.

There is actually a better solution than the one mentioned in the above error message, but it involves extra packages. You can install the webshot package (Chang 2022) in R and also install PhantomJS:

install.packages("webshot")

webshot::install_phantomjs()

Then if you knit an Rmd document with HTML widgets to a non-HTML format, the HTML widgets will be displayed as static screenshots. The screenshots are automatically taken in knitr.

I have also opened a StackOverFlow question: https://stackoverflow.com/questions/76534162/how-to-include-html-flextable-inside-an-md-github-document-using-webshot-a

[cderv]

Relevant to this issue: html tables automatic conversion to images through webshot and phantomjs, bookdown.org/yihui/rmarkdown-cookbook/html-widgets.html. I had forgotten but this is what I have been using, and the issue is thus probably with either webshot or phantomjs.

Just adding some precision on the doc linked: knitr will indeed use webshot or webshot2 to screenshot HTML widgets when in a non HTML output. But flextable is not a HTML widgets, so it does not get handle by this.

My understanding as external to flextable is that Markdown is not among the native output format, so you would need to transform the output. Either by saving to an image as a step, or just using the new Grid output which is just a plot that knitr will know how to include.

---
title: "flextable as plot"
output: 
  github_document: default
---

```{r}
plot(flextable::flextable(head(iris)))

It seems the right way to do it to me within a markdown output format. 

@davidgohel you could consider maybe adapting `knit_print.flextable` to use `plot()` for a github document output. 🤷  

[rempsyc]

Thanks @cderv! For posteriority, here is a "reprex" comparing the old workaround and the new one you propose:

---
title: "flextable as plot"
output: 
  github_document: default
---

```{r, eval=FALSE}
# What users see
flextable::flextable(head(iris))

# hidden cderv workaround (fig.height needs to be specified case-by-case, or it leads to extra tall margins)
plot(flextable::flextable(head(iris)))

Warning in grid.Call.graphics(C_text, as.graphicsAnnot(x$label), x$x, x$y, :

font family not found in Windows font database

![table1](https://github.com/davidgohel/flextable/assets/13123390/da9e8885-0eec-4520-95d6-6e847ca184bd)

```` r
```{r, include=FALSE}
# hidden old workaround
flextable::set_flextable_defaults(background.color = "white")
table_temp <- flextable::flextable(head(iris))
table_temp <- flextable::autofit(table_temp)
flextable::save_as_image(table_temp, path = "test.png", expand = 0, res = 300)

![table2](https://github.com/davidgohel/flextable/assets/13123390/ba217235-e474-4357-8a2d-479cc24df8ed)

Although this workaround still requires an extra hidden chunk compared to the original solution (so users see only what they should see), it is still shorter then my current workaround. In fact, when specifying `fig.height`, they are pretty similar. That said, on my system and for my needs (`nice_table`), the plot method does not work properly because it cannot find the Times New Roman font so it defaults to the base font.

Note: this is similar to #216 (`plot` method vs `save_as_image` method)

[rempsyc]

Thanks everyone for the help. I have now identified the real source of the issue, which is knitr. Version 1.40 works, and 1.41 breaks it. Reprex:

---
output: 
  github_document: default
---

```{r}
# remotes::install_version("knitr", version = "1.40", repos = "http://cran.us.r-project.org")
packageVersion("knitr")
## [1] '1.40'
flextable::flextable(head(iris))

![table3](https://github.com/davidgohel/flextable/assets/13123390/957afab4-97b7-4e9b-ae48-95de86485712)

Thus, I will close wch/webshot#118, but will keep this one open in case we want to mention some of this information in the `flextable` book.

[davidgohel]

Thanks @cderv. I forgot about it but the fallback knit print method already use png output: https://github.com/davidgohel/flextable/blob/34697f88aaa8bbe61da4ef98b0964b5c56b587d8/R/printers.R#L689-L697

I can fix that in flextable that because I use is_html_output() which returns TRUE when the output if gfm. That's why the fallback is not used. Not sure if it changed or if it happened when I refactored the print method...

because it cannot find the Times New Roman font

@rempsyc to get good font support, I strongly recommend using sytemfonts and ragg. To register a font if not system-installed, you can use systemfonts::register_font(). (To use any font from google fonts, use gdtools::register_gfont()). RStudio let you easily define that in their menu:

The documentation of plot.flextable is mentioning it: https://davidgohel.github.io/flextable/reference/plot.flextable.html

[rempsyc]

Thanks @davidgohel. But there is something I don't understand. I don't want to use google fonts, but Times New Roman, which is one of the most universal system fonts available on all systems, including Windows. But it still triggers that warning even after changing the backend graphics device to AGG like in the screenshot above.

If I try to register Times New Roman, I get:

systemfonts::register_font("Times New Roman")
Error: A system font with that family name already exists

Full reprex:

---
output: github_document
---

```{r, echo=FALSE, fig.height=2}
windowsFonts()
## $serif
## [1] "TT Times New Roman"
## 
## $sans
## [1] "TT Arial"
## 
## $mono
## [1] "TT Courier New"

extrafont::loadfonts(device = "win")
x <- flextable::flextable(head(iris))
x <- flextable::font(x, part = "all", fontname = "Times New Roman")
plot(x)
## Warning in grid.Call.graphics(C_text, as.graphicsAnnot(x$label), x$x, x$y, :
## font family not found in Windows font database

![not times new roman](https://github.com/davidgohel/flextable/assets/13123390/4dd0869d-248e-4953-b8d2-a0f5c9d4e2ba)

Edit: this problem does not arise when running `plot(x)` directly from the console (it shows Times New Roman correctly), only when knitted.

Still not Times New Roman. But perhaps this should be discussed in a separate issue(?).

[davidgohel]

systemfonts::register_font("Times New Roman")
Error: A system font with that family name already exists

And I wrote if not system-installed, so it means it is already registered

only when knitted

I am sure you did not use ragg.

[rempsyc]

Thanks, indeed, the trick was to include knitr::opts_chunk$set(dev = "ragg_png") in an earlier chunk, then it works regardless of the graphic device selected in RStudio options.

[davidgohel]

I think I can close this issue. I added a note in the book "For markdown output, an image is used." It should be available in 30 minutes

The fallback to save_as_image() is now implemented 'gfm' (@cderv I let you judge about that knitr::is_html_output(), not sure if something should change on your side)

[cderv]

@davidgohel that is good! is_html_output() is a helper wrapper of pandoc_to() where we consider every html-supported output. Markdown is one of them in general. There is a excludes argument for the purpose of restricting the detection.

I see you added gfm - you could add every markdown output maybe. Or switch to knitr::pandoc_to() and support only html4, html5, maybe revealjs and other slide framework but that is all.

Thanks to @rempsyc we got to the bottom of the story at https://github.com/yihui/knitr/issues/2265 - this was a change in knitr to fix a long standing wrong behavior of is_html_output(). For a long time, github_document() was not seen as gfm because of Pandoc extension.

Thanks everyone !

[github-actions[bot]]

This old thread has been automatically locked. If you think you have found something related to this, please open a new issue and link to this old issue if necessary.