yihui / knitr

A general-purpose tool for dynamic report generation in R
https://yihui.org/knitr/
2.38k stars 876 forks source link

Leaflet map is cut off when rendering qmd to PDF, wrong view of snapshot is taken. #2276

Closed icejean closed 1 year ago

icejean commented 1 year ago

The problem may be reproduced by the following qmd code, please delete the '\' in the '`\``' mark inside for running the code, , it seems that a wrong view of the map is set when it's rendered to PDF.

---
format:
  pdf:
    documentclass: scrreprt
---

## htmlwidgets

It's O.K. with HTML & Word, but the marker is cut off from the screenshot when rendered to PDF:

```{r}
#| fig-alt: Leaflet map of Maungawhau / Mount Eden.
#| error: true

library(leaflet)
map<-leaflet(width="90%") |>
  setView(lng = 116.3125774825, lat = 39.9707249401, zoom = 16) |>
  addTiles(urlTemplate =  # Tencent tile map provider URL
    "http://rt0.map.gtimg.com/realtimerender?z={z}&x={x}&y={-y}&type=vector&style=0") |>
  addMarkers(lng=116.3125774825, lat=39.9707249401, popup="The birthplace of COS")
map

It's O.K. by taking a snapshot and saving to png, then being loaded later.

library(htmlwidgets)
library(webshot2)

## save html to png
saveWidget(map, "leaflet_map.html", selfcontained = FALSE)
webshot("leaflet_map.html", file = "leaflet_map.png",
        cliprect = "viewport")

## optionally display image when using knitr
knitr::include_graphics("leaflet_map.png")


Here's the rendered PDF output:
[test.pdf](https://github.com/yihui/knitr/files/12246601/test.pdf)
yihui commented 1 year ago

It's a matter of the figure size. When you call webshot2::webshot() manually, you are using a different default size. You can increase the size in chunk options, e.g.,

```{r}
#| fig.width: 13
#| fig.height: 10

library(leaflet)
map<-leaflet(width="90%") |>
  setView(lng = 116.3125774825, lat = 39.9707249401, zoom = 16) |>
  addTiles(urlTemplate =  # Tencent tile map provider URL
    "http://rt0.map.gtimg.com/realtimerender?z={z}&x={x}&y={-y}&type=vector&style=0") |>
  addMarkers(lng=116.3125774825, lat=39.9707249401, popup="The birthplace of COS")
map


[test.pdf](https://github.com/yihui/knitr/files/12384187/test.pdf)

> please delete the `\` in the ``` `\`` ``` mark inside for running the code

Please read and follow [the issue guide](new/choose), and you'll know how to write ```` ``` ```` when you file an issue. Thanks!
icejean commented 1 year ago

Hi, YiHui, thanks for your reply. But it doesn't work in my environment, just don't know why. Will it be something related to locale or package versions?

> xfun::session_info('knitr')
R version 4.1.2 (2021-11-01)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: CentOS Linux 7 (Core), RStudio 2023.6.0.421

Locale:
  LC_CTYPE=zh_CN.UTF-8       LC_NUMERIC=C               LC_TIME=zh_CN.UTF-8       
  LC_COLLATE=zh_CN.UTF-8     LC_MONETARY=zh_CN.UTF-8    LC_MESSAGES=zh_CN.UTF-8   
  LC_PAPER=zh_CN.UTF-8       LC_NAME=C                  LC_ADDRESS=C              
  LC_TELEPHONE=C             LC_MEASUREMENT=zh_CN.UTF-8 LC_IDENTIFICATION=C       

Package version:
  evaluate_0.21   graphics_4.1.2  grDevices_4.1.2 highr_0.10      knitr_1.43      methods_4.1.2  
  stats_4.1.2     tools_4.1.2     utils_4.1.2     xfun_0.40       yaml_2.3.7     
> 

I 'v updated the packages related to knitr:

#  scl enable devtoolset-11 bash

xfun::session_info('knitr')
tobeUpdated<-c("cli", "evaluate", "glue", "graphics", "grDevices", "highr", "knitr", "lifecycle",
               "magrittr", "methods", "rlang", "stats", "stringi", "stringr", "tools", "utils", 
               "vctrs", "xfun", "yaml")
update.packages(oldPkgs=tobeUpdated, ask = TRUE, INSTALL_opts=c("--no-multiarch"))
yihui commented 1 year ago

I have no idea... I was testing on macOS, and don't have a CentOS machine to test with.

icejean commented 1 year ago

Not worked on Windows too.

#| fig-alt: Leaflet map of Maungawhau / Mount Eden.
#| error: true
#| fig.width: 13
#| fig.height: 10

Sys.setenv(CHROMOTE_CHROME =
    "C:/Users/Jean/AppData/Local/Google/Chrome/Application/chrome.exe")

library(leaflet)
map<-leaflet(width = "90%") |>
  setView(174.764, -36.877, zoom = 16) |> 
  addTiles() |>
  addMarkers(174.764, -36.877, popup = "Maungawhau") 
map

test.pdf

> xfun::session_info('knitr')
R version 4.1.0 (2021-05-18)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 10 x64 (build 19045), RStudio 2023.6.1.524

Locale:
  LC_COLLATE=Chinese (Simplified)_China.936  LC_CTYPE=Chinese (Simplified)_China.936   
  LC_MONETARY=Chinese (Simplified)_China.936 LC_NUMERIC=C                              
  LC_TIME=Chinese (Simplified)_China.936    

Package version:
  evaluate_0.21   graphics_4.1.0  grDevices_4.1.0 highr_0.10      knitr_1.43     
  methods_4.1.0   stats_4.1.0     tools_4.1.0     utils_4.1.0     xfun_0.40      
  yaml_2.3.5     
> 
yihui commented 1 year ago

@cderv Are you able to reproduce the problem on Windows?

cderv commented 1 year ago

Yes I can reproduce. I don't get the same pdf as you using your code @yihui

---
title: "test"
format: pdf
---

```{r}
#| fig.width: 13
#| fig.height: 10

library(leaflet)
map<-leaflet(width="90%") |>
  setView(lng = 116.3125774825, lat = 39.9707249401, zoom = 16) |>
  addTiles(urlTemplate =  # Tencent tile map provider URL
    "http://rt0.map.gtimg.com/realtimerender?z={z}&x={x}&y={-y}&type=vector&style=0") |>
  addMarkers(lng=116.3125774825, lat=39.9707249401, popup="The birthplace of COS")
map


[test2.pdf](https://github.com/yihui/knitr/files/12393952/test2.pdf)

It seems to be related to Quarto only though, as in Rmd file I get the correct behavior...

@yihui did you try inside Quarto on Mac too ? 
cderv commented 1 year ago

I noticed that DPI for PDF is set to 300 in format pdf in Quarto https://github.com/quarto-dev/quarto-cli/blob/4961d39635dcfd6ddb49f16f21d93951469b298b/src/format/pdf/format-pdf.ts#L125

I tried setting #| fig-dpi: 72 to match default in pdf_document() but it did not help...

It seems to work well for Rmd as far as I tested, so something is off in Qmd. I don't understand yet what can be different in Quarto for this.

cderv commented 1 year ago

I tried setting #| fig-dpi: 72 to match default in pdf_document() but it did not help...

Ok in fact, fig-dpi from Quarto does not seem to work at chunk level. But doing #| dpi: 72 works.

So this is indeed a sizing matter. PDF is 300 dpi by default on Quarto, and the screenshot size in px is calculated as fig.(height|width) x dpi.

So either adapting DPI or fig.height accordingly should work.

```{r}
#| fig-width: 13
#| fig-height: 10
#| dpi: 72
#| fig-alt: Leaflet map of Maungawhau / Mount Eden.
library(leaflet)
map<-leaflet(width="90%") |>
  setView(lng = 116.3125774825, lat = 39.9707249401, zoom = 16) |>
  addTiles(urlTemplate =  # Tencent tile map provider URL
    "http://rt0.map.gtimg.com/realtimerender?z={z}&x={x}&y={-y}&type=vector&style=0") |>
  addMarkers(lng=116.3125774825, lat=39.9707249401, popup="The birthplace of COS")
map
#| fig-width: 3.12
#| fig-height: 2.4
#| fig-alt: Leaflet map of Maungawhau / Mount Eden.
library(leaflet)
map<-leaflet(width="90%") |>
  setView(lng = 116.3125774825, lat = 39.9707249401, zoom = 16) |>
  addTiles(urlTemplate =  # Tencent tile map provider URL
    "http://rt0.map.gtimg.com/realtimerender?z={z}&x={x}&y={-y}&type=vector&style=0") |>
  addMarkers(lng=116.3125774825, lat=39.9707249401, popup="The birthplace of COS")
map


I suspect though something with `saveWidget()` step and those dimension and not just webshot(). I'll check that
cderv commented 1 year ago

I suspect though something with saveWidget() step and those dimension and not just webshot(). I'll check that

It was not that. I believe the issue is that

If I do as in first example (https://github.com/yihui/knitr/issues/2276#issue-1834216047), and open the html saved with saveWidget to run a manual Print in Chrome browser, then I get a truncated leaflet map.

Current webshot has no way to pass dimension to screenshot_pdf (https://github.com/rstudio/webshot2/issues/11) and printing does not work as screenshoting in a png.

What happens is that vwidth and vheight that we set based on chunk value, are used to initialize the viewport of the chrome session only, and then a print is done but with the default letter page size. Not good.

This brings us back to initial example: if you set the dev: png for the chunk with the leaflet, then it will correctly use size and show you good results

---
title: "test"
format: pdf
---

```{r}
#| dev: png
#| fig-alt: Leaflet map of Maungawhau / Mount Eden.
library(leaflet)
map<-leaflet(width="90%") |>
  setView(lng = 116.3125774825, lat = 39.9707249401, zoom = 16) |>
  addTiles(urlTemplate =  # Tencent tile map provider URL
    "http://rt0.map.gtimg.com/realtimerender?z={z}&x={x}&y={-y}&type=vector&style=0") |>
  addMarkers(lng=116.3125774825, lat=39.9707249401, popup="The birthplace of COS")
map


[test.pdf](https://github.com/yihui/knitr/files/12396210/test.pdf)

@yihui it seems using `webshot2::webshot()` for pdf output won't be as good as using `webshot::webshot()` was with PhantomJS. 

I wonder if we should tweak the feature in **knitr** when extension if PDF or just rely on **webshot2** to improve over time. 

I think I went to the bottom of this. If something is not clear, please do not hesitate to ask
yihui commented 1 year ago

Thanks a lot! So the solution is either set dev: png if using webshot2 + Chrome is preferred, or use the webshot package + PhantomJS to take the screenshot if dev: pdf is preferred. In either case, the dpi option needs to be set to a smaller value, e.g., 72.

```{r}
#| dev: png
#| webshot: webshot2
#| fig.width: 13
#| fig.height: 10
#| dpi: 72
#| dev: pdf
#| webshot: webshot
#| fig.width: 13
#| fig.height: 10
#| dpi: 72


Since simple solutions exist, I don't think we need to do anything in **knitr**'s codebase. We can wait for improvements in **webshot2** (https://github.com/rstudio/webshot2/issues/11).
icejean commented 1 year ago

Great, it works! I add an option to export the code chunk options together with the source so that people can know how to do it.

```{r}
#| echo: fenced
#| dev: png
#| webshot: webshot2
#| fig.width: 13
#| fig.height: 10
#| dpi: 72


Thanks to everybody, have a good day!
cderv commented 1 year ago

In either case, the dpi option needs to be set to a smaller value, e.g., 72.

Not completely. If setting dev: png then there is no need to change figure size. This works for me as shared in https://github.com/yihui/knitr/issues/2276#issuecomment-1686298985

---
title: "test"
format: pdf
---

```{r}
#| dev: png
#| fig-alt: Leaflet map of Maungawhau / Mount Eden.
library(leaflet)
map <- leaflet(width="90%") |>
  setView(lng = 116.3125774825, lat = 39.9707249401, zoom = 16) |>
  addTiles(urlTemplate =  # Tencent tile map provider URL
    "http://rt0.map.gtimg.com/realtimerender?z={z}&x={x}&y={-y}&type=vector&style=0") |>
  addMarkers(lng=116.3125774825, lat=39.9707249401, popup="The birthplace of COS")
map


If `dev: pdf` is used, then figure size needs to be adapted, taking into account `dpi` to set the `fig.*` option, or setting dpi to a lower value like `dpi: 72` (and possibly also adapted size)

This would be the current situation from all my tests. 

> Since simple solutions exist, I don't think we need to do anything in knitr's codebase. We can wait for improvements in webshot2 (https://github.com/rstudio/webshot2/issues/11).

@yihui as screenshot does not work really well with **webshot2**, pdf and quarto I was wondering if we could do : 

* If 
  * `webshot = "webshot2"`
  * `dev = 'pdf'`  
  * inside quarto (or just check dpi value maybe) 
* Then we either  
  * set dpi to 72
  * set dev to png

Or maybe just throw a warning for high dpi and dev pdf with webshot2. 

Idea is just to help users.  
yihui commented 1 year ago

Not completely. If setting dev: png then there is no need to change figure size.

Okay, thanks for the clarification!

as screenshot does not work really well with webshot2, pdf and quarto I was wondering if we could do :

Yes, we could. It may be a better idea to set dev: png since pdf doesn't use the correct figure size. Throwing a warning in this case is also fine. I don't really have a strong opinion here.

cderv commented 1 year ago

Thanks @yihui !

A thoughts related to warnings: I wonder if we should find a system to allow to customization of error by Quarto for example. Thinking about that because dev = 'png' is fig-format: png on quarto context. Using classes on exceptions would allow easily to catch and rethrow.

Anyhow, not urgent, but I thought of that because I believe this will throw a warning for any quarto user as dev = "pdf" is set by default;

yihui commented 1 year ago

Yes, good idea. I just tweaked the message to show fig-format to Quarto users.

github-actions[bot] commented 7 months ago

This old thread has been automatically locked. If you think you have found something related to this, please open a new issue by following the issue guide (https://yihui.org/issue/), and link to this old issue if necessary.