search

rstudio / rmarkdown

Dynamic Documents for R
https://rmarkdown.rstudio.com
GNU General Public License v3.0
2.84k stars 966 forks source link

render() with intermediates_dir argument fails when rendering to PDF #2183

Open beerda opened 3 years ago

beerda commented 3 years ago

test.Rmd:

---
title: "title"
author: "author"
output:
    pdf_document: default
---

foo

The following command fails:

> rmarkdown::render('test.Rmd', intermediates_dir='./tmp')

processing file: test.Rmd
  |..........................................................................................................| 100%
  ordinary text without R code

output file: /home/michal/.syncthing/data/R-packages/rmake/_experiments/multiple_markdown_targets/tmp/test.knit.md

/usr/lib/rstudio/bin/pandoc/pandoc +RTS -K512m -RTS /home/michal/.syncthing/data/R-packages/rmake/_experiments/multiple_markdown_targets/tmp/test.knit.md --to latex --from markdown+autolink_bare_uris+tex_math_single_backslash --output test.tex --lua-filter /home/michal/R/x86_64-pc-linux-gnu-library/4.1/rmarkdown/rmarkdown/lua/pagebreak.lua --lua-filter /home/michal/R/x86_64-pc-linux-gnu-library/4.1/rmarkdown/rmarkdown/lua/latex-div.lua --self-contained --highlight-style tango --pdf-engine pdflatex --variable graphics --variable 'geometry:margin=1in' 
Error in readLines(con, warn = FALSE) : cannot open the connection
In addition: Warning message:
In readLines(con, warn = FALSE) :
  cannot open file 'test.tex': No such file or directory
cderv commented 3 years ago

Unfortunately, I can't reproduce this 

> rmarkdown::render('test.Rmd', intermediates_dir="./tmp")

processing file: test.Rmd
  |.................................................................................| 100%
  ordinary text without R code

output file: C:/Users/chris/Documents/tmp/test.knit.md

"C:/Users/chris/scoop/shims/pandoc" +RTS -K512m -RTS "C:/Users/chris/Documents/tmp/test.knit.md" --to latex --from markdown+autolink_bare_uris+tex_math_single_backslash --output pandoc4b48636a5ee0.tex --lua-filter "C:\Users\chris\Documents\R\win-library\4.1\rmarkdown\rmarkdown\lua\pagebreak.lua" --lua-filter "C:\Users\chris\Documents\R\win-library\4.1\rmarkdown\rmarkdown\lua\latex-div.lua" --self-contained --highlight-style tango --pdf-engine pdflatex --variable graphics --variable "geometry:margin=1in" 

Output created: test.pdf

I am on windows with last versions of Packages and with R 4.1.0

Can you give more details with your environment ?

Can I ask why do you need to rename the intermediate dir ? It is always useful for us to know how those arguments are used. Playing with files and folders path with rmarkdown::render can be tricky sometimes.

beerda commented 3 years ago

I run the rendering of the same Rmd file into multiple output files with different arguments concurently and intermediate dir is a workaround to not to mess different executions.

My environment is:

> xfun::session_info('rmarkdown')
R version 4.1.0 (2021-05-18)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 20.04.2 LTS, RStudio 1.4.1717

Locale:
  LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C               LC_TIME=cs_CZ.UTF-8        LC_COLLATE=en_US.UTF-8    
  LC_MONETARY=cs_CZ.UTF-8    LC_MESSAGES=en_US.UTF-8    LC_PAPER=cs_CZ.UTF-8       LC_NAME=C                 
  LC_ADDRESS=C               LC_TELEPHONE=C             LC_MEASUREMENT=cs_CZ.UTF-8 LC_IDENTIFICATION=C       

Package version:
  base64enc_0.1.3   digest_0.6.27     evaluate_0.14     glue_1.4.2        graphics_4.1.0    grDevices_4.1.0  
  highr_0.9         htmltools_0.5.1.1 jsonlite_1.7.2    knitr_1.33        magrittr_2.0.1    markdown_1.1     
  methods_4.1.0     mime_0.11         rlang_0.4.11      rmarkdown_2.9     stats_4.1.0       stringi_1.6.2    
  stringr_1.4.0     tinytex_0.32      tools_4.1.0       utils_4.1.0       xfun_0.24         yaml_2.2.1       

Pandoc version: 2.11.4
beerda commented 3 years ago

I have also updated to pandoc 2.14.0.3 but the problem is still there:

> xfun::session_info('rmarkdown')
R version 4.1.0 (2021-05-18)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 20.04.2 LTS, RStudio 1.4.1717

Locale:
  LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C               LC_TIME=cs_CZ.UTF-8        LC_COLLATE=en_US.UTF-8    
  LC_MONETARY=cs_CZ.UTF-8    LC_MESSAGES=en_US.UTF-8    LC_PAPER=cs_CZ.UTF-8       LC_NAME=C                 
  LC_ADDRESS=C               LC_TELEPHONE=C             LC_MEASUREMENT=cs_CZ.UTF-8 LC_IDENTIFICATION=C       

Package version:
  base64enc_0.1.3   digest_0.6.27     evaluate_0.14     glue_1.4.2        graphics_4.1.0    grDevices_4.1.0  
  highr_0.9         htmltools_0.5.1.1 jsonlite_1.7.2    knitr_1.33        magrittr_2.0.1    markdown_1.1     
  methods_4.1.0     mime_0.11         rlang_0.4.11      rmarkdown_2.9     stats_4.1.0       stringi_1.6.2    
  stringr_1.4.0     tinytex_0.32      tools_4.1.0       utils_4.1.0       xfun_0.24         yaml_2.2.1       

Pandoc version: 2.14.0.3
cderv commented 3 years ago

Can you run a traceback() after you got the error ? https://adv-r.hadley.nz/debugging.html?q=traceback()#traceback

Thanks.

beerda commented 3 years ago 
> traceback()
4: readLines(con, warn = FALSE)
3: read_utf8(file)
2: patch_tex_output(texfile)
1: rmarkdown::render("test.Rmd", intermediates_dir = "./tmp")
beerda commented 3 years ago

The error does not occur if I delete

output:
    pdf_document: default

from the test.Rmd file.

cderv commented 3 years ago

You last comment means that this is something that happens with pdf_document only. Your previous comment means that it happens when we are trying to patch the resulting output. It seems that the file is not found where it should and R can't read it.

I still need to be able to reproduce to dig further. I'll try it on Ubuntu.

nick-ulle commented 2 years ago

I also get this error. Here's my session info:

R version 4.1.0 (2021-05-18)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Arch Linux

Locale:
  LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C
  LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8
  LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8
  LC_PAPER=en_US.UTF-8       LC_NAME=C
  LC_ADDRESS=C               LC_TELEPHONE=C
  LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C

Package version:
  base64enc_0.1.3 digest_0.6.25   evaluate_0.14   glue_1.4.2
  graphics_4.1.0  grDevices_4.1.0 highr_0.8       htmltools_0.5.0
  jsonlite_1.7.2  knitr_1.33      magrittr_2.0.1  markdown_1.1
  methods_4.1.0   mime_0.9        rlang_0.4.10    rmarkdown_2.10
  stats_4.1.0     stringi_1.6.2   stringr_1.4.0   tinytex_0.33
  tools_4.1.0     utils_4.1.0     xfun_0.25       yaml_2.2.1

Pandoc version: 2.14.0.2
cderv commented 2 years ago

I can now reproduce on Linux only. It seems there is a difference between Windows and Linux on this. I need to look into that closer. Thanks for the report both of you !

cderv commented 2 years ago

So it seems this come from https://github.com/rstudio/rmarkdown/blob/69ce94ec161d5dc1b710ed3c7229c53ceb9ebea3/R/render.R#L912-L919

When intermediate dir is set, input will be the md file and will be in the intermediate directory. output will be the tex file. If no shell char in the name, the conversion will happen directly (call to pandoc_convert()) and the .tex file will be created in the intermediate directory. The next step will be https://github.com/rstudio/rmarkdown/blob/69ce94ec161d5dc1b710ed3c7229c53ceb9ebea3/R/render.R#L971 but texfile is still the filename (test.tex) and will be in the intermediate directory after md conversion. Current working dir will be the document one so patch_tex_output(texfile) does not find the file. Hence the issue.

On Windows, when intermediate dir is set, output will be a full path and so it will start with something like 

C:/...

This means it contains a shell char and the rendering will happen in a temp directory, which means no issue as everything will be in the current working dir https://github.com/rstudio/rmarkdown/blob/69ce94ec161d5dc1b710ed3c7229c53ceb9ebea3/R/render.R#L924-L955

Hence no issue.

Thanks for the report on this one. It is among the issue with all the path handling internally.

I would really suggest using as less as possible the intermediate_dir and other argument related to path. rmarkdown::render() works best when inside the same directory.