JSS article throw error upon knitting

cimentadaj commented 3 years ago

By filing an issue to this repo, I promise that

[X] I have fully read the issue guide at https://yihui.name/issue/.
[X] I have provided the necessary information about my issue.
- If I'm asking a question, I have already asked it on Stack Overflow or RStudio Community, waited for at least 24 hours, and included a link to my question there.
- If I'm filing a bug report, I have included a minimal, self-contained, and reproducible example, and have also included xfun::session_info('rticles'). I have upgraded all my packages to their latest versions (e.g., R, RStudio, and R packages), and also tried the development version: remotes::install_github('rstudio/rticles').
- If I have posted the same issue elsewhere, I have also mentioned it in this issue.
[X] I have learned the Github Markdown syntax, and formatted my issue correctly.

I understand that my issue may be closed if I don't fulfill my promises.

I'm trying to knit a fresh template of JSS and I get a LaTeX error. I remember knitting this template a few months ago and everything was fine. Here's the template:

---
documentclass: jss
author:
  - name: FirstName LastName
    affiliation: University/Company
    # use this syntax to add text on several lines
    address: |
      | First line
      | Second line
    email: \email{name@company.com}
    url: http://rstudio.com
  - name: Second Author
    affiliation: 'Affiliation \AND'
    # To add another line, use \AND at the end of the previous one as above
  - name: Third Author
    address: |
      | Department of Statistics and Mathematics,
      | Faculty of Biosciences,
      | Universitat Autònoma de Barcelona
    affiliation: |
      | Universitat Autònoma 
      | de Barcelona
    # use a different affiliation in adress field (differently formated here)
    affiliation2: Universitat Autònoma de Barcelona
title:
  formatted: "A Capitalized Title: Something about a Package \\pkg{foo}"
  # If you use tex in the formatted title, also supply version without
  plain:     "A Capitalized Title: Something about a Package foo"
  # For running headers, if needed
  short:     "\\pkg{foo}: A Capitalized Title"
abstract: >
  The abstract of the article.
keywords:
  # at least one keyword must be supplied
  formatted: [keywords, not capitalized, "\\proglang{Java}"]
  plain:     [keywords, not capitalized, Java]
preamble: >
  \usepackage{amsmath}
output: rticles::jss_article
---

```{r, setup, include=FALSE}
options(tinytex.verbose = TRUE)
options(prompt = 'R> ', continue = '+ ')```

# Introduction

This template demonstrates some of the basic LaTeX that you need to know to create a JSS article.

## Code formatting

In general, don't use Markdown, but use the more precise LaTeX commands instead:

* \proglang{Java}
* \pkg{plyr}

One exception is inline code, which can be written inside a pair of backticks (i.e., using the Markdown syntax).

If you want to use LaTeX commands in headers, you need to provide a `short-title` attribute. You can also provide a custom identifier if necessary. See the header of Section \ref{r-code} for example.

# \proglang{R} code {short-title="R code" #r-code}

Can be inserted in regular R markdown blocks.

```{r}
x <- 1:10
x```

## Features specific to \pkg{rticles} {short-title="Features specific to rticles"}

* Adding short titles to section headers is a feature specific to \pkg{rticles} (implemented via a Pandoc Lua filter). This feature is currently not supported by Pandoc and we will update this template if [it is officially supported in the future](https://github.com/jgm/pandoc/issues/4409).
* Using the `\AND` syntax in the `author` field to add authors on a new line. This is a specific to the `rticles::jss_article` format.

I saved that as jss.Rmd and here's the output:

processing file: jss.Rmd
  |..............                                                        |  20%
  ordinary text without R code

  |............................                                          |  40%
label: setup (with options) 
List of 1
 $ include: logi FALSE

  |..........................................                            |  60%
  ordinary text without R code

  |........................................................              |  80%
label: unnamed-chunk-1
  |......................................................................| 100%
  ordinary text without R code

/usr/bin/pandoc +RTS -K512m -RTS jss.utf8.md --to latex --from markdown+autolink_bare_uris+tex_math_single_backslash --output jss.tex --lua-filter /usr/local/lib/R/site-library/rmarkdown/rmarkdown/lua/pagebreak.lua --lua-filter /usr/local/lib/R/site-library/rmarkdown/rmarkdown/lua/latex-div.lua --self-contained --template /home/jorge/R/x86_64-pc-linux-gnu-library/4.0/rticles/rmarkdown/templates/jss/resources/template.tex --highlight-style tango --pdf-engine pdflatex --natbib --lua-filter /home/jorge/R/x86_64-pc-linux-gnu-library/4.0/rticles/rmarkdown/lua/short-title.lua 
output file: jss.knit.md

This is pdfTeX, Version 3.14159265-2.6-1.40.21 (TeX Live 2020) (preloaded format=pdflatex)
 restricted \write18 enabled.
entering extended mode
I was unable to find any missing LaTeX packages from the error log jss.log.
! Undefined control sequence.
\hyper@linkurl ...tionraw >>}\relax \Hy@colorlink 
                                                  \@urlcolor #1\Hy@xspace@en...
l.56 \begin{document}

Error: LaTeX failed to compile jss.tex. See https://yihui.org/tinytex/r/#debugging for debugging tips. See jss.log for more info.
Execution halted

I saw that this error could be related to the email field but this is the default template from RStudio for JSS, so I don't understand where this is happening. Just in case, I removed the email field and searched for any hyperlinks and can't find where this error is coming from.

Here's the session info for rticles:

R version 4.0.2 (2020-06-22)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 20.04.1 LTS, RStudio 1.3.1093

Locale:
  LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C               LC_TIME=de_DE.UTF-8        LC_COLLATE=en_US.UTF-8    
  LC_MONETARY=de_DE.UTF-8    LC_MESSAGES=en_US.UTF-8    LC_PAPER=de_DE.UTF-8       LC_NAME=C                 
  LC_ADDRESS=C               LC_TELEPHONE=C             LC_MEASUREMENT=de_DE.UTF-8 LC_IDENTIFICATION=C       

Package version:
  base64enc_0.1.3 digest_0.6.25   evaluate_0.14   glue_1.4.2      graphics_4.0.2  grDevices_4.0.2 highr_0.8       htmltools_0.5.0
  jsonlite_1.7.1  knitr_1.30      magrittr_1.5    markdown_1.1    methods_4.0.2   mime_0.9        rlang_0.4.7     rmarkdown_2.4  
  rticles_0.16    stats_4.0.2     stringi_1.5.3   stringr_1.4.0   tinytex_0.26    tools_4.0.2     utils_4.0.2     xfun_0.18      
  yaml_2.2.1

I have the latest tinytex installation as well. I also tried running other articles (JOSS) and everything works fine.

cderv commented 3 years ago

I can reproduce this locally, but it is not happening on travis. I wonder why. 🤔 I am looking into this

cderv commented 3 years ago

Ok I was able to find out that this works with an old version of texlive but does not work with the last texlive version installed with tinytex 📦 I believe it works on CI because TinyTeX may be cached there...

cimentadaj commented 3 years ago

I see, let me try it out with a previous texlive version. Which version did you try?

cderv commented 3 years ago

I am currently on tlmgr revision 56458 (2020-09-28 00:20:18 +0200)

cimentadaj commented 3 years ago

That's odd, I have the same revision/time number and I can't make this work:

~$: tlmgr --version
tlmgr revision 56458 (2020-09-28 00:20:18 +0200)
tlmgr using installation: /home/jorge/.TinyTeX
TeX Live (https://tug.org/texlive) version 2020

cderv commented 3 years ago

Oh yeah sorry. I can't make this work with this one. But when I tried on another environment, it worked and after updating TinyTeX it did not work. Unfortunatly I can't find the version I had... :(

cderv commented 3 years ago

Ok I took advantage of TinyTeX new versioned released

It works after install version of september: tinytex::install_tinytex(version = "2020.09.19")
It does not work after installing last version tinytex::install_tinytex(version = "2020.10")

Not yet sure what happens between both...

cimentadaj commented 3 years ago

I can reproduce this. Version 2020.09.19 works as well.

noeliarico commented 3 years ago

Hi! (specially to @cderv as is the second time I run into him today haha)

I'm getting the same error in RStudio Cloud and it does not go anywhere after I do tinytex::install_tinytex(version = "2020.09.19"), any ideas?

cderv commented 3 years ago

@noeliarico I don't know why this is difference in RStudio Cloud. I just tested and indeed 2020.09.19 is not working on RStudio cloud. 🤷‍♂️

This is an odd issue, I am a bit short now - I leave @yihui throw some ideas for this.

Anyway, if I come up with something I'll post here.

stefvanbuuren commented 3 years ago

For what it's worth. I had a working system, issued tinytex::tlmgr(), and then I got the error reported above on the same .tex file that previously worked. Used tinytex 0.25, rticles 0.15. When I changed to output: pdf_document the error disappeared (though of course the nice JSS formatting was gone).

cderv commented 3 years ago

Thanks for sharing.

When I changed to output: pdf_document the error disappeared (though of course the nice JSS formatting was gone).

Regarding this, yes this is something happening (like a conflict) with the jss.cls and its template with last tlmgr version.

stefvanbuuren commented 3 years ago

Saw a warning on line 466 of jss.cls. Do not remember exactly, but was something about double declaration of a package. Perhaps that could shine a light.

jdblischak commented 3 years ago

Just to add another data point, I was also able to fix the issue by following the advice from @cderv to install a previous release of TinyTeX.

Error message:

! Undefined control sequence.
\hyper@linkurl ...tionraw >>}\relax \Hy@colorlink 
                                                  \@urlcolor #1\Hy@xspace@en...
l.69 \begin{document}

Initial texlive version:

> tinytex::tlmgr("--version")
tlmgr --version
tlmgr revision 56566 (2020-10-06 05:40:54 +0200)
tlmgr using installation: C:/Users/john/AppData/Roaming/TinyTeX
TeX Live (https://tug.org/texlive) version 2020

Solution:

tinytex::reinstall_tinytex(version = "2020.09.19")

Note that during the reinstallation, it ran tlmgr update --self. So the final version ended up being the same:

> tinytex::tlmgr("--version")
tlmgr --version
tlmgr revision 56566 (2020-10-06 05:40:54 +0200)
tlmgr using installation: C:/Users/john/AppData/Roaming/TinyTeX
TeX Live (https://tug.org/texlive) version 2020

Thus I have to admit it's unclear to me how this fixed the problem, but it did.

System information:

> xfun::session_info(c("rticles", "tinytex"))
R version 4.0.2 (2020-06-22)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 10 x64 (build 19041), RStudio 1.3.1056

Locale:
  LC_COLLATE=English_United States.1252  LC_CTYPE=English_United States.1252   
  LC_MONETARY=English_United States.1252 LC_NUMERIC=C                          
  LC_TIME=English_United States.1252    

Package version:
  base64enc_0.1.3 digest_0.6.25   evaluate_0.14   glue_1.4.1      graphics_4.0.2  grDevices_4.0.2
  highr_0.8       htmltools_0.5.0 jsonlite_1.7.0  knitr_1.30      magrittr_1.5    markdown_1.1   
  methods_4.0.2   mime_0.9        rlang_0.4.7     rmarkdown_2.3   rticles_0.16    stats_4.0.2    
  stringi_1.4.6   stringr_1.4.0   tinytex_0.26    tools_4.0.2     utils_4.0.2     xfun_0.15      
  yaml_2.2.1

yihui commented 3 years ago

I guess a recent update of a certain LaTeX package broke it (but I don't know which). The breakage shouldn't be due to tlmgr itself, which is why tlmgr update --self doesn't matter.

To figure out which package(s) broke it, we need to compare the output of

tinytex::tlmgr('info --list --only-installed --data name,cat-version')

between TinyTeX 2020.09.19 and 2020.10. You can paste the output to https://www.diffchecker.com and find the diffs.

After we identify the possible culprit, we may need to contact the author of the package.

Another possible way to go is to check if jss.cls in rticles is the latest version (https://www.jstatsoft.org/pages/view/style). If not, we need to test against the latest version. If it doesn't work, we'll have to either contact the author of the LaTeX package that broke jss.cls, or contact JSS and see if they could with the latest version of LaTeX packages on CTAN.

I don't have time for these tasks myself, so will appreciate it if someone could help out. Thanks!

yihui commented 3 years ago

To further pin down the offending LaTeX package, you may try to install TinyTeX built between 2020.09.19 and 2020.10.04 The daily builds can be found at https://ci.appveyor.com/project/yihui/tinytex/history. Go to the Artifacts tab and download TinyTeX-1.zip, e.g., https://ci.appveyor.com/project/yihui/tinytex/builds/35428252/job/0ihcbltadyvu3myt/artifacts Then pass the path of this zip file to tinytex::install_prebuilt() to install it.

You can manually bisect these daily builds until you find the day when it was broken. This may be time-consuming, though (with 15 daily builds, I think you need to bisect 3 times).

cderv commented 3 years ago

Thanks for the hints @yihui ! That will definitly be helpful.

Also for everyone wanted to help:

Another possible way to go is to check if jss.cls in rticles is the latest version (https://www.jstatsoft.org/pages/view/style). If not, we need to test against the latest version.

We have version 3.0 from 2015 but I tested again the last one (version 3.1 from 2020/08/03) and we get the same result. So once we find the culprit, there may be a fix to push into jss.cls

cderv commented 3 years ago

OK I found the culprit. Not a small one !

I followed the advice on looking at the diff of the result of

tinytex::tlmgr('info --list --only-installed --data "name,cat-version"')

There was a few packages so I decide to test by installing them one by one starting with the more obvious one: latex

Install previous version

tinytex::install_tinytex(version = "2020.09.19")

The JSS document compiles correctly

Update latex

tinytex::tlmgr_update(all = FALSE, self = FALSE, "latex")

Try recompiling : You got the error !

There has been an update of latex package on CTAN on the 01/10/2020: https://www.latex-project.org/news/2020/10/01/issue32-of-latex2e-released/ and this is the one that break things. Something has changed in the hook management, including \AtBeginDocument which is used by jss.cls

So I guess this is the jss.cls file that has an issue and an issue should be opened with JSS on this one.

If someone want to look deeper from here on what has changed, please do !

Hope it helps.

yihui commented 3 years ago

@cderv Awesome! Thank you! In this case, I think we need to report to JSS, because they'll have to fix it sooner or later (I'm not sure if @statibk subscribed to Github notifications).

cderv commented 3 years ago

Hi @cimentadaj , @jdblischak , @noeliarico , @stefvanbuuren

We fixed the issue with template by pinning down the version of the problematic package. It is not perfect but at least it will now render. You need to install the last rticles version from github.

There was another issue with the latexrelease update in #331 and the editor of Copernicus journal are working on a fix for their class. The same should be done with JSS editors.

Has one of you already email the JSS editors regarding this issue ?

stefvanbuuren commented 3 years ago

Just sent a note to Achim Zeileis.

zeileis commented 3 years ago

Thanks for the pointer! I don't get the problem with my system TeXLive (I'm on Debian testing with tlmgr revision 56372), yet, but I guess it should arrive soon enough. I didn't have the time/resources to install a separate TinyTeX version so will have to wait with debugging the issue.

Or did anyone find out already what exactly caused the problem? This isn't clear to me from the above discussion. Further pointers would be much appreciated.

P.S.: I'm using this GitHub account now for which I do get notifications.

yihui commented 3 years ago

@zeileis Yes, @cderv found out a fix: https://github.com/rstudio/rticles/pull/337/files I'm not sure if it's the best solution, though.

cderv commented 3 years ago

Or did anyone find out already what exactly caused the problem? This isn't clear to me from the above discussion. Further pointers would be much appreciated.

@zeileis We isolate the issue to this specific package latexrelease update: https://www.latex-project.org/news/2020/10/01/issue32-of-latex2e-released/ See https://github.com/rstudio/rticles/issues/329#issuecomment-705713973

The current workaround is to pin down the version in the template. This was also the workaround for this other issue (https://github.com/rstudio/rticles/issues/331) where Copernicus Journal editor seems to work on a fix (https://github.com/rstudio/rticles/issues/331#issuecomment-709941348)

I do not know what exactly is causing the issue with the new version but I believe this is something to do with the use of \AtBeginDocument as there were some changes with the hook management system for LaTeX in the new latexrelease version.

zeileis commented 3 years ago

Thank you @yihui and @cderv for the follow-up. I understood that your fix is to not use the latest version - but ideally I would want to provide a jss.cls file that cooperates with both versions of the package. And I'm not sure how to do that.

I took some time today to investigate this issue further and can confirm that it is indeed related to \AtBeginDocument, more specifically \AtBeginDocument{\maketitle}. If I this is in the jss.cls file, I get the error. However, if I put it into the article.tex file, it works as intended. So a workaround that can currently be used is to do

\documentclass[article,notitle]{jss}
\AtBeginDocument{\maketitle}
...

Note the notitle option in the \documentclass.

What I still need to find out is which part of the \maketitle definition in jss.cls is not compatible with the new hook. Probably one of the variables that I define and then want to use which are now defined at a later point.

I'll let you know if I find out more about this. Thanks again for your work/insights regarding this issue!

zeileis commented 3 years ago

OK, I think I have a better solution, albeit possibly still not ideal. When I replace the line from jss.cls

\AtBeginDocument{\maketitle}

with

\@ifundefined{AddToHook}{\AtBeginDocument{\maketitle}}{\AddToHook{begindocument}[maketitle]{\maketitle}}

then it appears to work for both old and new LaTeX versions.

Does anyone here know whether this is a good/reliable way of checking for the availability of \AddToHook?

zeileis commented 3 years ago

Just for the record: The fix posted above has now been officially released as version 3.2 of jss.cls. The updated style files are available both from https://www.jstatsoft.org/pages/view/style and in the base R sources:

https://github.com/wch/r-source/commit/3d6893497f18e62e4f058ec1953ac2df4b121dcf#diff-40d315f66f364ab42b39fbf99b30ddebea15425296b4c844827d478059fe061b

yihui commented 3 years ago

@zeileis That's great to know! Thanks for posting back!

@cderv Please update jss.cls in our package accordingly when you have a chance. Thanks!

cderv commented 3 years ago

That is great ! I'll do that !

cderv commented 3 years ago

jss.cls is now updated in dev version of the package.

zeileis commented 3 years ago

Perfect, thanks a lot for the rapid update!

nbenn commented 3 years ago

@cderv Do you have any guidance on how to deal with this issue for older R versions in a gh-actions setting? Currently, for a repo of mine, release runs through fine, while oldrel fails with the OP's error. I tried freezing TinyTex by setting

- uses: r-lib/actions/setup-tinytex@v1
  env:
    TINYTEX_VERSION: 2020.09.19

but this resulted in other issues, something along the lines of

tlmgr: Local TeX Live (2020) is older than remote repository (2021).
Cross release updates are only supported with
  update-tlmgr-latest(.sh/.exe) --update
See https://tug.org/texlive/upgrade.html for details.

cderv commented 3 years ago

@nbenn This issue has been fixed at the end with an update of the jss.cls in rticles. Are you using this updated version of the R package ? It should work with last TeX Live version. You could also get the last jss.cls from JSS website: https://www.jstatsoft.org/pages/view/style

Also, the error you see with TeX Live and update is that it gets updated each year and all CTAN mirror will work with the new version so it is my understanding that you can't install packages from a new CTAN mirror with an old TeX Live version.

nbenn commented 3 years ago

@cderv Thanks for getting back to me. Also, if you feel SO would be a better place for me to ask questions like this, I'm happy to be redirected.

I tried isolating my issue in a standalone package: nbenn/rticles329, created as

usethis::create_package("rticles329")
usethis::use_mit_license()
rmarkdown::draft("vignettes.Rmd", template = "jss", package = "rticles",
                 edit = FALSE)

rmd <- readLines("vignettes/vignettes.Rmd")
ind <- grep("---", rmd)[2L]
rmd <- c(
   rmd[seq_len(ind - 1L)],
   "vignette: >",
   "  %\\VignetteIndexEntry{A Capitalized Title}",
   "  %\\VignetteEngine{knitr::rmarkdown}",
   "  %\\VignetteEncoding{UTF-8}",
   rmd[seq.int(ind, length(rmd))]
)
write(rmd, file = "vignettes/vignettes.Rmd")

usethis:::use_dependency("knitr", "Suggests")
usethis:::use_dependency("rmarkdown", "Suggests")
usethis:::use_dependency("rticles", "Suggests")
usethis:::use_description_field("VignetteBuilder", "knitr", overwrite = TRUE)
usethis:::use_git_ignore("inst/doc")

usethis::use_github_action("build",
   url = paste("https://raw.githubusercontent.com/eth-mds/ricu",
               "f7558ed99017dcd3a36ef2a5c92a9c8043621472/.github/workflows",
               "build.yaml", sep = "/")
)

One thing I don't fully understand, is which jss.cls file is used in the end. From what I have observed so far, it seems that if available (i.e. not build-ignored), the package-local jss.cls is used and if build-ignored, this defaults to the jss.cls file that comes with R.

This leaves me with the choice of either having a note as

❯ checking files in ‘vignettes’ ... NOTE
  The following files are already in R: ‘jss.bst’, ‘jss.cls’
  Please remove them from your package.

on all R versions during checking (see this run) or build-ignoring the files as

usethis::use_build_ignore("^vignettes/jss\\.(cls|bst)$", escape = FALSE)

and then all is fine with R-release, but R-oldrel fails (see this run). Is there any way out of this conundrum?

github-actions[bot] commented 2 years ago

This old thread has been automatically locked. If you think you have found something related to this, please open a new issue by following the issue guide (https://yihui.org/issue/), and link to this old issue if necessary.

rstudio / rticles

JSS article throw error upon knitting #329