feature request: tweak non-escaped ampersand

HedvigS commented 3 months ago

By filing an issue to this repo, I promise that

[ X] I have fully read the issue guide at https://yihui.org/issue/.
[ X] I have provided the necessary information about my issue.
- If I'm asking a question, I have already asked it on Stack Overflow or RStudio Community, waited for at least 24 hours, and included a link to my question there.
- If I'm filing a bug report, I have included a minimal, self-contained, and reproducible example, and have also included xfun::session_info('knitr'). I have upgraded all my packages to their latest versions (e.g., R, RStudio, and R packages), and also tried the development version: remotes::install_github('yihui/knitr').
- If I have posted the same issue elsewhere, I have also mentioned it in this issue.
[ X] I have learned the Github Markdown syntax, and formatted my issue correctly.

I understand that my issue may be closed if I don't fulfill my promises.

For reasons beyond my understanding, some bibtex entries generated by citation() from Description files have non-escaped ampersands - which causes problems with LaTeX later.

See for example spam2017 below.


> knitr::write_bib(x = "spam")
@Manual{R-spam,
  title = {spam: SPArse Matrix},
  author = {Reinhard Furrer and Florian Gerber and Roman Flury},
  year = {2022},
  note = {R package version 2.9-1},
  url = {https://www.math.uzh.ch/pages/spam/},
}

@Article{spam2010,
  title = {{spam}: A Sparse Matrix {R} Package with Emphasis on {MCMC} Methods for {G}aussian {M}arkov Random Fields},
  author = {Reinhard Furrer and Stephan R. Sain},
  journal = {Journal of Statistical Software},
  year = {2010},
  volume = {36},
  number = {10},
  pages = {1--25},
  doi = {10.18637/jss.v036.i10},
}

@Article{spam2015,
  title = {Pitfalls in the Implementation of {B}ayesian Hierarchical Modeling of Areal Count Data: An Illustration Using {BYM} and {L}eroux Models},
  author = {Florian Gerber and Reinhard Furrer},
  journal = {Journal of Statistical Software, Code Snippets},
  year = {2015},
  volume = {63},
  number = {1},
  pages = {1--32},
  doi = {10.18637/jss.v063.c01},
}

@Article{spam2017,
  title = {Extending {R} packages to support 64-bit compiled code: An illustration with spam64 and {GIMMS} {NDVI3g} data},
  author = {Florian Gerber and Kaspar Moesinger and Reinhard Furrer},
  journal = {Computer & Geoscience},
  year = {2017},
  volume = {104},
  pages = {109--119},
  issn = {0098-3004},
  doi = {10.1016/j.cageo.2016.11.015},
}

> xfun::session_info('knitr')
R version 4.3.3 (2024-02-29)
Platform: aarch64-apple-darwin20 (64-bit)
Running under: macOS Ventura 13.4.1, RStudio 2023.12.1.402

Locale: en_US.UTF-8 / en_US.UTF-8 / en_US.UTF-8 / C / en_US.UTF-8 / en_US.UTF-8

time zone: Europe/Berlin
tzcode source: internal

Package version:
  evaluate_0.21   graphics_4.3.3  grDevices_4.3.3 highr_0.10      knitr_1.43      methods_4.3.3   stats_4.3.3     tools_4.3.3    
  utils_4.3.3     xfun_0.39       yaml_2.3.7

Currently, I'm doing a fairly "hacky" solve to this by reading in the output from write_bib and then doing some find and replace with stringr.

knitr::write_bib(x = pkgs_to_cite, file = output_fn)

readLines(output_fn) %>% 
  str_replace_all("\\&", "\\\\&") %>%
  str_replace_all("\\\\\\\\&", "\\\\&") %>% 
    writeLines(output_fn)

Could knitr::write_bib please check for non-escpaped ampersands and do this tweak if necessary?

Thank you! Love the package!

cderv commented 3 months ago

Thanks for the report.

This is indeed an issue with natbib for example, but not with pandoc citeproc

---
title: "test"
output: 
  pdf_document:
    keep_tex: true
    citation_package: natbib
bibliography: packages.bib
---

```{r}
knitr::write_bib(x = "spam", "packages.bib")

@spam2017


leads to

! Misplaced alignment tab character &.

Computer & Geoscience l.11 \newblock \emph{Computer & Geoscience} , 104:\penalty0 109--119, 2017. ```` while no error if the default is used (meaning Pandoc citeproc handles the citation processing for PDF) > For reasons beyond my understanding, some bibtex entries generated by citation() from Description files have non-escaped ampersands Though, as you noticed, this comes directly from `citation()` converted by `toBibTex()` which does no escaping. They have a `escape = FALSE` by default - see [`?bibentry`](https://stat.ethz.ch/R-manual/R-devel/library/utils/html/bibentry.html) but it seems it covers only non-ASCII > escape : a logical indicating whether non-ASCII characters should be translated to LaTeX escape sequences. It doesn't not help here with `&` ``` r toBibtex(citation("spam"), espace = TRUE) #> @Manual{, #> year = {2023}, #> title = {{spam}: SPArse Matrix}, #> author = {Reinhard Furrer and Roman Flury and Florian Gerber}, #> note = {R package version 2.10-0}, #> url = {https://CRAN.R-project.org/package=spam}, #> } #> #> @Article{, #> title = {{spam}: A Sparse Matrix {R} Package with Emphasis on {MCMC} Methods for {G}aussian {M}arkov Random Fields}, #> author = {Reinhard Furrer and Stephan R. Sain}, #> journal = {Journal of Statistical Software}, #> year = {2010}, #> volume = {36}, #> number = {10}, #> pages = {1--25}, #> doi = {10.18637/jss.v036.i10}, #> } #> #> @Article{, #> title = {Pitfalls in the Implementation of {B}ayesian Hierarchical Modeling of Areal Count Data: An Illustration Using {BYM} and {L}eroux Models}, #> author = {Florian Gerber and Reinhard Furrer}, #> journal = {Journal of Statistical Software, Code Snippets}, #> year = {2015}, #> volume = {63}, #> number = {1}, #> pages = {1--32}, #> doi = {10.18637/jss.v063.c01}, #> } #> #> @Article{, #> title = {Extending {R} packages to support 64-bit compiled code: An illustration with spam64 and {GIMMS} {NDVI3g} data}, #> author = {Florian Gerber and Kaspar Moesinger and Reinhard Furrer}, #> journal = {Computer & Geoscience}, #> year = {2017}, #> volume = {104}, #> pages = {109--119}, #> issn = {0098-3004}, #> doi = {10.1016/j.cageo.2016.11.015}, #> } ``` It seems to be an issue in R utils function after all. 🤔 @yihui what do you think ?

yihui commented 3 months ago

We can probably escape & when there isn't a backslash before it (using negative lookbehind (?<!) in Perl regex).

gsub('(?<!\\\\)&', '\\\\&', x, perl = TRUE)

We need to verify if pandoc-citeproc is also fine with \& in .bib. If that's the case, please feel free to submit a pull request. Thanks!

HedvigS commented 3 months ago

We can probably escape & when there isn't a backslash before it (using negative lookbehind (?<!) in Perl regex).
gsub('(?<!\\\\)&', '\\\\&', x, perl = TRUE)
We need to verify if pandoc-citeproc is also fine with \& in .bib. If that's the case, please feel free to submit a pull request. Thanks!

Ok! Is that an encouragement to me or @cderv to submit a PR?

cderv commented 3 months ago

Is that an encouragement to me or @cderv to submit a PR?

Either of us I would say :)

Feel free to contribute if you are interested. Do you want to give it a try ?

HedvigS commented 3 months ago

Is that an encouragement to me or @cderv to submit a PR?

Either of us I would say :)

Feel free to contribute if you are interested. Do you want to give it a try ?

I could... but.. I have a deadline for a paper on Sunday and 53 exams to grade and I haven't edited knitr source code before. I think you're probably going to do it better and faster ;)

cderv commented 3 months ago

I totally get the workload you're juggling right now – the paper deadline and all those exams to grade must be keeping you super busy! Don't worry; It was appreciated you were willing to. We'll do it. Thanks again for opening the issue

HedvigS commented 3 months ago

I totally get the workload you're juggling right now – the paper deadline and all those exams to grade must be keeping you super busy! Don't worry; It was appreciated you were willing to. We'll do it. Thanks again for opening the issue

Thanks!!

yihui / knitr

feature request: tweak non-escaped ampersand #2335