citation_style report: CSL short form of 'no date' does not render on the website as intended

[florisvdh]

The csl defines a short form of the term "no date" as s.d., both for English and Dutch. When rendering from Rstudio (below pandoc statement is run), this locally results in the citation "Databank Ondergrond Vlaanderen (s.d.).".

/usr/lib/rstudio/bin/pandoc/pandoc +RTS -K512m -RTS ./index.split.md ./010_making_bib.split.md ./100_references.split.md --to html4 --from markdown+autolink_bare_uris+tex_math_single_backslash --output citation_style.html --email-obfuscation none --wrap preserve --csl /home/floris/lib/R/library/INBOmd/research-institute-for-nature-and-forest.csl --standalone --section-divs --table-of-contents --toc-depth 3 --template css/gitbook.html --highlight-style pygments --number-sections --include-in-header /tmp/Rtmp8rS0F7/rmarkdown-str14817c0b00b9.html --mathjax --lua-filter /home/floris/lib/R/library/rmarkdown/rmd/lua/pagebreak.lua --lua-filter /home/floris/lib/R/library/rmarkdown/rmd/lua/latex-div.lua --file-scope --filter /usr/lib/rstudio/bin/pandoc/pandoc-citeproc

However it appears to render on the website (5th bullet here) as "n.d.".

It may have to do with a different locale used in the build, and probably defining a locale fallback in the csl will then be the easiest way to solve this.

@ThierryO How can we test this for the build? Will I put the csl tweak in a PR of INBOmd, after which you merge in INBOmd master and retrigger the build here?

I'll also make a few updates of the report itself, regarding the issues at BBT.

[ThierryO]

They easiest way to test it "locally" is through Docker. The Docker image and the scripts are listed in https://github.com/inbo/inbomd_examples/blob/master/wercker.yml

[ThierryO]

The locale in the Docker image is en_US.UTF-8. This is set by one of the parent images. Maybe you try to set the locale on your machine to en_US.UTF-8 and see if you can reproduce it.

[florisvdh]

I'll first try the second option from a shell session.

[florisvdh]

I could not reproduce it locally (with shell session as below); still shows 's.d.' here. And the bibliography correctly comes up with English terms.

Code and output

```bash floris@inbofloris citation_style $ export LC_ALL="en_US.UTF-8" floris@inbofloris citation_style $ export LANG="en_US.UTF-8" floris@inbofloris citation_style $ export LANGUAGE="en_US:en" floris@inbofloris citation_style $ locale LANG=en_US.UTF-8 LANGUAGE=en_US:en LC_CTYPE="en_US.UTF-8" LC_NUMERIC="en_US.UTF-8" LC_TIME="en_US.UTF-8" LC_COLLATE="en_US.UTF-8" LC_MONETARY="en_US.UTF-8" LC_MESSAGES="en_US.UTF-8" LC_PAPER="en_US.UTF-8" LC_NAME="en_US.UTF-8" LC_ADDRESS="en_US.UTF-8" LC_TELEPHONE="en_US.UTF-8" LC_MEASUREMENT="en_US.UTF-8" LC_IDENTIFICATION="en_US.UTF-8" LC_ALL=en_US.UTF-8 floris@inbofloris citation_style $ cat render_csl.R rmarkdown::render_site( output_format = bookdown::gitbook( split_by = "chapter+number", template = INBOmd::inbo_rapport_css("html"), pandoc_args = c( "--csl", system.file( "research-institute-for-nature-and-forest.csl", package = "INBOmd" ) ) ) ) floris@inbofloris citation_style $ Rscript render_csl.R processing file: citation_style.Rmd |.......... | 14% ordinary text without R code |.................... | 29% label: unnamed-chunk-1 (with options) List of 2 $ echo : logi FALSE $ results: chr "asis" |.............................. | 43% ordinary text without R code |........................................ | 57% label: unnamed-chunk-2 (with options) List of 3 $ echo : logi FALSE $ eval : language !natbib $ engine: chr "asis" |.................................................. | 71% inline R code fragments |............................................................ | 86% label: unnamed-chunk-3 (with options) List of 3 $ eval : language !natbib $ echo : logi FALSE $ results: chr "asis" |......................................................................| 100% ordinary text without R code output file: citation_style.knit.md /usr/lib/rstudio/bin/pandoc/pandoc +RTS -K512m -RTS ./index.split.md ./010_making_bib.split.md ./100_references.split.md --to html4 --from markdown+autolink_bare_uris+tex_math_single_backslash --output citation_style.html --email-obfuscation none --wrap preserve --csl /home/floris/lib/R/library/INBOmd/research-institute-for-nature-and-forest.csl --standalone --section-divs --table-of-contents --toc-depth 3 --template css/gitbook.html --highlight-style pygments --number-sections --include-in-header /tmp/RtmppyCzXh/rmarkdown-str2a0e6fc10bf6.html --mathjax --lua-filter /home/floris/lib/R/library/rmarkdown/rmd/lua/pagebreak.lua --lua-filter /home/floris/lib/R/library/rmarkdown/rmd/lua/latex-div.lua --file-scope --filter /usr/lib/rstudio/bin/pandoc/pandoc-citeproc Output created: ../../docs/citation_style/index.html ```

[ThierryO]

What do you get when you make the Rmd file display the output of Sys.getlocale(). Just to make sure that render uses the locale you set at the CL.

[florisvdh]

What I could find:

• the system locale (which coincides with feedback from Sys.getlocale()) does not influence the language that is used by pandoc for citations. Hence, with current Rmd content it always falls back to English as a default. And at least locally, this means "s.d." (because it is defined as such for 'en' and 'nl' in our csl file) - but not on the website. I ran this with pandoc 2.3.1 (in older RStudio on bionic) and pandoc 2.7.3 (current Rstudio, on focal).
• setting the language for pandoc is done with lang: in the YAML header, see here.
  • just lang: en, or lang: en-XY (US, GB, AU) all gave the "s.d." result. (hence, same as without lang)
  • lang: nl indeed switched to Dutch bibliography
• The only CSL locales (see repo) where 'n.d.' is defined (and where 'n.d.' must come from), are:

./locales-af-ZA.xml:45:    <term name="no date" form="short">n.d.</term>
./locales-en-GB.xml:54:    <term name="no date" form="short">n.d.</term>
./locales-en-US.xml:54:    <term name="no date" form="short">n.d.</term>
./locales-km-KH.xml:45:    <term name="no date" form="short">n.d.</term>
./locales-mn-MN.xml:45:    <term name="no date" form="short">n.d.</term>
./locales-sk-SK.xml:51:    <term name="no date" form="short">n.d.</term>

Presumably the 'n.d.' result on the website just comes from en-GB or en-US, but anyhow it's weird that the English locale setting for this term in the csl is ignored. Likewise, "edn." (defined in the csl file) is not used but instead the English default "ed." for the term 'edition'. It's not clear whether explicitly setting lang: en would affect this for the result with the docker image.

For further experimenting with this (e.g. lang: en), the docker image would be needed indeed, though I'm hesitating to let it take my precious disk space ;-).

Anyhow, to me this remains quite a mystery. I can only think of pandoc doing this 'somehow' in the docker image (I think this runs debian buster, but it probably uses the same rstudio pandoc?). Maybe you have other ideas.

[florisvdh]

@ThierryO , in the docker image, what is the (shell) output of:

which pandoc
which pandoc-citeproc
which rstudio
rstudio --version
pandoc --version
pandoc-citeproc --version

I believe the inbomd_examples repo should reproduce what happens for the RStudio user, i.e. by using pandoc from RStudio. For shell commands (like Rscript), I set the PATH variable such that it first finds pandoc from RStudio, i.e. from /usr/lib/rstudio/bin/pandoc/.

[florisvdh]

Hmm :thinking: :smile: . @ThierryO the described issue seems to be solved? That's great! What did you change?

[ThierryO]

I don't have a clue. Maybe something changed in the parent Docker image?