Error rendering with latest pandoc 1.15.0.6 (Arch Linux)

[chendaniely]

I'm not sure how I make this problem reproducible for others. But I have the latest version of pandoc installed and cannot seem to render html (or any other of the drop down types) in rstudio from a .Rmd document.

> sessionInfo()
R version 3.2.1 (2015-06-18)
Platform: x86_64-unknown-linux-gnu (64-bit)
Running under: Arch Linux

locale:
 [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C               LC_TIME=en_US.UTF-8       
 [4] LC_COLLATE=en_US.UTF-8     LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8   
 [7] LC_PAPER=en_US.UTF-8       LC_NAME=C                  LC_ADDRESS=C              
[10] LC_TELEPHONE=C             LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C       

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

loaded via a namespace (and not attached):
[1] htmltools_0.2.6 tools_3.2.1     yaml_2.1.13     rmarkdown_0.7   digest_0.6.8   

The error that shows when I try to knit a document

Welcome at Sun Aug 16 20:51:23 2015 

*** Successfully loaded .Rprofile ***

pandoc: loadlocale.c:131: _nl_intern_locale_data: Assertion `cnt < (sizeof (_nl_value_type_LC_COLLATE) / sizeof (_nl_value_type_LC_COLLATE[0]))' failed.
Error in strsplit(version_info, "\n")[[1]] : subscript out of bounds
Calls: <Anonymous> ... pandoc_available -> find_pandoc -> lapply -> FUN -> get_pandoc_version
Execution halted

my .Rprofile

$ cat .Rprofile 
q <- function (save="no", ...) {
  quit(save=save, ...)
}

.First <- function(){
 cat("\nWelcome at", date(), "\n")
}

.Last <- function(){
 cat("\nGoodbye at ", date(), "\n")
}

message("\n*** Successfully loaded .Rprofile ***\n")

My pandoc version and location:

$ pandoc -v
pandoc 1.15.0.6
Compiled with texmath 0.8.2.2, highlighting-kate 0.6.
Syntax highlighting is supported for the following languages:
    abc, actionscript, ada, agda, apache, asn1, asp, awk, bash, bibtex, boo, c,
    changelog, clojure, cmake, coffee, coldfusion, commonlisp, cpp, cs, css,
    curry, d, diff, djangotemplate, dockerfile, dot, doxygen, doxygenlua, dtd,
    eiffel, email, erlang, fasm, fortran, fsharp, gcc, glsl, gnuassembler, go,
    haskell, haxe, html, idris, ini, isocpp, java, javadoc, javascript, json,
    jsp, julia, kotlin, latex, lex, lilypond, literatecurry, literatehaskell,
    lua, m4, makefile, mandoc, markdown, mathematica, matlab, maxima, mediawiki,
    metafont, mips, modelines, modula2, modula3, monobasic, nasm, noweb,
    objectivec, objectivecpp, ocaml, octave, opencl, pascal, perl, php, pike,
    postscript, prolog, pure, python, r, relaxng, relaxngcompact, rest, rhtml,
    roff, ruby, rust, scala, scheme, sci, sed, sgml, sql, sqlmysql,
    sqlpostgresql, tcl, tcsh, texinfo, verilog, vhdl, xml, xorg, xslt, xul,
    yacc, yaml, zsh
Default user data directory: /home/dchen/.pandoc
Copyright (C) 2006-2015 John MacFarlane
Web:  http://pandoc.org
This is free software; see the source for copying conditions.
There is no warranty, not even for merchantability or fitness
for a particular purpose.

$ which pandoc
/usr/bin/pandoc

I tried setting self_contained: no per https://github.com/rstudio/rmarkdown/issues/228, but I still get the same error.

[jjallaire]

It looks like this is a recently introduced issue related to locales and glibc in Arch. Googling for the error message yields the following two links (both reports of the same error message on Arch) from the past few days:

http://forum.cyanogenmod.org/topic/113812-build-fails/

http://forum.xda-developers.com/oneplus-one/development/custom-slimsaber-builds-mods-t3175742/post62348751

[jjallaire]

The Arch issue with locales is likely triggered by the following code in rmarkdown (which is there to work around some GHC crashing issues we found with locales):

https://github.com/rstudio/rmarkdown/blob/master/R/pandoc.R#L552-L582

We're stripping a couple of the LC_ environment variables (which caused the previous crashes) and sometimes setting the LANG variable. If you change the implementation of this function to just:

with_pandoc_safe_environment <- function(code) {
   force(code)
}

I'm sure everything will work correctly. Perhaps we need the pandoc locale environment manipulation to be done only when an option (on by default) is set? Alternatively, perhaps there's something about the environment you are calling from that's not standard which could be changed to make things work?

[chendaniely]

running

with_pandoc_safe_environment <- function(code) {
   force(code)
}

in the console and then clicking the knit button still fails. Not sure if that's what you meant.

my locale:

$ locale -a
C
en_US.utf8
POSIX

[jjallaire]

I don't think that you actually overwrote the function when you did that. You need to modify the source of the package and then rebuild it.

If that doesn't solve the problem then it's a bug in Arch (as per the other links I sent) or at least an incompatibility with the latest version of Arch and how we build pandoc. I focus on Arch because this doesn't fail on any other platform that I have tested on and those recent simlar errors found in Google seem quite similar.

I don't have an Arch configuration to test in so will have to rely on users to sort out what's going on and what (if anything) to change in rmarkdown.

On Mon, Aug 17, 2015 at 4:38 PM, Daniel Chen notifications@github.com wrote:

running

with_pandoc_safe_environment <- function(code) { force(code) }

in the console and then clicking the knit button still fails. Not sure if that's what you meant.

my locale:

$ locale -a C en_US.utf8 POSIX

— Reply to this email directly or view it on GitHub https://github.com/rstudio/rmarkdown/issues/498#issuecomment-131955255.

[chendaniely]

I downloaded the source from CRAN and changed the with_pandoc_safe_environment before installing it from a tar. I get the same error

[mkesslerct]

I have the same problem after updating in Arch. The suggested fix changing with_pandoc_safe_environment does not work for me neither. In fact it seems that the pandoc shipped with rstudio does not work anymore: /usr/lib/rstudio/bin/pandoc/pandoc -v gives the same error about loadlocale.c.

[jjallaire]

Okay, this just reinforces the fact that there is an incompatibility between the way we build our "universal" standalone pandoc binary and the current version of Arch. We can't really change the way we build our binary as it relies extensively on the Haskell toolchain (which we don't maintain or understand). So there either needs to be:

1) A fix in Arch; or

2) Arch users need a different pandoc binary (perhaps based on the deb available from pandoc: https://github.com/jgm/pandoc/releases/tag/1.15.0.6)

On Tue, Aug 25, 2015 at 7:13 AM, mkesslerct notifications@github.com wrote:

I have the same problem after updating in Arch. The suggested fix changing with_pandoc_safe_environment does not work for me neither. In fact it seems that the pandoc shipped with rstudio does not work anymore: /usr/lib/rstudio/bin/pandoc/pandoc -v gives the same error about loadlocale.c.

— Reply to this email directly or view it on GitHub https://github.com/rstudio/rmarkdown/issues/498#issuecomment-134556592.

[mkesslerct]

Thanks very much! I have used option 2) and installed from source pandoc using cabal following instructions in [http://pandoc.org/installing.html]. I then created a symbolic link in /usr/lib/rstudio/bin/pandoc/ to the new pandoc executable. The knitting functionality in Rstudio now works again!

[chendaniely]

thanks @jjallaire and @mkesslerct I've confirmed the fix.

My steps (for future reference)

1. I installed pandoc-static from the aur
2. Went into the pandoc directory installed by rstudio: cd /usr/lib/rstudio/bin/pandoc/
3. delete/rename the existing pandoc binary mv pandoc pandoc_rstudio (might need sudo for this)
4. create the symbolic link: ln -s /usr/bin/pandoc /usr/lib/rstudio/bin/pandoc/pandoc

There's already an existing Arch linux fix in the AUR: https://aur.archlinux.org/packages/pandoc-rstudio/

But in my mass googling and trying everything it did not seem to work. I flagged it as out of date. The Aur package was only creating symbolic links anyway, so it seems like it's been hacked/fixed in Arch for a while, just something recent broke the hack.

my pandoc version:

    $ pandoc -v
    pandoc 1.15.0.6

[Oliph]

Thank you @chendaniely, I followed your steps to fix it. A quick question, if I update rstudio later, will it destroy the symlink and update the pandoc itself?

[chendaniely]

@Oliph I'm not sure, it might, but only one way to find out!

[jeroenjanssens]

Thanks @chendaniely, that worked for me. I got an error installing the AUR package, so I ended up installing pandoc using cabal (which takes a while):

$ sudo cabal update
$ sudo cabal install --global pandoc
$ which pandoc
/usr/local/bin/pandoc
$ pandoc -v | head -n 1
pandoc 1.15.0.6

[chendaniely]

@Oliph I just re-installed rstudio using the PKGBUILD and it seems that it does destroy the symlink.

[Oliph]

@chendaniely I just had the same problem, fixed by repeating the above commands

[mingrisch]

@jeroenjanssens, thank you! I have followed your approach:

$sudo pacman -S ghc
$sudo pacman -S cabal-install
$cabal install pandoc

I have then added .cabal/bin to my path in .zshrc:

export PATH=~/.cabal/bin:$PATH

and then created a symbolic link to this pandoc version like @chendaniely

sudo ln -s /home/michi/.cabal/bin/pandoc /usr/lib/rstudio/bin/pandoc/pandoc

Finally, and knitting in RStudio work again.

[dhsvendsen]

@chendaniely Just writing to confirm chendaniely's fix and thank you a bunch. Im new to arch (and very grateful).

[chendaniely]

If you already have pandoc installed, this should be a more general script (will need sudo)

SYS_PANDOC=`which pandoc`
echo $SYS_PANDOC

mv /usr/lib/rstudio/bin/pandoc/pandoc /usr/lib/rstudio/bin/pandoc/pandoc_old

ln -s $SYS_PANDOC /usr/lib/rstudio/bin/pandoc/pandoc

[chmue]

The easiest way to implement this for most Arch users might be to patch the PKGBUILD, for example like this for the rstudio-desktop-bin package I use. This has the added benefit that the files are managed by pacman, so they will be deleted when the package is removed.

EDIT: You maybe saw this already, but the Arch wiki has tips on installing pandoc in Arch. I personally use the precompiled pandoc-static package from Parabola.

[chendaniely]

thanks @chmue. I just made the relevant changes and update to the rstudiodesktop preview AUR package: https://aur.archlinux.org/packages/rstudio-desktop-preview-bin/

[ASalvail]

@chendaniely The changes to the PKGBUILD assume you have pandoc installed in /usr/bin/pandoc. You should add a dependency to pandoc-bin (in the AUR) to make sure that is correct (or maybe add a check for the existence of the file?).

[chmue]

Good catch @ASalvail! I just assumed that /usr/bin/pandoc is available. I would suggest to add a dependency for just pandoc instead of a specific aur package though, because all the packages in the AUR provide pandoc.

[chendaniely]

@chmue how are you making a dependency for pandoc in the PKGBUILD without specifying a specific AUR package?

I got around the /usr/bin/pandoc hard-coded path by using which pandoc and which pandoc-citeproc

[chmue]

Just include a dependency or optional dependency for pandoc. pandoc-bin, pandoc-cabal, and pandoc-static in the AUR all include provides=("pandoc") so pacman will view the dependency as satisfied if any of the packages is installed. Depending on the AUR manager you use, a caveat may be that if there is no pandoc package installed pacman will try to install the official pandoc package which has a ton of Haskell dependencies. The solution is to install the preferred pandoc package before installing RStudio.

Personally, I would leave the path hard-coded. If a package that provides pandoc is installed, the hard-coded symlink will work because they all place the executable or a symlink there. Even if you have a very specific setup, /usr/bin is the standard Arch directory for executables. So the pandoc executable will most likely be symlinked to /usr/bin even if it resides in a different folder. Additionally, if you add pandoc to optdepends the hard-coded symlink will still work when you decide to install pandoc. Otherwise you'd need to rebuild the package.

[ASalvail]

You can keep the symlink. But make sure there is at least a notice to install another package providing the pandoc executable if you don't want to add it as a dependency. It took me some time to try and understand why the problem wasn't fixed after installing this version or RStudio.

[chendaniely]

I re-built my arch machine (antergos) and no longer have this issue with pandoc rendering in rstudio. I've also commented out the symlinks in the PKGBUILD recently.

It seems to be working for me. Can someone else confirm?

[Oliph]

Works for me too, no longer an issue with pandoc in rstudio

[chmue]

:+1: Works for me as well. I use the stable rstudio-desktop-bin package.

[chendaniely]

can confirm this is working. I've been maintaining the rstudio-desktop-preview-bin package.

[iztokstotl]

For anyone still experiencing this issue on Fedora 28 and 29.

This bug is still present in Fedora 28 an 29 (November 2018 !!!).

I have solved it with:

---

sudo dnf install pandoc sudo rm /usr/lib/rstudio/bin/pandoc/pandoc sudo ln -s /usr/bin/pandoc /usr/lib/rstudio/bin/pandoc/pandoc`

---

[github-actions[bot]]

This old thread has been automatically locked. If you think you have found something related to this, please open a new issue by following the issue guide (https://yihui.org/issue/), and link to this old issue if necessary.