pandoc2 and rmarkdown interaction for metropolis

[eddelbuettel]

When using pandoc 2.2.1 (as shipped by RStudio, here linked to ~/bin/pandoc as you see below) an otherwise fine RMarkdown presentation using the Metropolis theme fails with

/home/edd/bin/pandoc +RTS -K512m -RTS minex.utf8.md --to beamer \
   --from markdown+autolink_bare_uris+ascii_identifiers+tex_math_single_backslash \
   --output minex.tex --variable theme=metropolis --highlight-style tango \
   --pdf-engine xelatex --self-contained            
tlmgr search --file --global '/ucharcat.sty'        
Cannot determine type of tlpdb from /home/edd/texmf!            
cannot setup TLPDB in /home/edd/texmf at /usr/bin/tlmgr line 6071.             
! LaTeX Error: File `ucharcat.sty' not found.  

The file is of course present, and running with the older pandoc version 1.19.2.1 works fine (as shown below).

This happens with both the CRAN and the development versions of RMarkdown.

I used the following file minex.Rm:

---
author: Some Author
title: A Talk
subtitle: Subtitle
date: 1 Jan 2018
output:
  beamer_presentation:
    keep_tex: no
    theme: metropolis
    latex_engine: xelatex
classoption: compress
fontsize: 12pt
---

```{r, setup, include=FALSE}
knitr::opts_chunk$set(cache=TRUE)

New Slide

New Title

Once I remove the softlink, all is well:

edd@rob:~$ render.r minex.Rmd

processing file: minex.Rmd |...................... | 33% ordinary text without R code

|........................................... | 67% label: setup (with options) List of 1 $ include: logi FALSE

|.................................................................| 100% ordinary text without R code

output file: minex.knit.md

/usr/bin/pandoc +RTS -K512m -RTS minex.utf8.md --to beamer --from markdown+autolink_bare_uris+ascii_identifiers+tex_math_single_backslash --output minex.tex --variable theme=metropolis --highlight-style tango --latex-engine xelatex --self-contained

Output created: minex.pdf edd@rob:~$

As I would like to move to pandoc2 at some point, it would nice if this could be addressed. 

Could this be a preference for tinytex over a standard (working, complete, ...) texlive installation?

Ubuntu 17.10, R 3.4.4, everything from CRAN current

[yihui]

I cannot reproduce this issue with the latest version of xelatex:

$ xelatex --version
XeTeX 3.14159265-2.6-0.99999 (TeX Live 2018)
kpathsea version 6.3.0
Copyright 2018 SIL International, Jonathan Kew and Khaled Hosny.
There is NO warranty.  Redistribution of this software is
covered by the terms of both the XeTeX copyright and
the Lesser GNU General Public License.
For more information about these matters, see the file
named COPYING and the XeTeX source.
Primary author of XeTeX: Jonathan Kew.
Compiled with ICU version 61.1; using 61.1
Compiled with zlib version 1.2.11; using 1.2.11
Compiled with FreeType2 version 2.9.0; using 2.9.0
Compiled with Graphite2 version 1.3.11; using 1.3.11
Compiled with HarfBuzz version 1.7.6; using 1.7.6
Compiled with libpng version 1.6.34; using 1.6.34
Compiled with poppler version 0.63.0
Using Mac OS X Core Text and Cocoa frameworks

The tinytex package does not have a preference over a specific LaTeX installation. It is just that if you use TinyTeX, your missing packages can be automatically installed, otherwise you will have to figure out how to install the missing packages by yourself (tinytex tried but failed, probably because you installed TeXLive as a Debian package which requires sudo).

Assuming your TeXLive installation is standard, working, and complete, we have to figure out why Pandoc 2.2.1 starts to require ucharcat.sty. You may use keep_tex: yes, and diff the .tex output files with Pandoc 1.x and 2.x, respectively.

[eddelbuettel]

My TeXLatex is pretty standard; looks like the same xelatex but one version of kpathsea behind.

edd@rob:~$ xelatex --version
XeTeX 3.14159265-2.6-0.99998 (TeX Live 2017/Debian)
kpathsea version 6.2.3
Copyright 2017 SIL International, Jonathan Kew and Khaled Hosny.
There is NO warranty.  Redistribution of this software is
covered by the terms of both the XeTeX copyright and
the Lesser GNU General Public License.
For more information about these matters, see the file
named COPYING and the XeTeX source.
Primary author of XeTeX: Jonathan Kew.
Compiled with ICU version 57.1; using 57.1
Compiled with zlib version 1.2.11; using 1.2.11
Compiled with FreeType2 version 2.8.0; using 2.8.0
Compiled with Graphite2 version 1.3.10; using 1.3.10
Compiled with HarfBuzz version 1.4.2; using 1.4.2
Compiled with libpng version 1.6.31; using 1.6.34
Compiled with poppler version 0.57.0
Compiled with fontconfig version 2.11.94; using 2.11.94
edd@rob:~$ 

My TeXLive is a very standard Ubuntu installation; this has always worked.

The diff is a good idea, this is pretty long which is surprising--the files are 2044 and 2788 bytes long, respectively. Does anything in the diff jump at you?

--- minex_pandoc19.tex  2018-07-03 04:56:31.079727228 -0500
+++ minex_pandoc22.tex  2018-07-03 04:57:07.471337874 -0500
@@ -1,4 +1,8 @@
+\PassOptionsToPackage{unicode=true}{hyperref} % options for packages loaded elsewhere
+\PassOptionsToPackage{hyphens}{url}
+%
 \documentclass[12pt,ignorenonframetext,compress]{beamer}
+\usepackage{pgfpages}
 \setbeamertemplate{caption}[numbered]
 \setbeamertemplate{caption label separator}{: }
 \setbeamercolor{caption name}{fg=normal text.fg}
@@ -10,12 +14,9 @@
 \ifnum 0\ifxetex 1\fi\ifluatex 1\fi=0 % if pdftex
   \usepackage[T1]{fontenc}
   \usepackage[utf8]{inputenc}
+  \usepackage{textcomp} % provides euro and other symbols
 \else % if luatex or xelatex
-  \ifxetex
-    \usepackage{mathspec}
-  \else
-    \usepackage{fontspec}
-  \fi
+  \usepackage{unicode-math}
   \defaultfontfeatures{Ligatures=TeX,Scale=MatchLowercase}
 \fi
 \usetheme[]{metropolis}
@@ -23,48 +24,69 @@
 \IfFileExists{upquote.sty}{\usepackage{upquote}}{}
 % use microtype if available
 \IfFileExists{microtype.sty}{%
-\usepackage{microtype}
+\usepackage[]{microtype}
 \UseMicrotypeSet[protrusion]{basicmath} % disable protrusion for tt fonts
 }{}
-\newif\ifbibliography
+\IfFileExists{parskip.sty}{%
+\usepackage{parskip}
+}{% else
+\setlength{\parindent}{0pt}
+\setlength{\parskip}{6pt plus 2pt minus 1pt}
+}
+\usepackage{hyperref}
 \hypersetup{
             pdftitle={A Talk},
             pdfauthor={Some Author},
             pdfborder={0 0 0},
             breaklinks=true}
 \urlstyle{same}  % don't use monospace font for urls
-
+\newif\ifbibliography
 % Prevent slide breaks in the middle of a paragraph:
 \widowpenalties 1 10000
 \raggedbottom
-
+\setbeamertemplate{part page}{
+\centering
+\begin{beamercolorbox}[sep=16pt,center]{part title}
+  \usebeamerfont{part title}\insertpart\par
+\end{beamercolorbox}
+}
+\setbeamertemplate{section page}{
+\centering
+\begin{beamercolorbox}[sep=12pt,center]{part title}
+  \usebeamerfont{section title}\insertsection\par
+\end{beamercolorbox}
+}
+\setbeamertemplate{subsection page}{
+\centering
+\begin{beamercolorbox}[sep=8pt,center]{part title}
+  \usebeamerfont{subsection title}\insertsubsection\par
+\end{beamercolorbox}
+}
 \AtBeginPart{
-  \let\insertpartnumber\relax
-  \let\partname\relax
   \frame{\partpage}
 }
 \AtBeginSection{
   \ifbibliography
   \else
-    \let\insertsectionnumber\relax
-    \let\sectionname\relax
     \frame{\sectionpage}
   \fi
 }
 \AtBeginSubsection{
-  \let\insertsubsectionnumber\relax
-  \let\subsectionname\relax
   \frame{\subsectionpage}
 }
-
-\setlength{\parindent}{0pt}
-\setlength{\parskip}{6pt plus 2pt minus 1pt}
 \setlength{\emergencystretch}{3em}  % prevent overfull lines
 \providecommand{\tightlist}{%
   \setlength{\itemsep}{0pt}\setlength{\parskip}{0pt}}
 \setcounter{secnumdepth}{0}

+% set default figure placement to htbp
+\makeatletter
+\def\fps@figure{htbp}
+\makeatother
+
+
 \title{A Talk}
+\providecommand{\subtitle}[1]{}
 \subtitle{Subtitle}
 \author{Some Author}
 \date{1 Jan 2018}
@@ -72,8 +94,10 @@
 \begin{document}
 \frame{\titlepage}

-\section{New Slide}\label{new-slide}
+\hypertarget{new-slide}{%
+\section{New Slide}\label{new-slide}}

-\subsection{New Title}\label{new-title}
+\hypertarget{new-title}{%
+\subsection{New Title}\label{new-title}}

 \end{document}

[eddelbuettel]

BTW you had that wrong for texlive/tinytex and this system:

edd@rob:~$ tlmgr 
(running on Debian, switching to user mode!)
tlmgr: no action given; try --help if you need it.
edd@rob:~$ 

It only runs in user mode (and I prefer system wide fixes for latex). I poked into that eg via https://tex.stackexchange.com/questions/137428/tlmgr-cannot-setup-tlpdb but no luck.

It seems more like a bad pandoc / beamer / metropolis interaction but would be nice if we could sail around it.

[eddelbuettel]

But I think we can close this. I just dropped into a Docker container with Debian unstable/testing, and everything is fine there too:

root@20f7a7d68357:/mnt# render.r minex.Rmd
List of 5
 $ --help : logi FALSE
 $ --usage: logi FALSE
 $ FILES  : chr "minex.Rmd"
 $ help   : logi FALSE
 $ usage  : logi FALSE
NULL

processing file: minex.Rmd
  |......................                                           |  33%
  ordinary text without R code

  |...........................................                      |  67%
label: setup (with options) 
List of 1
 $ include: logi FALSE

  |.................................................................| 100%
  ordinary text without R code

output file: minex.knit.md

/usr/bin/pandoc +RTS -K512m -RTS minex.utf8.md --to beamer --from markdown+autolink_bare_uris+ascii_identifiers+tex_math_single_backslash --output minex.tex --variable theme=metropolis --highlight-style tango --pdf-engine xelatex --self-contained 

Output created: minex.pdf
root@20f7a7d68357:/mnt# 
root@20f7a7d68357:/mnt# pandoc --version
pandoc 2.2.1
Compiled with pandoc-types 1.17.4.2, texmath 0.10.1.1, skylighting 0.6
Default user data directory: /root/.pandoc
Copyright (C) 2006-2018 John MacFarlane
Web:  http://pandoc.org
This is free software; see the source for copying conditions.
There is no warranty, not even for merchantability or fitness
for a particular purpose.
root@20f7a7d68357:/mnt# 

You may want to keep the bug open, or tag it, as others may stumble on it when using similar texlive versions.

[yihui]

Thanks for the diff! So Pandoc 2.x introduced \usepackage{unicode-math}, which seems to be the culprit in your case, since it requires uchar­cat. I did a quick google search, and it seems this has been problematic before (although it was said to have been fixed): https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=811508

[eddelbuettel]

Good catch! I actually tried to triage that and uncommented a few, but I did not hit the pot of gold you found. As the bug report you found is older, maybe this is a regression?

[eddelbuettel]

Follow-up: Discussed this with @norbusan, Debian's texlive maintainer, over email and he correctly pointed out that I could simply use the 'old' pandoc 1.9 template.tex with pandoc 2.2, either explicitly or maybe via a YAML header argument. Maybe I will try that, else while I still have pandoc 1.9.* in my $PATH all is good anyway...

[norbusan]

i am suprised that loading unicode-math on Ubuntu 17.04 does not work. As @yihui said, this should have been fixed back in Feb 2016, so it is suprising to me.

@eddelbuettel if you have Ubuntu 17.04 running, could you run a minimal test file

\documentclass{article}
\usepackage{unicode-math}
\begin{document}
Hello World
\end{document}

and run it with xelatex -recorder and send me the .fls and .log file (or attach it here). In current TL this file is not loaded at all.

[eddelbuettel]

Did I say 17.10? I meant 17.04. I am also suprised it failed.

edd@rob:/tmp$ xelatex -recorder foo.tex 
This is XeTeX, Version 3.14159265-2.6-0.99998 (TeX Live 2017/Debian) (preloaded format=xelatex)
 restricted \write18 enabled.
entering extended mode
(./foo.tex
LaTeX2e <2017-04-15>
Babel <3.12> and hyphenation patterns for 7 language(s) loaded.
(/usr/share/texlive/texmf-dist/tex/latex/base/article.cls
Document Class: article 2014/09/29 v1.4h Standard LaTeX document class
(/usr/share/texlive/texmf-dist/tex/latex/base/size10.clo))
(/usr/share/texlive/texmf-dist/tex/latex/unicode-math/unicode-math.sty
(/usr/share/texlive/texmf-dist/tex/generic/ifxetex/ifxetex.sty)
(/usr/share/texlive/texmf-dist/tex/generic/oberdiek/ifluatex.sty)
(/usr/share/texlive/texmf-dist/tex/latex/l3kernel/expl3.sty
(/usr/share/texlive/texmf-dist/tex/latex/l3kernel/expl3-code.tex)
(/usr/share/texlive/texmf-dist/tex/latex/l3kernel/l3xdvipdfmx.def))

! LaTeX Error: File `ucharcat.sty' not found.

Type X to quit or <RETURN> to proceed,
or enter new name. (Default extension: sty)

Enter file name: 
(/usr/share/texlive/texmf-dist/tex/latex/l3packages/xparse/xparse.sty)
(/usr/share/texlive/texmf-dist/tex/latex/l3packages/l3keys2e/l3keys2e.sty)
(/usr/share/texlive/texmf-dist/tex/latex/fontspec/fontspec.sty
(/usr/share/texlive/texmf-dist/tex/latex/fontspec/fontspec-xetex.sty
(/usr/share/texlive/texmf-dist/tex/latex/base/fontenc.sty
(/usr/share/texlive/texmf-dist/tex/latex/base/tuenc.def))
(/usr/share/texlive/texmf-dist/tex/latex/fontspec/fontspec.cfg)))
(/usr/share/texlive/texmf-dist/tex/latex/base/fix-cm.sty
(/usr/share/texlive/texmf-dist/tex/latex/base/ts1enc.def))
(/usr/share/texlive/texmf-dist/tex/latex/filehook/filehook.sty)
(/usr/share/texlive/texmf-dist/tex/latex/unicode-math/unicode-math-xetex.sty
(/usr/share/texlive/texmf-dist/tex/latex/unicode-math/unicode-math-table.tex)))
No file foo.aux.
[1] (./foo.aux) )
Output written on foo.pdf (1 page).
Transcript written on foo.log.
edd@rob:/tmp$ 

I could have sworn locate ucharcat.sty found it days ago when I looked for this, now it doesn't. I may have looked at packages.debian.org when I should have looked at Ubuntu,

In any event, I just learned that it is in texlive-luatex which I had not installed. I have it now, and with that rmarkdown works as well with pandoc2:

My life is good again. This was a false alert: I was the luser not being able to find ucharcat.sty. The only remaining thought is to maybe extend the depends of some other package to texlive-luatex.

[norbusan]

The dependency was added due to the bug report mentioned above, and that was in 2/2016, so it should have been in Ubuntu 17.04 ...

[eddelbuettel]

That is indeed weird, but the more embarassing issue is that fell over a missing file in a missing package -- usually it is simple enough to find the missing file. Someone I convinced myself it was present when it wasn't.... Anyway, may help the next guy who stumbles over this and googles it: install texlive-luatex.

[github-actions[bot]]

This old thread has been automatically locked. If you think you have found something related to this, please open a new issue by following the issue guide (https://yihui.org/issue/), and link to this old issue if necessary.