Rowcolors colors table captions and headings when multiple tables span pages

[jamesgarriss]

The Goal

I am converting Markdown to PDF. The content includes multiple, long tables that often break over the pages. My goal is to alternate the coloring of the rows.

The Problem

The problem that I'm seeing is that the coloring appears on table captions and headings, not on the first table but on following tables. I'm not sure if this is because I'm doing something poorly or because there's a bug in Pandoc.

To re-create the problem:

1. Create a Markdown file with multiple, long tables (see attached ipad.md).
2. Convert Markdown to PDF and observe that everything looks good.

pandoc ipad.md
  --output=ipad.pdf

3. Extract the template.

pandoc -D latex > new_template.latex

4. Modify the template in two places to use rowcolors (see attached new_template.latex, which I had to rename to .txt, as GitHub won't allow me to upload a .latex). a. Find \usepackage{xcolor} and change to \usepackage[table]{xcolor}. b. Find \usepackage{longtable,booktabs,array} and add the following line \rowcolors{1}{}{yellow}.
5. Convert Markdown to PDF using the template.

pandoc ipad.md
  --output=ipad.pdf
  --template=new_template.latex

6. Open the resulting PDF (see attached ipad.pdf). Notice that caption and heading for tables 2 and 4 are highlighted yellow.

It seems like there should be a smarter/simpler way to solve this problem, but it was all I could figure out. If there's not, then I assume this is a bug in Pandoc.

Pandoc Version

pandoc 2.14.0.1 macOS 11.4 (Big Sur)

ipad.md new_template.txt ipad.pdf

[mb21]

Well, if you're modifying your latex template and things aren't as expected, I wouldn't say that's a bug in pandoc.

For questions like this, please use the pandoc-discuss mailing list instead of this issue tracker. Thanks!

[jgm]

I don't think this is a pandoc issue, unless you see something wrong in the code pandoc spits out for the tables. You might ask a TeX expert how to achieve what you're trying to do with the colors. If there is a specific change to pandoc's table output that would facilitate this, you can suggest it. Otherwise we should probably close this.

[jamesgarriss]

You might be right. Please give me a few days to try and come up with an example that either proves or disproves.

[jamesgarriss]

Thank you for your patience. I think I have an example of what I'm expecting and what I'm actually getting.

1. Per your request, I've created a Latex example of what I want to happen. (See simple.tex, which I had to rename to simple.txt)
2. To convert the Latex to PDF, run:

pdflatex simple.tex

3. Look at the resulting PDF file, observing that neither the caption nor the first row (on the first page and the second) are colored yellow. This is exactly what I want. (See simple.pdf)
4. I've created an equivalent example in Markdown (because in my world, my source content is Markdown, not Latex). (See simple.md)
5. I modified the template by adding a "PassOptionsToPackage" command. (See line 4 in new_template.latex, which I had to rename new_template.txt)
6. I created a styles file. (See styles.tex, which I had to rename styles.txt)
7. To convert the Markdown to PDF, run:

pandoc simple.md 
  --output=simple-from-md.pdf 
  --template=new_template.latex 
  --include-in-header=styles.tex

8. Look at the resulting PDF file, observing that the first row on the second page is colored yellow, which is not desired. (See simple-from-md.pdf)
9. To see the difference between the Latex I created by hand and the Latex that Pandoc created, run:

pandoc simple.md 
  --output=simple-from-md.tex 
  --template=new_template.latex 
  --include-in-header=styles.tex

10. Look at the resulting Latex file, observing that the primary difference is the lack of \hiderowcolors and \showrowcolors. (See simple-from-md.tex, which I had to rename simple-from-md.txt)

Is this a bug? No, probably not. Is this a feature request? I think so. I need some way to inject these commands into a longtable around the caption and first rows when converting Markdown to PDF.

Thanks for listening.

simple.txt simple.pdf simple.md new_template.txt styles.txt simple-from-md.pdf simple-from-md.txt

[jamesgarriss]

@jgm , any thoughts? TIA.

[jgm]

Haven't had a chance to look at this in detail. But if the standard pandoc output colors the column headings on the second page, that definitely seems wrong and I'd consider it a bug that should be fixed. Can you suggest a minimal diff from the tex pandoc generates now that would fix this?

[jamesgarriss]

I think so. Here's the pertinent part of the Latex that I wrote by hand:

  \begin{longtable}{ccc}
    \hiderowcolors
    \caption{Table With Colors} \label{table1} \\
    \hline
    \textbf{Column A} & \textbf{Column B} & \textbf{Column C}\\
    \hline
    \showrowcolors
    \endfirsthead
    \hiderowcolors
    \hline
    \textbf{Column A} & \textbf{Column B} & \textbf{Column C}\\
    \hline
    \showrowcolors
    \endhead
    a & b & c\\
    ...
    \hline
  \end{longtable}

Here's the corresponding part of the Latex created by Pandoc:

\begin{longtable}[]{@{}lll@{}}
  \caption{Table With Colors}\tabularnewline
  \toprule
  \textbf{Column A} & \textbf{Column B} & \textbf{Column C} \\
  \midrule
  \endfirsthead
  \toprule
  \textbf{Column A} & \textbf{Column B} & \textbf{Column C} \\
  \midrule
  \endhead
  a & b & c \\
  ...
  \bottomrule
\end{longtable}

If you compare these two, it appears that the only significant difference is the use of the \hiderowcolors and \showrowcolors commands. Thus, the output from Pandoc could be adjusted to look like this (differences highlighted by <--- add this):

\begin{longtable}[]{@{}lll@{}}
  \hiderowcolors         <-- add this
  \caption{Table With Colors}\tabularnewline
  \toprule
  \textbf{Column A} & \textbf{Column B} & \textbf{Column C} \\
  \midrule
  \showrowcolors         <-- add this
  \endfirsthead
  \hiderowcolors         <-- add this
  \toprule
  \textbf{Column A} & \textbf{Column B} & \textbf{Column C} \\
  \midrule
  \showrowcolors         <-- add this
  \endhead
  a & b & c \\
  ...
  \bottomrule
\end{longtable}

[jgm]

Here's a pure LaTeX solution for you:

\usepackage{etoolbox}
\AtBeginEnvironment{longtable}{\rowcolors{2}{}{yellow}}
\apptocmd{\toprule}{\hiderowcolors}{}{}
\apptocmd{\endhead}{\showrowcolors}{}{}
\apptocmd{\endfirsthead}{\showrowcolors}{}{}

[jgm]

We might consider adding the table option to xcolor in the default template. I think it would be harmless, since we already include xcolor by default. However, according to this a different syntax is needed for beamer, which would mean more conditionals...

[jamesgarriss]

For any who come along and read this discussion and like me are looking for a way to set alternating row colors with grey, here's the complete set of steps to follow:

1. Create a new template file:

pandoc -D latex > new_template.latex

2. Modify the template file by adding a new PassOptionsToPackage instruction just after the others. For me, this was on the 4th line of the file:

\PassOptionsToPackage{table}{xcolor}

3. Create a styles.tex file and add the code @jgm provided. You can make the row color more or less gray by adjusting the percentage (currently set to 25%):

\usepackage{etoolbox}
\AtBeginEnvironment{longtable}{\rowcolors{2}{}{gray!25}}
\apptocmd{\toprule}{\hiderowcolors}{}{}
\apptocmd{\endhead}{\showrowcolors}{}{}
\apptocmd{\endfirsthead}{\showrowcolors}{}{}

4. Bring it all together with Pandoc:

pandoc input.md 
  --output=output.pdf 
  --template=new_template.latex
  --include-in-header=styles.tex

Thank you, @jgm, for this solution!

[jgm]

Note: if you're using a custom template anyway, you could just insert these lines into the template directly instead of using an external styles.tex and --include-in-header.

If we modify the default template, we might be able to avoid the need for a custom template.

[jgm]

I noticed one small problem when I tried this: the colored bands extend beyond the rows.

[jgm]

Maybe we can leave this open til we decide whether to tweak the custom template.

[jamesgarriss]

I don't understand what the correlation is, but if you put this line in styles.tex, you will see a large extension:

\setlength{\tabcolsep}{10pt}

If you set this value to 0, however, there is no extension:

\setlength{\tabcolsep}{0pt}

Perhaps the value of the separation should be subtracted from the total width of the row to avoid extensions?

[jgm]

This is probably related to the fact that in pandoc we start and end the table column descriptions with @{}. That causes the heading text to line up with the lines; otherwise space is left. I assume that the table support in xcolor is not paying attention to this.

[spgarbet]

The earlier versions of pandoc worked fine for me as I added xcolor later with the options I wanted. The current template doesn't have any way for me to turn on the table option to xcolor. I have a large body of work that depends on this and now it's utterly and completely broken. My only option is to create a custom LaTeX template now. I could change the options with \PassOptionsToPackage, but this has to be after the \documentclass which is at the top of the template.

[jgm]

@spgarbet what earlier versions? We've been starting and ending table column descriptions with @{} since 2013.

[spgarbet]

Using RStudio it packaged earlier versions for a long time. The recent update has completely destroyed my stack, all because I can't do this:

\usepackage[table]{xcolor}

I need some means to inject/control which options are invoked to xcolor.

The current framework rigidly makes that choice for me and it doesn't include table as an option.

[jgm]

Steps toward improving this:

• [ ] include the table option in xcolor options (note: something different may be needed for beamer, as noted above, but we could deal with that later)
• [ ] add some code that is sensitive to a color-tables variable and adds the coloring macros using etoolbox, as suggested above (but this needs tweaking; in my tests it didn't work well) (EDIT: Actually it would be ideal to allow coloring to be selected via table attributes, on a table by table basis, rather than globally; this would require changes to the writer.)
• [ ] figure out how to deal with the width issue noted above

[mon]

I need some means to inject/control which options are invoked to xcolor.

The current framework rigidly makes that choice for me and it doesn't include table as an option.

As a solution to this issue, adding table to classoption gave xcolor the table option for me (pandoc 2.19.2).

[sparlund]

EDIT: maybe this is better suited in discussions rather than issues. Let me know and I'll move it.

Hi @jgm and @jamesgarriss , I've employed your below solution to create a table of alternating colors.

\usepackage{etoolbox} \AtBeginEnvironment{longtable}{\rowcolors{2}{}{yellow}} \apptocmd{\toprule}{\hiderowcolors}{}{} \apptocmd{\endhead}{\showrowcolors}{}{} \apptocmd{\endfirsthead}{\showrowcolors}{}{}

It works - up to a point. I have long tables, and after a while it just stops alternating the colors. If I output to latex instead of just PDF, I see no change in the the latex code for the table. It's just repeated entries. Example below:

\begin{longtable}[]{@{}
  >{\raggedright\arraybackslash}p{(\columnwidth - 4\tabcolsep) * \real{0.2955}}
  >{\raggedright\arraybackslash}p{(\columnwidth - 4\tabcolsep) * \real{0.3636}}
  >{\raggedright\arraybackslash}p{(\columnwidth - 4\tabcolsep) * \real{0.3295}}@{}}
\toprule\noalign{}
\begin{minipage}[b]{\linewidth}\raggedright
\textbf{A}
\end{minipage} & \begin{minipage}[b]{\linewidth}\raggedright
\textbf{B}
\end{minipage} & \begin{minipage}[b]{\linewidth}\raggedright
\textbf{C}
\end{minipage} \\
\midrule\noalign{}
\endhead
\bottomrule\noalign{}
\endlastfoot
D & E & F \\
G & H & I \\
G & H & I \\
G & H & I \\
G & H & I \\
G & H & I \\
G & H & I \\
 % repeat like this ~100 times
 G & H & I \\
 \end{longtable}

This latex looks good to me?

But like I said, after a while the table just gives up on alternating colors:

Would this issue point towards an issue in the latex template, pandoc PDF generation, or lualatex..? Or maybe just towards me 😂

If it's a short table with just a few entries, no issue 🤔