Language switch in table with bidi can fail

[jspitz]

The following document does not compile:

% !TeX TS-program = xelatex
\documentclass{article}
\usepackage{polyglossia}
\setmainfont{FreeSerif}
\setdefaultlanguage{hebrew}
\setotherlanguage{english}

\begin{document}

\begin{table}[t]

\begin{tabular}{cc}
\begin{english}1\end{english} & 2\tabularnewline
\end{tabular}

\end{table}

\end{document}

The reason is that bidi swaps the column order in tables, and in this context, the aux file order stream seems to be swapped, so we get \bgroup after \egroup in the aux file.

No idea how to solve this. Anyone?

[reutenauer]

Wow. That’s something. I have no idea how to work around this, but we must in any case warn about this in the doc. It’s not very likely that people will really need a language environment here, \textlang should work just as well. But of course it would be nice to at least avoid having TeX crash ...

[jspitz]

I think we can only fix this if a check is implemented in bidi that tells us whether we are in such a table.

[seloumi]

@jspitz
From bidi manual (Tabular Conditional) \if@RTLtab If the tabular is typeset RTL,\if@RTLtab is true and if the tabular is typeset LTR,\if@RTLtab is false.

[reutenauer]

Good catch @seloumi, thank you!

[jspitz]

Yes, but this does not suffice. This conditional is also true outside tables. We need to patch bidi.

[Udi-Fogiel]

The reason is that bidi swaps the column order in tables, and in this context, the aux file order stream seems to be swapped, so we get \bgroup after \egroup in the aux file.

It isn't really related to the fact that bidi reflect the columns. The reason for the wrong order of the \write's comes from the (unfortunate) fact that the TeX--XeT algorithm reverses all the nodes of the hlist in RTL context, including the whatsit nodes that holds the contents of the \write's until shipout.

Consider the following code

\newwrite\test
\openout\test=test.txt
\TeXXeTstate=1
\beginR\write\test{Test 1}\write\test{Test 2}
\bye

You can see how Test 2 was written first to the text file. This is one of the reasons colors and hyperlinks are a pain in bidirectional typesetting, since they usually implemented using whatsit nodes that should appear in a certain order.

We can actually observe this phenomena using polyglossia. The .aux file from the following code

\documentclass{article}

\usepackage{polyglossia}
\setdefaultlanguage{english}
\setotherlanguage{hebrew}
\setmainfont{FreeSerif}

\begin{document}
\begin{hebrew}
Hello
\end{hebrew}
\end{document}

contains the lines

\@writefile{toc}{\selectlanguage *{english}}
\@writefile{toc}{\selectlanguage *{hebrew}}
\@writefile{toc}{\selectlanguage *{english}}
\@writefile{toc}{\selectlanguage *[variant=us,ordinalmonthday=false]{english}}

While with

\documentclass{article}

\usepackage{polyglossia}
\setdefaultlanguage{english}
\setotherlanguage{french}
\setmainfont{FreeSerif}

\begin{document}
\begin{french}
Hello
\end{french}
\end{document}

we get

\@writefile{toc}{\selectlanguage *{english}}
\@writefile{toc}{\selectlanguage *{french}}
\@writefile{toc}{\selectlanguage *[variant=us,ordinalmonthday=false]{english}}
\@writefile{toc}{\selectlanguage *{english}}

Observe how the last two lines are in different oreder, this is because polyglossia writes the information at the end of the environment, without entering vertical mode. If we will make sure to enter vertical mode before exiting the hebrew environment

\documentclass{article}

\usepackage{polyglossia}
\setdefaultlanguage{english}
\setotherlanguage{hebrew}
\setmainfont{FreeSerif}

\begin{document}
\begin{hebrew}
Hello

\end{hebrew}
\end{document}

the aux will contain

\@writefile{toc}{\selectlanguage *{english}}
\@writefile{toc}{\selectlanguage *{hebrew}}
\@writefile{toc}{\selectlanguage *[variant=us,ordinalmonthday=false]{english}}
\@writefile{toc}{\selectlanguage *{english}}

[jspitz]

@Udi-Fogiel thanks for the clear explanations. Pull requests are always welcome.