LuaLaTeX des not correctly render Sanskrit and Tamil scripts; XeLaTeX does: from the same source file with babel

[kleinbottle]

% lua-xe-sa-ta-gk-lua.tex
%
\RequirePackage{latexbug}
\documentclass[12pt,a4paper]{article}
\usepackage{fontspec}
\usepackage[british]{babel}
%
\setmainfont[Scale=1,BoldFont={* Medium}]{Noto Serif}
\setsansfont[BoldFont={* Semibold}]{Source Sans Pro}
\setmonofont{Fira Mono}
%
% Fonts for sa, ta, gk
%
\newfontfamily\sanskritfont{Noto Serif Devanagari}[Scale=1.3]
\newfontfamily\tamilfont{Noto Sans Tamil}
\newfontfamily\greekfont{Noto Serif}
%
\babelprovide[import onchar=ids fonts]{sanskrit}
\babelprovide[import onchar=ids fonts]{tamil}
\babelprovide[import onchar=ids fonts]{greek}
%
\defaultfontfeatures{Ligatures=TeX,Scale=MatchLowercase,Renderer=Harfbuzz}
%
\setlength{\parindent}{0pt}
%
\begin{document}
%
\bigskip
Sanskrit in Devanāgarī

{\sanskritfont क्षिप्रं हि मानुषे लोके सिद्धिर्भवति कर्मजा ॥}

\medskip

Śrīmad Bhagavad Gītā 4:12

\bigskip

Tamil

{\tamilfont கற்க கசடறக் கற்பவை கற்றபின்\newline
நிற்க அதற்கு தக

\medskip

திருவள்ளுவர், திருக்குறள்} 40:391
\bigskip

Greek.

\medskip

{\greekfont Ὁδοὶ δύο εἰσί, μία τῆς ζωῆς καὶ μία τοῦ θανάτου, διαφορὰ δὲ πολλὴ μεταξὺ τῶν δύο ὁδῶν.}
\end{document}

Two image files showing the output from the above file with lualatex and xelatex are attached. The second one, produced by xelatex is correct.

Why is the lualatex output incorrect? Is lualatex still not quite ready for routine use with Indic scripts?

[u-fischer]

You need the harfbuzz renderer. So either move \defaultfontfeatures{Ligatures=TeX,Scale=MatchLowercase,Renderer=Harfbuzz} before your font definitions, or use Renderer=Harfbuzz in the optional argument when declaring a font.

[kleinbottle]

You need the harfbuzz renderer. So either move \defaultfontfeatures{Ligatures=TeX,Scale=MatchLowercase,Renderer=Harfbuzz} before your font definitions, or use Renderer=Harfbuzz in the optional argument when declaring a font.

I have now done that, but get similar errors as before with lualatex.

[kleinbottle]

I re-read the babel manual more carefully and now believe that for use with lualatex, babel provides a self-contained mechanism for selection of language-based scripts in accordance with the characters in the text. We do not need to explicitly invoke \setmainfont and friends from fontsepc. Nor do we need to fence characters from other languages with font- or language-declarations. The script and language seem to be matched automagically by babel with lualatex. At present this approach does not work with xelatex.

My new file is:

\documentclass[12pt,a4paper]{article}
\usepackage{fontspec}
\usepackage[british]{babel}
%
\defaultfontfeatures{Ligatures=TeX,Scale=MatchLowercase,Renderer=Harfbuzz}
%
% Fonts for the British variant of English
%
\babelfont[british]{rm}[Scale=0.9]{Noto Serif}
\babelfont[british]{sf}[Scale=0.9]{Source Sans Pro}
\babelfont[british]{tt}[Scale=0.9]{Fira Mono}
%
% Fonts for sa, ta, gk
%
\babelprovide[import,onchar=ids fonts]{sanskrit}
\babelprovide[import,onchar=ids fonts]{tamil}
\babelprovide[import,onchar=ids fonts]{greek}
%
% Select fonts for sa, ta, gk
%
\babelfont[sanskrit]{rm}[Scale=1]{Noto Serif Devanagari}
\babelfont[sanskrit]{sf}[Scale=1]{Noto Sans Devanagari}
\babelfont[tamil]{rm}[Scale=0.9]{Noto Serif Tamil}
\babelfont[tamil]{sf}[Scale=0.9]{Noto Sans Tamil}
\babelfont[greek]{rm}[Scale=0.9]{Noto Serif}
%
\setlength{\parindent}{0pt}
%
\begin{document}
%
\bigskip
Sanskrit in Devanāgarī

क्षिप्रं हि मानुषे लोके सिद्धिर्भवति कर्मजा ॥

\medskip

Śrīmad Bhagavad Gītā 4:12

\bigskip

Tamil

\sffamily
கற்க கசடறக் கற்பவை கற்றபின்\newline
நிற்க அதற்கு தக

\medskip

திருவள்ளுவர், திருக்குறள் 40:391
\rmfamily

\bigskip

Greek.

\medskip

 Ὁδοὶ δύο εἰσί, μία τῆς ζωῆς καὶ μία τοῦ θανάτου, διαφορὰ δὲ πολλὴ μεταξὺ τῶν δύο ὁδῶν.

\bigskip

English

\textrm{Hello World!}

\textsf{Hello World!}

\texttt{Hello World!}

\end{document}

which gives the correctly formed Indic scripts as well as English and Greek, as shown above.

[kleinbottle]

I found that compound font-features like bold smallcaps are not displayed from the file given above. It appears that if the lines

%
% Fonts for the British variant of English
%
\babelfont[british]{rm}[Scale=0.9]{Noto Serif}
\babelfont[british]{sf}[Scale=0.9]{Source Sans Pro}
\babelfont[british]{tt}[Scale=0.9]{Fira Mono}

are replaced by:

%
% Fonts set according to fontspec
%
\setmainfont[Scale=1,BoldFont={* Medium}]{Noto Serif}
\setsansfont[BoldFont={* Semibold}]{Source Sans Pro}
\setmonofont[Scale=0.9]{Fira Mono}

everything works as expected.

Obviously, the \setmainfont and friends invocation from fontspecis necessary.

[jbezos]

There declarations are not equivalent. Have you tried with the following?

\babelfont{rm}[Scale=1,BoldFont={* Medium}]{Noto Serif}
\babelfont{sf}[BoldFont={* Semibold}]{Source Sans Pro}
\babelfont{tt}[Scale=0.9]{Fira Mono}

[jbezos]

I’m closing this issue because it just seems an improper use of \babelfont.

[kleinbottle]

I’m closing this issue because it just seems an improper use of \babelfont.

I am sorry I did not see your reply until today.

I thought I was using \babelfont properly. Kindly tell me why what I have done is "an improper use of \babelfont".

Are you saying that omitting the BoldFont={* Medium} declaration in \babelfont led me to lose the bold smallcaps, and that if that had been correctly done, I would have got the same results as with \setmainfont?

Thanks.

[jbezos]

Please, have you tried with the declarations above? Can you post a MWE with this particular issue?

[kleinbottle]

Please, have you tried with the declarations above? Can you post a MWE with this particular issue?

I have tried what you have suggested. To make it an MWE, I have removed the foreign scripts. There are two versions of the file, one with a language declaration along with the font, and another without. It appears that setting the language of the main font causes the error I have encountered. The MWEs and the resulting outputs are shown below:

\documentclass[12pt,a4paper]{article}
\usepackage{fontspec}
\usepackage[british]{babel}
%
\defaultfontfeatures{Ligatures=TeX,Scale=MatchLowercase,Renderer=HarfBuzz}
%
% Without language declaration for the main language
%
\babelfont{rm}[Scale=1,BoldFont={* Medium}]{Noto Serif}
\babelfont{sf}[BoldFont={* Semibold}]{Source Sans Pro}
\babelfont{tt}[Scale=0.9]{Fira Mono}
%
\setlength{\parindent}{0pt}
%
\begin{document}
%
\section*{Latin Script}

\textrm{Roman}

\textsf{Sans Serif}

\texttt{Monospace}

\textsc{Small Caps}

{\bfseries\scshape Bold Small Caps}

{\bfseries\scshape\itshape Bold Small Caps Italic}

\textit{Italic}

\textbf{\textit{Bold Italic}}
%
\end{document}

gives and this is both correct and as expected. Your suggestion works correctly.

However, if the main language is specified, as in this MWE, the small caps bold does not work:

\documentclass[12pt,a4paper]{article}
\usepackage{fontspec}
\usepackage[british]{babel}
%
\defaultfontfeatures{Ligatures=TeX,Scale=MatchLowercase,Renderer=HarfBuzz}
%
% With language declaration for the main language
%
\babelfont[british]{rm}[Scale=1,BoldFont={* Medium}]{Noto Serif}
\babelfont[british]{sf}[BoldFont={* Semibold}]{Source Sans Pro}
\babelfont[british]{tt}[Scale=0.9]{Fira Mono}
%
\setlength{\parindent}{0pt}
%
\begin{document}
%
\section*{Latin Script}

\textrm{Roman}

\textsf{Sans Serif}

\texttt{Monospace}

\textsc{Small Caps}

{\bfseries\scshape Bold Small Caps}

{\bfseries\scshape\itshape Bold Small Caps Italic}

\textit{Italic}

\textbf{\textit{Bold Italic}}
%
\end{document}

and the result is:

It appears that the "improper use of \babelfont" you have alluded to stems from declaring a language for the font associated with the main language.

[jbezos]

I see. It seems related to https://github.com/latex3/babel/issues/92.

[kleinbottle]

Yes, indeed. So it seems.