gregorio-project / gregorio

The Gregorio Project
http://gregorio-project.github.io
Other
164 stars 43 forks source link

Square notes and lyrics disppear with \usepackage{luatexja} #1107

Closed anthonyfok closed 8 years ago

anthonyfok commented 8 years ago

There appear to be some kind of incompatibility (glyph encoding clash) with Gregorio and LuaTeX-ja. With \usepackage{luatexja} enabled, the Gregorio neumes and lyrics tend to disappear or are replaced by a CJK character. See @benstox report on MusicaSacra forum from October 2013:

About two years ago around February to April 2014, I had to try to work around a similar issue, with gregorio (2.3-1) running on Ubuntu. I eventually used a self-defined LaTeX command to turn off Chinese (\zhOff) before including a Gregorio chant, and turn it back on afterward, with something like the following:

% LuaTeX-ja for typesetting Chinese
\let\printglossary\relax        % duplicate definition in "memoir"
\usepackage{luatexja-fontspec}
% Glyph conflict with Gregorio
%\ltjsetparameter{jacharrange={-1, +2, +3, -4, -5, +6, +7, +8}}
%%\ltjdefcharrange{10}{"3C06-"3C08} % 㰆  䐤 
%\ltjdefcharrange{10}{"3400-"4DBF} % 㰆  䐤 
%\ltjsetparameter{jacharrange={-2, -3, -8, -10}}

% U+2014: EM DASH, also used as 破折號, originally in LuaTeX-ja Range 3
% U+2026: …
% U+25CB: WHITE CIRCLE, originally in LuaTeX-ja Range 3
\ltjdefcharrange{10}{"2014, "2026, "25CB}
\ltjsetparameter{jacharrange={-2, -3, -8}}

\newcommand*{\zhOn}{%
  \ltjsetparameter{jacharrange={-1, -2, -3, -4, -5, +6, +7, -8, +10}}%
}
\newcommand*{\zhOff}{%
  \ltjsetparameter{jacharrange={-1, -2, -3, -4, -5, -6, -7, -8, -10}}%
}

Earlier today, I was trying to print a simple-tone Regina Cæli with Chinese translation underneath, but upon adding \uselatex{luatexja} to the example main-lualatex.tex, all square notes and lyrics disappear, though interestingly, the St. Gall neumes (nabc) notations remain. The kludge I used two years ago does not seem to work any more, though perhaps I was doing something wrong.

Anyhow, I was wondering if there are better and hopefully easier ways to keep Gregorio and LuaTeX-ja in harmony? It would also be a nice addition to the website or reference manual for those with this special need, whether it be typesetting Gregorian chant with Chinese/Japanese lyrics/translation, or typesetting Gregorian chant within a Chinese/Japanese book. :-) Many thanks!

Anthony

rpspringuel commented 8 years ago

The Gregorio glyphs were moved into one of the designated private use areas of the Unicode spec in 3.0 (IIRC) so any work around I would expect font hacks like this to be different on either side of the change.

That said, I'm not sure why the Chinese characters would be invading the Gregorio fonts in the first place. Doesn't the Unicode spec have Chinese characters in it? And even if it doesn't, why would luatexja be effecting the Gregorio fonts instead of (or possibly in addition to) the main document font?

In any event, if you can turn the Chinese characters on and off and only need them in the translation, then I would alter the translation style so that the characters get turned on just for that portion of the score. Unfortunately there is not an equivalent for the lyrics, but that's something we could probably add relatively easily.

✝✝✝✝✝✝✝✝✝✝✝✝✝✝✝✝✝✝✝✝✝✝✝✝ Br. Samuel, OSB (R. Padraic Springuel) PAX ☧ ΧΡΙΣΤΟΣ

On May 14, 2016, at 5:29 PM, Anthony Fok notifications@github.com wrote:

There appear to be some kind of incompatibility (glyph encoding clash) with Gregorio and LuaTeX-ja. With \usepackage{luatexja} enabled, the Gregorio neumes and lyrics tend to disappear or are replaced by a CJK character. See @benstox report on MusicaSacra forum from October 2013:

http://forum.musicasacra.com/forum/discussion/9808/chinese-plainchant-in-gregorio About two years ago around February to April 2014, I had to try to work around a similar issue, with gregorio (2.3-1) running on Ubuntu. I eventually used a self-defined LaTeX command to turn off Chinese (\zhOff) before including a Gregorio chant, and turn it back on afterward, with something like the following:

% LuaTeX-ja for typesetting Chinese \let\printglossary\relax % duplicate definition in "memoir" \usepackage{luatexja-fontspec} % Glyph conflict with Gregorio %\ltjsetparameter{jacharrange={-1, +2, +3, -4, -5, +6, +7, +8}} %%\ltjdefcharrange{10}{"3C06-"3C08} % 㰆 䐤 %\ltjdefcharrange{10}{"3400-"4DBF} % 㰆 䐤 %\ltjsetparameter{jacharrange={-2, -3, -8, -10}}

% U+2014: EM DASH, also used as 破折號, originally in LuaTeX-ja Range 3 % U+2026: … % U+25CB: WHITE CIRCLE, originally in LuaTeX-ja Range 3 \ltjdefcharrange{10}{"2014, "2026, "25CB} \ltjsetparameter{jacharrange={-2, -3, -8}}

\newcommand{\zhOn}{% \ltjsetparameter{jacharrange={-1, -2, -3, -4, -5, +6, +7, -8, +10}}% } \newcommand{\zhOff}{% \ltjsetparameter{jacharrange={-1, -2, -3, -4, -5, -6, -7, -8, -10}}% } Earlier today, I was trying to print a simple-tone Regina Cæli with Chinese translation underneath, but upon adding \uselatex{luatexja} to the example main-lualatex.tex, all square notes and lyrics disappear, though interestingly, the St. Gall neumes (nabc) notations remain. The kludge I used two years ago does not seem to work any more, though perhaps I was doing something wrong.

Anyhow, I was wondering if there are better and hopefully easier ways to keep Gregorio and LuaTeX-ja in harmony? It would also be a nice addition to the website or reference manual for those with this special need, whether it be typesetting Gregorian chant with Chinese/Japanese lyrics/translation, or typesetting Gregorian chant within a Chinese/Japanese book. :-) Many thanks!

Anthony

— You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub

rpspringuel commented 8 years ago

Okay, now that I have a little more time and am at my desk, let me add information.

The Gregorio glyphs are now in the BMP private use area (U+E000 to U+F8FF, though we do not currently use all of these code points). This was implemented in 6cad4bb (which means it first appeared in 3.0). As such these are in what luatexjp defines as Range 4. Further, these glyphs are in their own font (Grecillae, Gregorio, Grana Panado, and their Dominican variants) which is invoked by the \gre@font@music command internally. As such, in order to work correctly this character range, in this font, must not be manipulated by luatexjp. Not knowing exactly how luatexjp works, I cannot say for sure that this is the case. Indeed, given that you're reporting a conflict between the two packages, I would say that this is not happening (i.e. luatexjp is some how manipulating that character range in the gregorio font.)

The St. Gall glyphs are also in the same BMP private use area, but start at U+E400 and only run to U+E63E. They are also in their own font (Gregall) which is invoked with \gre@font@nabc. If they are appearing while the regular glyphs are not, then that implies that luatexjp treats this range differently (documentation to the contrary notwithstanding).

One thing you might try is changing the package loading order. Depending on how luatexjp works, when you load gregoriotex (and thus define \gre@font@music) might change the way it manipulates the code points in our fonts.

If that makes no difference, then you should try building an environment/command that allows you to set a single Chinese word in an otherwise English paragraph. If you can do that then I might be able to adapt that into some sort of work around.

Along those lines, it looks like there are other packages (CJKutf8, for instance) which might make that task easier.

anthonyfok commented 8 years ago

Dear Br. Samuel:

Thank you for your very quick and informative reply, with lots of great and helpful ideas for resolving this issue. I will try out your suggestions in the coming days. (I am a very slow TeX learner, hehe.)

Thanks again, and God bless!

anthonyfok commented 8 years ago

Hi again!

I have had some time today to play with it a bit more under TeX Live 2016 pretest, and after much struggle (because I know too little about TeX), came up with the following plain LuaTeX example:

\input luatexja.sty
\input gregoriotex.tex

日本語

{\makeatletter\gre@fontchar@flat}
{\makeatletter\gre@font@music\GreCPTorculusOneTwoNothing}

\gregorioscore[a]{PopulusSion}

{\makeatletter\gre@fontchar@flat}
{\makeatletter\gre@font@music\GreCPTorculusOneTwoNothing}

日本語
\end

luatexja-and-gregoriotex-example

In this example, printing an individual glyph using \makeatletter\gre@font@music (which I found in one of your earlier posts in the mailing list) works perfectly! It is only when a score is included, say, with \gregorioscore, that the output is mostly incomplete. Strangely, even the lyrics are gone too, but the italic Ps. remains.

It would seem that I have made a wrong assumption based on the somewhat different behaviour in Gregorio 2.13 (and TeX Live 2013). In TeX Live 2016, I think it is no longer the case where Japanese char range overlaps with U+E000 to U+F8FF (so we saw CJK characters appeared where the neumes should be), but perhaps some other primitives (?) or font settings that LuaTeX-ja may have overridden.

By the way, loading luatexja and gregoriotex in the opposite order in plain LuaTeX, i.e.:

\input gregoriotex.tex
\input luatexja.sty

would result in the following error when loading xkeyval.sty:

(/usr/share/texlive/texmf-dist/tex/latex/xkeyval/xkeyval.sty
(/usr/share/texlive/texmf-dist/tex/generic/xkeyval/xkeyval.tex
(/usr/share/texlive/texmf-dist/tex/generic/xkeyval/xkvutils.tex
(/usr/share/texlive/texmf-dist/tex/generic/xkeyval/keyval.tex))
Runaway definition?
#1#2#3#4#5->\expandafter 
! Forbidden control sequence found while scanning definition of \XKV@d@f@ne@b@o
lkey.
<inserted text> 
}
<to be read again> 
\newif 
l.224   \expandafter\newif
                        \csname if#3\endcsname
? 

Not sure if that is related to this issue.

Thank you for letting me know of CJKutf8. I used to use its older non-UTF-8 version "CJK" quite a few years ago before LuaTeX.

And, to my pleasant surprise, using only fontspec allows one to typeset some minimal Chinese characters, e.g., the following example found at http://tex.stackexchange.com/questions/222980/how-to-use-chinese-with-lualatex:

\documentclass{article}
\usepackage{fontspec}
\setmainfont{MS PGothic}
\begin{document}
我希望這是,翻譯是由谷歌提供的中國文字。
\end{document}

(Caveat: the Chinese text does not wrap properly.)

But yes, since LuaTeX-ja seems to be the solution the community has settled on for professional Japanese publishing with LuaTeX, and even CTeX for Chinese publishing now simply provides a wrapper around LuaTeX-ja, it would be wonderful if a solution could be found eventually. :-)

Cheers, Anthony

anthonyfok commented 8 years ago

Note-to-self: Remember to file an issue at LuaTeX-ja too, as it might be an issue that need fixing on their side, for example, LuaTeX-ja not yet catching up to the new TU encoding (rather than the old EU2 encoding) in 2016 fontspec. ← (A wild random guess from me who doesn't know any better.)

rpspringuel commented 8 years ago

Looking at that output, it appears that \GreBarSyllable works fine but \GreSyllable does not. I have no idea why that's the case, but it gives me something to go on.

rpspringuel commented 8 years ago

Potential fix in the works. Just need to check it against the full battery of tests.

rpspringuel commented 8 years ago

@anthonyfok, if you have the capability to build Gregorio from source, please grab the branch from #1123 and make sure that the fix works for you.

henryso commented 8 years ago

Merged into both master and release-4.2 branches.