avoid patching \resetMathstrut@ for recent amsmath.

[davidcarlisle]

amsmath 2.15 (for luatex 0.87+)

development builds of luatex always report \Umathchar in \meaning strings even if the token was defined with \mathchar, this broke several places where amsmath queried \meaning.

amsmath 2.15 hopefully addresses that and has some basic knowledge of \umathchar in other places (eg mathchar and Umathchar tokens should be treated the same by \dots now)

However in order to make it load at all i needed to fix the sdt@minus and mathstrut tests to be luatex/xetex aware.

The end result is that you now get

*************************************************
* lualatex-math warning: "wrong-meaning"
* 
* I've expected \resetMathstrut@ to have the meaning
* macro:->\setbox \z@ \hbox {\mathchardef \@tempa \mathcode `\(\relax \def
* \@tempb ##1"##2##3{\the \textfont "##3\char "}\expandafter \@tempb \meaning
* \@tempa \relax }\ht \Mathstrutbox@ \ht \z@ \dp \Mathstrutbox@ \dp \z@ ,
* but it has the meaning
* macro:->\begingroup \Umathcharnumdef \@tempa \Umathcodenum `\(\relax \def
* \@tempb ##1"##2"##3"##4\relax {\endgroup \ht \Mathstrutbox@ =\fontcharht
* \textfont "##3 "##4\relax \dp \Mathstrutbox@ =\fontchardp \textfont "##3
* "##4\relax } \expandafter \@tempb \meaning \@tempa \relax .
*************************************************

Perhaps the simplest thing to do would be to check if it is the original version and if so, patch, and if not, silently do nothing and hope that the updated version is OK?

More generally hope to make amsmath work "out of the box" with luatex and xetex so there may be further places where patching fails if there are updates during texlive 2016 testing. If you have any particular things you think we should definitely update, then comments on amsmath welcome! (LaTeX3 project has taken over maintenance of amsmath from the AMS)

[jfbu]

@davidcarlisle I allow myself intervening here for a mathastext related reason, but don't want to derail from the original topic. In some circumstances mathastext makes ( math active. This was not compatible with amsmath's \resetMathstrut@ which caused then errors (in special circumstances see below). With the lualatex-math version, there were no errors but bad computation of the box parameters. The not satisfying way-out in mathastext is to use \char40 of operators instead (and the ( of the text font) for measuring dimensions.

Actually mathastext makes the ( active usually after the \resetMathStrut@ execution time, but

$...\hbox{...$..$..}...$

causes a problem nevertheless because in the inner math mode, then ( is mathematically active already before entering math mode. This is why I had to patch the amsmath macro anyway (in some circumstances).

All this to alert to the fact that this amsmath macro is not compatible with math active (.

[davidcarlisle]

@jfbu I don't see the issue, do you have a test document that shows what goes wrong, neither the original nor luatex version of \resetMathstrut@ typesets a ( in math mode so should not be affected by it being mathcode "8000 (which is what I assume you mean by math active)?

[davidcarlisle]

@jfbu oh sorry, yes

\mathcode`\(

doesn't do quite the right thing in that case:-)

[phst]

I even think that now, with amsmath fixed, the patch should be removed altogether. lualatex-math was always intended as a stopgap measure until the base packages are fixed to work with LuaTeX. Removing the mathstrut patch should only cause trouble if a user has a recent version of lualatex-math, but an outdated amsmath version, which seems rather unlikely. What do you think?

[davidcarlisle]

On 7 March 2016 at 21:39, Philipp Stephani notifications@github.com wrote:

I even think that now, with amsmath fixed, the patch should be removed altogether.

Sounds reasonable to me.

lualatex-math was always intended as a stopgap measure until the base packages are fixed to work with LuaTeX. Removing the mathstrut patch should only cause trouble if a user has a recent version of lualatex-math, but an outdated amsmath version, which seems rather unlikely. What do you think?

thanks for the quick response,

David

[jfbu]

What will happen with \newmcodes@ from amsopn.sty? I checked now after updating my installation and don't see changes so far. This was also patched by lualatex-math.

[davidcarlisle]

@jfbu that doesn't have the same issue as it is just setting rather than querying with \meaning, and also luatex has changed again 0.89.3 has reverted to \meaning on a \mathchardef defined token showing as \mathchar rather than \Umathchar but that is currently in luatex experimental branch, so until the next update reaches texlive trunk svn I think I'll hold off adding any more patches unless things are actually broken.

[jfbu]

@davidcarlisle I am not following closely the luatex developments... all I know is that, not related to use of \meaning, at some point

\mathchardef\std@minus\mathcode`\-\relax

was broken when a Umathcode had been assigned to the -.

[davidcarlisle]

On 8 March 2016 at 11:09, jfbu notifications@github.com wrote:

@davidcarlisle https://github.com/davidcarlisle I am not following closely the luatex developments... all I know is that, not related to use of \meaning, at some point

\mathchardef\std@minus\mathcode`-\relax

was broken when a Umathcode had been assigned to the -.

Possibly. I did fix the code in amsmath to handle that possibility, I'll try to test amsopn as well. (I have no test files from the AMS yet so want to limit changes until I have some approximation to a reasonable regression test suite in place. amsmath was generating an error in every document using math so fixing that was sufficiently urgent that got the AMS to agree on an accelerated handover:-)

[davidcarlisle]

@jfbu I am adding this test file, but xetex and luatex seem to work as expected with amsopn even if - has been set up via \Umathcode

\documentclass{article}

\input{test2e}

% Check that tests for the mathcode of - during package load
% do not fail if Umathcode already assigned

\ifx\Umathcode\undefined\else
\Umathcode`\- 2 2 0
\fi
\START
\usepackage{amsopn}

\DeclareMathOperator{\foobar}{foo-bar}

\begin{document}

foo-bar

$\phi -\foobar \phi$

\end{document}

(just remove test2e and \START if you don't have them)

[jfbu]

@davidcarlisle

could you try with

\documentclass{article}
\usepackage{fontspec}

\input{test2e}

% Check that tests for the mathcode of - during package load
% do not fail if Umathcode already assigned
\usepackage{amsopn}

\ifx\Umathcode\undefined\else
\Umathcode`\- 2 0 "2013
\fi
\START

% comment this out for successful compilation
% \makeatletter
% \let\newmcodes@\relax
% \makeatother

\DeclareMathOperator{\foobar}{foo-bar}

\begin{document}

foo-bar

$\phi -\foobar \phi$

\end{document}

(I don't have test2e or \START hence didn't try with them)

[davidcarlisle]

Works without error (and expected result) with luatex .89.3, gives an error in xelatex

! Extended mathchar used as mathchar (4202515).

will fix....

[jfbu]

ok on my installation (uptodate TL2015) both lualatex and xelatex give this error.

[jfbu]

but I forgot to say again that this issue is fixed by lualatex-math for lualatex compilation.

[davidcarlisle]

You don't want to know how many times the handling of \mathchardef has changed in luatex between 0.80 (texlive 2015) and 0.89.3:-)

David

On 8 March 2016 at 12:46, jfbu notifications@github.com wrote:

ok on my installation (uptodate TL2015) both lualatex and xelatex give this error.

— Reply to this email directly or view it on GitHub https://github.com/phst/lualatex-math/issues/17#issuecomment-193770998.

http://dpcarlisle.blogspot.com/

[jfbu]

@davidcarlisle I have been witness to some or your exchanges with Hans ;-) on luatex mailing list... but indeed can only deduce there were quite a few changes and back-changes, as I don't follow really what happens on luatex side.

regarding \newmcodes@ I understand topmost priority is maintaining identical compilation for legacy documents; but I was thinking that for new engines, in case for example the minus sign has an extended mathcode, perhaps then it would make sense to re-think what \newmcodes@ does. In particular it hard-codes font family 0 for some symbols such as * when used in operator names. But myself I have not given any serious thought about it apart from doing \let\newmcodes@\relax in mathastext.sty as I was too lazy (I was to improve it but don't have had the time to devote to this) and it was not really of my resort.

[davidcarlisle]

@jfbu I updated amsopn in latex svn so your example works in xe and lua tex. @phst I guess that means that the amsopn patch in lualatex-math needs to go too once that goes live (scheduled for latex release on 31st of this month)

https://latex-project.org/svnroot/latex2e-public/trunk/required/amsmath

[jfbu]

@davidcarlisle great, thanks. Is in

%    use `|\Umathcodedefnum| with xetex and LuaTeX to avoid problems

the `` there a typo ? (... and is really\Umathcodedefnum` intended in this comment ?)

I would argue that amsopn should have either set the characters to "variable family" rather than fam 0 or directly used \symoperators rather than 0.

An amsmath comment says

%    In operator names, it is sometimes desired to have text-mode
%    punctuation characters such as |*-/:'|.

But \newmcodes@ picks most of the non-letter characters it handles from \fam0 while the letters in operator names are typeset under \symoperators regime (from \operator@font= \mathgroup\symoperators). This ties \symoperators to be 0 which is annoying. And I see amsmath picks the dot character from \fam1, the letters font. Well, there must be a reason, but anyway I guess a modern re-write should use \symoperators and \symletters rather than hard-coded 0 and 1.

[davidcarlisle]

On 8 March 2016 at 22:48, jfbu notifications@github.com wrote:

@davidcarlisle https://github.com/davidcarlisle great, thanks. Is in

% use `|\Umathcodedefnum| with xetex and LuaTeX to avoid problems

the ` there a typo ?

looks that way I'll zap it, thanks:-)

I would argue that amsopn should have either set the characters to "variable family" rather than fam 0 or directly used \symoperators rather than 0. Indeed one finds in \qopname's definition. Perhaps this would not be ok for all concerned characters,

An amsmath comment says

% In operator names, it is sometimes desired to have text-mode % punctuation characters such as |*-/:'|.

But \newmcodes@ picks most of the non-letter characters it handles from \fam0 while the letters in operator names are typeset under \symoperators regime (from \operator@font= \mathgroup\symoperators). This ties \symoperators to be 0 which is annoying. And I see amsmath picks the dot character from \fam1, the letters font. Well, there must be a reason, but anyway I guess a modern re-write should use \symoperators and \symletters rather than hard-coded 0 and 1.

yes I wondered about those fam settings.

But probably we are abusing the lualatex-math issue tracker and should move this elsewhere,

I am pretty much restricting initially to things that give errors when used with Unicode texs.

Before doing general improvements I need more regression tests. but feel free to put issues in (currently latex-bug tracker, although we may set up a github tracker as well, as we do for the latex3 code)

[jfbu]

sorry @phst for having indeed abused the issue tracker here. A github tracker for latex would be nice, as I am now familiar with its interface...