PDF: Bold MathML not rendering

opoudjis commented 1 year ago

Follow on from https://github.com/metanorma/metanorma-iso/issues/876

I have introduced code to insert <mstyle fontweight="bold"> into MathML if its ancestor is <strong>. However, I have discovered that style directives to render MathML in boldface are being ignored in PDF, whether they are the older <mstyle fontweight="bold">, or the newer <mstyle style="font-weight:bold">, or the native <mstyle mathvariant="bold">.

So the following snippet renders the b in boldface (non-italic) as expected in HTML and DOC, but as plain style in PDF:

<stem type="MathML">
  <math xmlns="http://www.w3.org/1998/Math/MathML">
    <mi>a</mi> 
    <mstyle mathvariant="bold">
      <mi>b</mi>
    </mstyle>
  </math><!-- a b -->
  <asciimath>a bb(b)</asciimath>
</stem>

And it renders the same whether the mstyle has mathvariant="bold", style="font-weight:bold", or fontweight="bold".

Intelligent2013 commented 1 year ago

I can't repeat. This XML:

<p id="_test">
Bold math test: 
<stem type="MathML">
  <math xmlns="http://www.w3.org/1998/Math/MathML">
    <mi>a</mi> 
    <mstyle mathvariant="bold">
      <mi>b</mi>
    </mstyle>
  </math><!-- a b -->
  <asciimath>a bb(b)</asciimath>
</stem> 
</p>

renders so:

@opoudjis could you attach the full xml fragment?

Intelligent2013 commented 1 year ago

I've test for different styles mathvariant="bold", style="font-weight:bold", or fontweight="bold", and only mathvariant="bold" renders ok in PDF:

opoudjis commented 1 year ago

The mstyle is at the root of the MathML, and is still failing in PDF with mathvariant. Attached.

Αρχειοθήκη 2.zip

Intelligent2013 commented 1 year ago

In my environment PDF looks so:

vs. from Αρχειοθήκη 2.zip:

I don't figure out the reason. I'll investigate.

Intelligent2013 commented 1 year ago

The document from mn-samples-iso repo https://github.com/metanorma/mn-samples-iso/blob/main/sources/directives/part2/sections/28-figures.adoc contains similar math in bold:

| Position vector | stem:[bb(r)]
| Displacement | stem:[bb(Delta r)]

<tr><td valign="top" align="left">Position vector</td>
<td valign="top" align="left"><stem type="MathML"><math xmlns="http://www.w3.org/1998/Math/MathML"><mstyle mathvariant="bold"><mi>r</mi></mstyle></math><asciimath>bb(r)</asciimath></stem></td>
</tr><tr><td valign="top" align="left">Displacement</td>
<td valign="top" align="left"><stem type="MathML"><math xmlns="http://www.w3.org/1998/Math/MathML"><mstyle mathvariant="bold"><mrow><mo>Δ</mo><mi>r</mi></mrow></mstyle></math><asciimath>bb(Delta r)</asciimath></stem></td>
</tr>

and math in PDF rendered in bold on Win, Ubuntu and Mac (https://github.com/metanorma/mn-native-pdf/pull/488), Table 10 on the page 53: Windows (https://github.com/metanorma/mn-native-pdf/actions/runs/4073826433):

Ubuntu (https://github.com/metanorma/mn-native-pdf/actions/runs/4073826407):

MacOS (https://github.com/metanorma/mn-native-pdf/actions/runs/4073826414):

Will investigate further...

Intelligent2013 commented 1 year ago

I assume that wrong (old?) Cambria font without Math sub-font is using.

@opoudjis could you help make some actions?

re-generate PDF again
immediately after that, go to the folder /Users/<user name>/.metanorma/fonts/ and find the file pdf_fonts_config.xml.out.
attach it here (or via Skype)

in the file pdf_fonts_config.xml.out find two elements font with <font-triplet name="Cambria Math" style="normal" weight="normal"/> and <font-triplet name="Cambria Math" style="normal" weight="bold"/> inside it, like this:

<font embed-url="file:/C:/Windows/FONTS/cambria.ttc" kerning="yes" mn_default="false" sub-font="Cambria Math">
<alternate embed-url="cambria.ttf"/>
<font-triplet name="Cambria Math" style="normal" weight="normal"/>
</font>
<font embed-url="file:/C:/Windows/FONTS/cambria.ttc" kerning="yes" mn_default="false" simulate-style="true" sub-font="Cambria Math">
<alternate embed-url="cambria.ttf"/>
<font-triplet name="Cambria Math" style="normal" weight="bold"/>
</font>

attach the font files specified in the attribute embed-url.

Thanks.

Intelligent2013 commented 1 year ago

@opoudjis thanks.

From pdf_fonts_config.xml.out - Cambria.ttf is using for Cambria Math:

<font embed-url="file:/Library/Fonts/Microsoft/Cambria.ttf" kerning="yes" mn_default="false">
    <alternate embed-url="cambria.ttf"/>
    <font-triplet name="Cambria" style="normal" weight="normal"/>
</font>
<font embed-url="file:/Library/Fonts/Microsoft/Cambria.ttf" kerning="yes" mn_default="false">
    <alternate embed-url="cambria.ttf"/>
    <font-triplet name="Cambria Math" style="normal" weight="normal"/>
</font>
<font embed-url="file:/Library/Fonts/Microsoft/Cambria.ttf" kerning="yes" mn_default="false">
    <alternate embed-url="cambria.ttf"/>
    <font-triplet name="Cambria Math" style="normal" weight="bold"/>
</font>

But on the disk there is the font Cambria Math in Cambria Math.ttf. Also, missing simulate-style="true" for <font-triplet name="Cambria Math" style="normal" weight="bold"/>. Will fix it.

Intelligent2013 commented 1 year ago

I'm trying to generate PDF for a.doc on MacOS and get the error:

bundle exec metanorma -t iso -x pdf a.adoc:

Calling `DidYouMean::SPELL_CHECKERS.merge!(error_name => spell_checker)' has been deprecated. Please call `DidYouMean.correct_error(error_name, spell_checker)' instead.
bundler: failed to load command: metanorma (/usr/local/lib/ruby/gems/3.2.0/bin/metanorma)
/usr/local/lib/ruby/gems/3.2.0/gems/metanorma-cli-1.6.12/lib/metanorma/cli/commands/config.rb:57:in `block in load_configs': undefined method `exists?' for File:Class (NoMethodError)

            next unless File.exists?(config_path)
                            ^^^^^^^^
Did you mean?  exist?
    from /usr/local/lib/ruby/gems/3.2.0/gems/metanorma-cli-1.6.12/lib/metanorma/cli/commands/config.rb:56:in `each'
    from /usr/local/lib/ruby/gems/3.2.0/gems/metanorma-cli-1.6.12/lib/metanorma/cli/commands/config.rb:56:in `load_configs'
    from /usr/local/lib/ruby/gems/3.2.0/gems/metanorma-cli-1.6.12/lib/metanorma/cli/thor_with_config.rb:14:in `options'
    from /usr/local/lib/ruby/gems/3.2.0/gems/metanorma-cli-1.6.12/lib/metanorma/cli/command.rb:46:in `compile'
...

ruby -v:

ruby 3.2.0 (2022-12-25 revision a528908271) [x86_64-darwin20]

Will try to re-install metanorma.

Intelligent2013 commented 1 year ago

File.exists? were removed from Ruby 3.2.0.

Intelligent2013 commented 1 year ago

Issue with File.exists? fixed by adding this line into Gemfile: gem "metanorma-cli", git: "https://github.com/metanorma/metanorma-cli", branch: "main"

New error occurs:

Fatal Error: Refinement#include has been removed
bundler: failed to load command: metanorma (/usr/local/lib/ruby/gems/3.2.0/bin/metanorma)
/usr/local/lib/ruby/gems/3.2.0/gems/rdf-n3-3.1.2/lib/rdf/n3/refinements.rb:176:in `include': Refinement#include has been removed (TypeError)

    include ::RDF::Term
            ^^^^^^^^^^^
    from /usr/local/lib/ruby/gems/3.2.0/gems/rdf-n3-3.1.2/lib/rdf/n3/refinements.rb:176:in `block in <module:Refinements>'
    from /usr/local/lib/ruby/gems/3.2.0/gems/rdf-n3-3.1.2/lib/rdf/n3/refinements.rb:174:in `refine'
    from /usr/local/lib/ruby/gems/3.2.0/gems/rdf-n3-3.1.2/lib/rdf/n3/refinements.rb:174:in `<module:Refinements>'
    from /usr/local/lib/ruby/gems/3.2.0/gems/rdf-n3-3.1.2/lib/rdf/n3/refinements.rb:2:in `<top (required)>'
    from /usr/local/lib/ruby/gems/3.2.0/gems/rdf-n3-3.1.2/lib/rdf/n3.rb:26:in `require'
    from /usr/local/lib/ruby/gems/3.2.0/gems/rdf-n3-3.1.2/lib/rdf/n3.rb:26:in `<module:N3>'
    from /usr/local/lib/ruby/gems/3.2.0/gems/rdf-n3-3.1.2/lib/rdf/n3.rb:21:in `<module:RDF>'
    from /usr/local/lib/ruby/gems/3.2.0/gems/rdf-n3-3.1.2/lib/rdf/n3.rb:3:in `<top (required)>'
    from /usr/local/lib/ruby/gems/3.2.0/gems/linkeddata-3.1.6/lib/linkeddata.rb:26:in `require'
    from /usr/local/lib/ruby/gems/3.2.0/gems/linkeddata-3.1.6/lib/linkeddata.rb:26:in `<module:LinkedData>'
    from /usr/local/lib/ruby/gems/3.2.0/gems/linkeddata-3.1.6/lib/linkeddata.rb:1:in `<top (required)>'
    from /usr/local/lib/ruby/gems/3.2.0/gems/relaton-w3c-1.14.0/lib/relaton_w3c/data_fetcher.rb:2:in `require'
    from /usr/local/lib/ruby/gems/3.2.0/gems/relaton-w3c-1.14.0/lib/relaton_w3c/data_fetcher.rb:2:in `<top (required)>'
    from /usr/local/lib/ruby/gems/3.2.0/gems/relaton-w3c-1.14.0/lib/relaton_w3c.rb:8:in `require'
    from /usr/local/lib/ruby/gems/3.2.0/gems/relaton-w3c-1.14.0/lib/relaton_w3c.rb:8:in `<top (required)>'
    from /usr/local/lib/ruby/gems/3.2.0/gems/relaton-1.14.2/lib/relaton/registry.rb:28:in `require'
    from /usr/local/lib/ruby/gems/3.2.0/gems/relaton-1.14.2/lib/relaton/registry.rb:28:in `block in register_gems'
    from /usr/local/lib/ruby/gems/3.2.0/gems/relaton-1.14.2/lib/relaton/registry.rb:27:in `each'
    from /usr/local/lib/ruby/gems/3.2.0/gems/relaton-1.14.2/lib/relaton/registry.rb:27:in `register_gems'
    from /usr/local/lib/ruby/gems/3.2.0/gems/relaton-1.14.2/lib/relaton/registry.rb:21:in `initialize'
    from /usr/local/Cellar/ruby/3.2.0/lib/ruby/3.2.0/singleton.rb:127:in `new'
    from /usr/local/Cellar/ruby/3.2.0/lib/ruby/3.2.0/singleton.rb:127:in `block in instance'
    from /usr/local/Cellar/ruby/3.2.0/lib/ruby/3.2.0/singleton.rb:125:in `synchronize'
    from /usr/local/Cellar/ruby/3.2.0/lib/ruby/3.2.0/singleton.rb:125:in `instance'
    from /usr/local/lib/ruby/gems/3.2.0/gems/relaton-1.14.2/lib/relaton/db.rb:6:in `initialize'
    from /usr/local/lib/ruby/gems/3.2.0/gems/relaton-1.14.2/lib/relaton/db.rb:530:in `new'
    from /usr/local/lib/ruby/gems/3.2.0/gems/relaton-1.14.2/lib/relaton/db.rb:530:in `init_bib_caches'
    from /usr/local/lib/ruby/gems/3.2.0/gems/metanorma-standoc-2.3.7/lib/metanorma/standoc/ref_sect.rb:166:in `init_bib_caches'
    from /usr/local/lib/ruby/gems/3.2.0/gems/metanorma-standoc-2.3.7/lib/metanorma/standoc/base.rb:119:in `init_biblio'
    from /usr/local/lib/ruby/gems/3.2.0/gems/metanorma-standoc-2.3.7/lib/metanorma/standoc/base.rb:40:in `init'
    from /usr/local/lib/ruby/gems/3.2.0/gems/metanorma-iso-2.3.3/lib/metanorma/iso/base.rb:54:in `init'
    from /usr/local/lib/ruby/gems/3.2.0/gems/metanorma-standoc-2.3.7/lib/metanorma/standoc/base.rb:140:in `document1'
    from /usr/local/lib/ruby/gems/3.2.0/gems/metanorma-standoc-2.3.7/lib/metanorma/standoc/base.rb:130:in `document'
    from /usr/local/lib/ruby/gems/3.2.0/gems/metanorma-standoc-2.3.7/lib/metanorma/standoc/utils.rb:13:in `convert'
    from /usr/local/lib/ruby/gems/3.2.0/gems/asciidoctor-2.0.18/lib/asciidoctor/document.rb:956:in `convert'
    from /usr/local/lib/ruby/gems/3.2.0/gems/asciidoctor-2.0.18/lib/asciidoctor/convert.rb:120:in `convert'
    from /usr/local/lib/ruby/gems/3.2.0/gems/metanorma-1.5.5/lib/metanorma/input/asciidoc.rb:18:in `process'
    from /usr/local/lib/ruby/gems/3.2.0/gems/metanorma-1.5.5/lib/metanorma/processor.rb:21:in `input_to_isodoc'
    from /usr/local/lib/ruby/gems/3.2.0/gems/metanorma-1.5.5/lib/metanorma/compile.rb:59:in `process_input_adoc'
    from /usr/local/lib/ruby/gems/3.2.0/gems/metanorma-1.5.5/lib/metanorma/compile.rb:42:in `process_input'
    from /usr/local/lib/ruby/gems/3.2.0/gems/metanorma-1.5.5/lib/metanorma/compile.rb:33:in `compile'

opoudjis commented 1 year ago

bundle info rdf is ok, but let me upgrade to Ruby 3.2.0....

... replicated error. Aha.

Please use Ruby 3.1.3 until we've worked this out.

https://github.com/relaton/relaton-w3c/issues/34

Intelligent2013 commented 1 year ago

Please use Ruby 3.1.3 until we've worked this out.

@opoudjis thank you, it's working.

I've repeated the wrong PDF rendering with non-bold math on MacOS.

I've installed these fonts

Cambria.ttf
Cambria Bold.ttf
Cambria Math.ttf on clear MacOS installation.

Fontist manifest looks correctly:

---
Cambria:
  Bold:
    full_name: Cambria Bold
    paths:
    - "/Library/Fonts/Cambria Bold.ttf"
  Regular:
    full_name: Cambria
    paths:
    - "/Library/Fonts/Cambria.ttf"
Cambria Math:
  Regular:
    full_name: Cambria Math
    paths:
    - "/Library/Fonts/Cambria Math.ttf"
...

In the FOP font config (parsed from fontist manifest):

<fonts>
    <font embed-url="file:/Library/Fonts/Cambria.ttf" kerning="yes" mn_default="false" sub-font="Cambria">
        <alternate embed-url="cambria.ttf"/>
        <font-triplet name="Cambria" style="normal" weight="normal"/>
    </font>
    <font embed-url="file:/Library/Fonts/Cambria%20Math.ttf" kerning="yes" mn_default="false" sub-font="Cambria Math">
        <alternate embed-url="cambria.ttf"/>
        <font-triplet name="Cambria Math" style="normal" weight="normal"/>
    </font>
    <font embed-url="file:/Library/Fonts/Cambria.ttf" kerning="yes" mn_default="false">
        <alternate embed-url="cambria.ttf"/>
        <font-triplet name="Cambria Math" style="normal" weight="bold"/>
    </font>
    <font embed-url="file:/Library/Fonts/Cambria%20Bold.ttf" kerning="yes" mn_default="false">
        <alternate embed-url="Cambria Bold.ttf"/>
        <font-triplet name="Cambria" style="normal" weight="bold"/>
    </font>

there are a few wrong items:

for Cambria.ttf specified sub-font="Cambria", but in .ttf there aren't sub-fonts, only in .ttc
for Cambria.ttf - <font-triplet name="Cambria Math" style="normal" weight="bold"/>
there isn't <font-triplet name="Cambria Math" style="normal" weight="bold"/> for file:/Library/Fonts/Cambria%20Math.ttf

There is a differences between my FOP font config (left panel) and provided by @opoudjis (right panel):

there is a redundant sub-font on the left
there isn't Cambria Math on the right at all

I don't figure out why Cambria Math is missing.

At this moment I see two issues:

wrong fontist manifest parsing by mn2pdf
wrong fontist manifest generation on @opoudjis machine (don't sure).

@opoudjis to make sure that fontist manifest is correctly, could you make some actions. please? Fontist manifest .yml file generates as temporary file, therefore it deletes by Ruby garbage collector. To catch the fontist manifest file I've updated mn2pdf-ruby in the branch fontist_catch. Add this line into Gemfile: gem "mn2pdf", git: "https://github.com/metanorma/mn2pdf-ruby", branch: "fontist_catch" Execute bundle update and then run PDF generation.

In the log, before the line java -Xss5m -Xmx2048m you will see the line like C:/Users/TestUser/AppData/Local/Temp/fontist_locations20230203-7164-rldc4h.yml.copy.yml:

C:/Users/TestUser/AppData/Local/Temp/fontist_locations20230203-7164-rldc4h.yml.copy.yml
java -Xss5m -Xmx2048m -jar C:/tools/ruby31/lib/ruby/gems/3.1.0/bundler/gems/mn2pdf-ruby-fc9a124159b9/lib/../bin/mn2pdf.jar --xml-file "D:/Work/Metanorma/repositories/mn-samples-iso/sources/math_bold/a.presentati
on.xml" --xsl-file "C:/tools/ruby31/lib/ruby/gems/3.1.0/gems/metanorma-iso-2.3.3/lib/isodoc/iso/iso.international-standard.xsl" --pdf-file "D:/Work/Metanorma/repositories/mn-samples-iso/sources/math_bold/a.pdf"
--syntax-highlight  --font-manifest "C:/Users/TestUser/AppData/Local/Temp/fontist_locations20230203-7164-rldc4h.yml"
C:/tools/ruby31/lib/ruby/gems/3.1.0/bundler/gems/mn2pdf-ruby-fc9a124159b9/lib/mn2pdf.rb:83:in `mn2pdf'
C:/tools/ruby31/lib/ruby/gems/3.1.0/bundler/gems/mn2pdf-ruby-fc9a124159b9/lib/mn2pdf.rb:58:in `block in convert'
C:/tools/ruby31/lib/ruby/gems/3.1.0/bundler/gems/mn2pdf-ruby-fc9a124159b9/lib/mn2pdf.rb:104:in `block in dump_fontist_manifest_locations'
...
[mn2pdf] Fatal:  Error: Unable to access jarfile C:/tools/ruby31/lib/ruby/gems/3.1.0/bundler/gems/mn2pdf-ruby-fc9a124159b9/lib/../bin/mn2pdf.jar

Please attach this .yml.copy.yml.

The process ends with error due missing mn2pdf.jar, so ignore it. I need just fontist manifest yml.

Thanks.

Intelligent2013 commented 1 year ago

I'll make one more attempts to fix this issue...

Intelligent2013 commented 1 year ago

After update (mn2pdf v.1.64) Apache FOP config looks correct:

<fonts>
    <font embed-url="file:/Library/Fonts/Cambria.ttf" kerning="yes" mn_default="false">
        <alternate embed-url="cambria.ttf"/>
        <font-triplet name="Cambria" style="normal" weight="normal"/>
    </font>
    <font embed-url="file:/Library/Fonts/Cambria%20Math.ttf" kerning="yes" mn_default="false">
        <alternate embed-url="Cambria Math.ttf"/>
        <alternate embed-url="cambria.ttf"/>
        <font-triplet name="Cambria Math" style="normal" weight="normal"/>
    </font>
    <font embed-url="file:/Library/Fonts/Cambria Math.ttf" kerning="yes" mn_default="false" simulate-style="true">
        <alternate embed-url="Cambria Math.ttf" simulate-style="true"/>
        <alternate embed-url="cambria.ttf" simulate-style="true"/>
        <font-triplet name="Cambria Math" style="normal" weight="bold"/>
    </font>
    <font embed-url="file:/Library/Fonts/Cambria%20Bold.ttf" kerning="yes" mn_default="false">
        <alternate embed-url="Cambria Bold.ttf"/>
        <font-triplet name="Cambria" style="normal" weight="bold"/>
    </font>

but PDF still renders in the normal:

To do:

check missing %20 in Cambria Math.ttf
check simulate-style="true" is workable or not for Cambria Math.ttf

Intelligent2013 commented 1 year ago

Looks like this renders in non Cambria Math Bold at all.

Cambria Math bold should looks so:

I'll debug jEuclid rendering.

Intelligent2013 commented 1 year ago

I've spent a day, but didn't find the reason yet...

opoudjis commented 1 year ago

OK, I've already completely forgotten where this came from, and I don't want to add a never ending ticket to your workload, so...

Is this a font issue, specific to Cambria Math Bold?

Is there something I can do in markup to push it along?

I'm confused that you are getting boldface and I'm not; this looks like something that can be gotten around, but it also sounds like it needs explicit invocation of the distinct Cambria Bold. Does this issue generalise to other bold math fonts? I know we've had problems with boldface maths in the past—incomplete coverage of maths symbols in fonts.

Intelligent2013 commented 1 year ago

@opoudjis I assume that the reason in jEuclid. There is 'Cambria Math' only font, not 'Cambria Math Bold'. Apache FOP can 'simulate' font's styles. I.e. if there is only normal/regular font, then Apache FOP automatically 'build' bold/italic fonts. This works ok for text. jEuclid is using for mathml rendering, and it seems jEeuclid doesn't know anything about how to simulate the font. I didn't have enough time to focused on this issue. As soon I've ended with JIS XSLT, then I'll switch to here again.... I think about workaround solution - create on-fly 'Cambria Math Bold' .ttf and specify it for jEuclid...

Intelligent2013 commented 1 year ago

After today's investigation, as I understood

jEuclid uses ImageGraphics2D object (org.apache.xmlgraphics.image.loader.impl.ImageGraphics2D) for drawing formulas, and then converts it into the curves, and put into PDF, I don't sure that SVG is using
jEuclid uses own 'simulate style' algorithm than Apache FOP:

left - jEuclid curves, right - Apache FOP text

Cambria Math
- STIX Two Math:

Just for experiment, I've removed simulate-style="true" for STIX Two Math font, and jEuclid still renders in the bold:

The reason why math isn't bolded on MacOs in the investigation still.

Intelligent2013 commented 1 year ago

For further investigation, example XML:

    <clause id="id1" displayorder="1">
      <title depth="1">1<tab/>Math bold test</title>
      <p>Cambria math: <font font-family="Cambria Math">[a,b], <strong>[a,b]</strong>
        </font>
        <stem type="MathML" font-family="Cambria Math">
          <math xmlns="http://www.w3.org/1998/Math/MathML">
            <mstyle mathvariant="bold">
              <mfenced open="[" close="]">
                <mrow>
                  <mi>a</mi>
                  <mo>,</mo>
                  <mi>b</mi>
                </mrow>
              </mfenced>
            </mstyle>
          </math>
          <!-- [a , b] -->
          <asciimath>[a,b]</asciimath>
        </stem>
      </p>
      <p>STIX Two Math: <font font-family="STIX Two Math">[a,b], <strong>[a,b]</strong>
        </font>
        <stem type="MathML" font-family="STIX Two Math">
          <math xmlns="http://www.w3.org/1998/Math/MathML">
            <mstyle mathvariant="bold">
              <mfenced open="[" close="]">
                <mrow>
                  <mi>a</mi>
                  <mo>,</mo>
                  <mi>b</mi>
                </mrow>
              </mfenced>
            </mstyle>
          </math>
          <!-- [a , b] -->
          <asciimath>[a,b]</asciimath>
        </stem>
      </p>
    </clause>

XSLT updated for new tags (for testing only):

 <xsl:template match="*[local-name() = 'font']">
    <fo:inline font-family="{@font-family}"><xsl:apply-templates /></fo:inline>
 </xsl:template>

<xsl:template match="mathml:math">
...
<fo:inline xsl:use-attribute-sets="mathml-style">
      <xsl:copy-of select="ancestor::*[local-name() = 'stem']/@font-family"/>
...

Fontist manifest fragment (Cambria fonts from Mac machine specially for testing):

---
Cambria:
  Regular:
    full_name: Cambria
    paths:
    -"D:/Work/Metanorma/Mac_fonts/Cambria.ttf"
  Bold:
    full_name: Cambria Bold
    paths:
    - "D:/Work/Metanorma/Mac_fonts/Cambria Bold.ttf"
...
Cambria Math:
  Regular:
    full_name: Cambria Math
    paths:
    - "D:/Work/Metanorma/Mac_fonts/Cambria Math.ttf"
STIX Two Math:
  Regular:
    full_name: STIX Two Math Regular
    paths:
    - C:/Users/TestUser/.fontist/fonts/STIXTwoMath-Regular.otf

Currently rendering on Win10:

Note: first [a,b] is normal text, 2nd is bold text, 3rd is jEuclid graphics.

I'll try on Mac tomorrow, something wrong with VirtualBox.

Intelligent2013 commented 1 year ago

Rendering on Mac:

jEuclid renders bold in STIX Two Math, but doesn't in Cambria Math. Very uncertain result...

Further actions:

[x] remove space in the filename Cambria Math.ttf on the disk and in manifest (may be jEuclid doesn't understand spaces in the filename?)
[x] try cambria.ttc (contains both Cambria and Cambria Math) from Windows.

Intelligent2013 commented 1 year ago

[x] remove space in the filename Cambria Math.ttf on the disk and in manifest (may be jEuclid doesn't understand spaces in the filename?)

[x] try cambria.ttc (contains both Cambria and Cambria Math) from Windows.

The same result - no bold.

Intelligent2013 commented 1 year ago

JEuclid MathViewer with fontSerif attribute:

on Windows :

with mathvariant="bold"
without mathvariant="bold"

Note: bold looks different than as it renders in PDF.

on Mac

with and without mathvariant="bold" looks identically

Intelligent2013 commented 1 year ago

This code from Mo.java has affect on the bold style for brackets:

 info.setGraphicsObject(new TextObject(t, this.getLspaceAsFloat(now)
                + textLayoutInfo.getOffset() * calcScaleX, calcBaselineShift,
                AffineTransform.getScaleInstance(calcScaleX, calcScaleY), //calcScaleX, calcScaleY
                (Color) now.getParameter(Parameter.MATHCOLOR)));

For calcScaleX=1.0f and calcScaleY=1.0f, PDF renders (on Windows): For calcScaleX=1.5f and calcScaleY=1.0f, PDF renders (on Windows):

For both case in Mac:

The objects textLayout is identical on both platforms:

textLayout=sun.font.StandardTextSource@671c4166[start:0, len:1, cstart:0, clen:1, chars:"5b", level:0, flags:0, font:java.awt.Font[family=Cambria Math,name=Cambria Math,style=bold,size=11], frc:java.awt.font.FontRenderContext@78f755b2, cm:sun.font.CoreMetrics@41183d00]

textLayout=sun.font.StandardTextSource@228cea97[start:0, len:1, cstart:0, clen:1, chars:"5b", level:0, flags:0, font:java.awt.Font[family=Cambria Math,name=Cambria Math,style=bold,size=11], frc:java.awt.font.FontRenderContext@6e20c58e, cm:sun.font.CoreMetrics@41273c00]

Intelligent2013 commented 1 year ago

It's the Java issue. I've developed a small program to check the math fonts rendering in Java GUI (https://github.com/Intelligent2013/mathfontstest).

1st and 3rd lines in normal
2nd and 4th lines in bold

        JLabel label1 = new JLabel("Cambria Math: [a, b]");
        label1.setBounds(50, 50, 400, 30);
        label1.setFont(new Font("Cambria Math", Font.PLAIN, 30));

        JLabel label2 = new JLabel("Cambria Math: [a, b]");
        label2.setBounds(50, 100, 400, 30);
        label2.setFont(new Font("Cambria Math", Font.BOLD, 30));

        JLabel label3 = new JLabel("STIX Two Math: [a, b]");
        label3.setBounds(50, 150, 400, 30);
        label3.setFont(new Font("STIX Two Math", Font.PLAIN, 30));

        JLabel label4 = new JLabel("STIX Two Math: [a, b]");
        label4.setBounds(50, 200, 400, 30);
        label4.setFont(new Font("STIX Two Math", Font.BOLD, 30));

On Windows:

On Mac - no differences between 1st and 2nd lines:

And I don't figure out why Cambria Math looks thickness on Mac than on Windows.

Intelligent2013 commented 1 year ago

On Ubuntu:

Intelligent2013 commented 1 year ago

There are the similar issue on the openjdk bug tracker:

https://bugs.openjdk.org/browse/JDK-8176835
https://bugs.openjdk.org/browse/JDK-8222624
https://bugs.openjdk.org/browse/JDK-8185569 First issue raised in 2017 (6 years ago) and still opened.

Intelligent2013 commented 1 year ago

Previously, I've installed the font Cambria Math.ttf into the folder /Library/Fonts on my VB MacOs (just to repeat the environment from the original issue MacOS machine). But now I've moved it into ~/.fontist/fonts/... and now it renders ok in the test application MathFontsTest:

Note: now, 'normal' renders as normal font, not 'bold' as in https://github.com/metanorma/metanorma-iso/issues/910#issuecomment-1519686929

I'll test the PDF rendering and will think about one more workaround solution, specially for MacOs.

Intelligent2013 commented 1 year ago

Case for ~/.fontist/fonts/Cambria Math.ttf - PDF renders OK:

So, workaround solution for mn2pdf:

if OS is MacOS
if font file path doesn't contain .fontist (i.e. isn't installed by fontist), then
- copy font file into tmp folder (one folder for all fonts)
- change font path to the tmp folder font file
- generate PDF...
- flush font tmp folder

Intelligent2013 commented 1 year ago

Current FOP config:

<fonts>
    <font embed-url="file:/var/folders/vn/4jyz_ysn61lbh37__jrqsx1c0000gn/T/3e1e15c6-5e2b-40e8-93eb-48996f722bfc/Cambria.ttf" kerning="yes" mn_default="false">
        <alternate embed-url="cambria.ttf"/>
        <font-triplet name="Cambria" style="normal" weight="normal"/>
    </font>
    <font embed-url="file:/var/folders/vn/4jyz_ysn61lbh37__jrqsx1c0000gn/T/3e1e15c6-5e2b-40e8-93eb-48996f722bfc/Cambria%20Math.ttf" kerning="yes" mn_default="false">
        <alternate embed-url="Cambria Math.ttf"/>
        <alternate embed-url="cambria.ttf"/>
        <font-triplet name="Cambria Math" style="normal" weight="normal"/>
    </font>
    <font embed-url="file:/Library/Fonts/Cambria Math.ttf" kerning="yes" mn_default="false" simulate-style="true">
        <alternate embed-url="Cambria Math.ttf" simulate-style="true"/>
        <alternate embed-url="cambria.ttf" simulate-style="true"/>
        <font-triplet name="Cambria Math" style="normal" weight="bold"/>
    </font>
    <font embed-url="file:/var/folders/vn/4jyz_ysn61lbh37__jrqsx1c0000gn/T/3e1e15c6-5e2b-40e8-93eb-48996f722bfc/Cambria%20Bold.ttf" kerning="yes" mn_default="false">
        <alternate embed-url="Cambria Bold.ttf"/>
        <font-triplet name="Cambria" style="normal" weight="bold"/>
    </font>

There is file:/Library/Fonts/Cambria Math.ttf still. To do: for non-fontist manifest entries apply the same actions.

ronaldtse commented 1 year ago

@Intelligent2013 is there something that Fontist can do here? I don't like the idea of mn2pdf moving fonts around.

Intelligent2013 commented 1 year ago

For FOP font config:

<fonts>
    <font embed-url="file:/var/folders/vn/4jyz_ysn61lbh37__jrqsx1c0000gn/T/8a88be7d-05a5-4ed7-9b34-f9184598daef/Cambria.ttf" kerning="yes" mn_default="false">
        <alternate embed-url="cambria.ttf"/>
        <font-triplet name="Cambria" style="normal" weight="normal"/>
    </font>
    <font embed-url="file:/var/folders/vn/4jyz_ysn61lbh37__jrqsx1c0000gn/T/8a88be7d-05a5-4ed7-9b34-f9184598daef/Cambria%20Math.ttf" kerning="yes" mn_default="false">
        <alternate embed-url="Cambria Math.ttf"/>
        <alternate embed-url="cambria.ttf"/>
        <font-triplet name="Cambria Math" style="normal" weight="normal"/>
    </font>
    <font embed-url="file:/var/folders/vn/4jyz_ysn61lbh37__jrqsx1c0000gn/T/8a88be7d-05a5-4ed7-9b34-f9184598daef/Cambria%20Math.ttf" kerning="yes" mn_default="false" simulate-style="true">
        <alternate embed-url="Cambria Math.ttf" simulate-style="true"/>
        <alternate embed-url="cambria.ttf" simulate-style="true"/>
        <font-triplet name="Cambria Math" style="normal" weight="bold"/>
    </font>
    <font embed-url="file:/var/folders/vn/4jyz_ysn61lbh37__jrqsx1c0000gn/T/8a88be7d-05a5-4ed7-9b34-f9184598daef/Cambria%20Bold.ttf" kerning="yes" mn_default="false">
        <alternate embed-url="Cambria Bold.ttf"/>
        <font-triplet name="Cambria" style="normal" weight="bold"/>
    </font>

PDF again wrong:

Looks like JEuclid uses the system information about registered fonts, and therefore uses the font from /Library/Fonts/. I'll try to unregister font for Java...

Intelligent2013 commented 1 year ago

I can't find how to unregister font for JVM...

One more experiment without font removing: as soon I disable the font 'Cambria Math' in FontBox manually, PDF renders ok:

I need timeout...

Intelligent2013 commented 1 year ago

@Intelligent2013 is there something that Fontist can do here? I don't like the idea of mn2pdf moving fonts around.

@ronaldtse we need to isolate system fonts from Java somehow, but I don't know how yet.

Intelligent2013 commented 1 year ago

I've asked about this issue here https://stackoverflow.com/questions/76185073/java-how-to-render-system-font-in-bold-on-macos.

Intelligent2013 commented 1 year ago

I have one idea:

copy system font to the temp folder
register the font from the temp folder with a new name, for instance, Cambria Math Metanorma
replace Cambria Math to Cambria Math Metanorma in the XSL-FO

Java doesn't allow to change font name directly. Currently, I've found the method how to change the font name in the Java Font object (via reflection feature):

But GraphicsEnvironment ge.registerFont(ttfFont) doesn't register this updated font, because familyName and fullName still have value Cambria Math (without Metanorma suffix):

Locale l = getSystemStartupLocale();
String familyName = font.getFamily(l).toLowerCase();
String fullName = font.getFontName(l).toLowerCase();

The methods getFontName and getFamilyName in the class Font2D just return fullName and familyName:

public String getFontName(Locale l) {
        return fullName;
}

public String getFamilyName(Locale l) {
        return familyName;
 }

Intelligent2013 commented 1 year ago

Actually, the right class is TrueTypeFont (Font2D is abstract class):

    @Override
    public String getFontName(Locale locale) {
        if (locale == null) {
            return fullName;
        } else if (locale.equals(nameLocale) && localeFullName != null) {
            return localeFullName;
        } else {
            short localeID = getLCIDFromLocale(locale);
            String name = lookupName(localeID, FULL_NAME_ID);
            if (name == null) {
                return fullName;
            } else {
                return name;
            }
        }
    }

The font's name depends on locale and stores very deep into the byte array:

public static final int nameTag = 0x6E616D65; // 'name'
...
   /* Return the requested name in the requested locale, for the
     * MS platform ID. If the requested locale isn't found, return US
     * English, if that isn't found, return null and let the caller
     * figure out how to handle that.
     */
    protected String lookupName(short findLocaleID, int findNameID) {
        String foundName = null;
        byte[] name = new byte[1024];

        ByteBuffer buffer = getTableBuffer(nameTag);
        if (buffer != null) {
            ShortBuffer sbuffer = buffer.asShortBuffer();
            sbuffer.get(); // format - not needed.
            short numRecords = sbuffer.get();

            /* The name table uses unsigned shorts. Many of these
             * are known small values that fit in a short.
             * The values that are sizes or offsets into the table could be
             * greater than 32767, so read and store those as ints
             */
            int stringPtr = ((int) sbuffer.get()) & 0xffff;

            for (int i=0; i<numRecords; i++) {
                short platformID = sbuffer.get();
                if (platformID != MS_PLATFORM_ID) {
                    sbuffer.position(sbuffer.position()+5);
                    continue; // skip over this record.
                }
                short encodingID = sbuffer.get();
                short langID     = sbuffer.get();
                short nameID     = sbuffer.get();
                int   nameLen    = ((int) sbuffer.get()) & 0xffff;
                int   namePtr    = (((int) sbuffer.get()) & 0xffff) + stringPtr;
                if (nameID == findNameID &&
                    ((foundName == null && langID == ENGLISH_LOCALE_ID)
                     || langID == findLocaleID)) {
                    buffer.position(namePtr);
                    buffer.get(name, 0, nameLen);
                    foundName = makeString(name, nameLen, encodingID);
                    if (langID == findLocaleID) {
                        return foundName;
                    }
                }
            }
        }
        return foundName;
    }

So, it's very hard to change it. I assume that would be easier to change the locale before font registering to en_US and restore it to the previous value after. In this case the values from localeFamilyName and localeFontName will be used.

Intelligent2013 commented 1 year ago

I assume that would be easier to change the locale before font registering to en_US and restore it to the previous value after. In this case the values from localeFamilyName and localeFontName will be used.

No, this approach isn't working, because 'locale' determines via the system properties:

from TrueTypeFont:

nameLocale = sun.awt.SunToolkit.getStartupLocale();

...

} else if (locale.equals(nameLocale) && localeFullName != null) { return localeFullName;


where `locale` determines in `SunFontManager`:
```java
                String fileEncoding = System.getProperty("file.encoding", "");
                String sysEncoding = System.getProperty("sun.jnu.encoding");
                if (sysEncoding != null && !sysEncoding.equals(fileEncoding)) {
                    return Locale.ROOT;
                }

                String language = System.getProperty("user.language", "en");
                String country  = System.getProperty("user.country","");
                String variant  = System.getProperty("user.variant","");
                return new Locale(language, country, variant);

and nameLocale determines in SunToolkit:

public static Locale getStartupLocale() {
    if (startupLocale == null) {
        String language, region, country, variant;
        language = AccessController.doPrivileged(
                        new GetPropertyAction("user.language", "en"));
        // for compatibility, check for old user.region property
        region = AccessController.doPrivileged(
                        new GetPropertyAction("user.region"));
        if (region != null) {
            // region can be of form country, country_variant, or _variant
            int i = region.indexOf('_');
            if (i >= 0) {
                country = region.substring(0, i);
                variant = region.substring(i + 1);
            } else {
                country = region;
                variant = "";
            }
        } else {
            country = AccessController.doPrivileged(
                            new GetPropertyAction("user.country", ""));
            variant = AccessController.doPrivileged(
                            new GetPropertyAction("user.variant", ""));
        }
        startupLocale = new Locale(language, country, variant);
    }
    return startupLocale;
}

Need find the another way for the workaround solution.

metanorma / metanorma-iso

PDF: Bold MathML not rendering #910

on Windows :

on Mac