Unable to render SVGs if they contained text that requires fonts

ronaldtse commented 1 year ago

In mn-samples-bsi, the PD 19650-0 document uses SVG images that has text that uses other fonts. We need to find a way to make that work.

Method 1: (for now) just use the default fonts to render them
Method 2: (for future) install those fonts via Fontist

Please test with this PR:

https://github.com/metanorma/mn-samples-bsi/pull/228

Intelligent2013 commented 1 year ago

Method 1: (for now) just use the default fonts to render them

I've tried to generate the PDF by

bundle exec metanorma -t bsi -x pdf document.adoc

the process ends with error about not found font file:

Font "FrutigerLTStd-Bold,normal,400" not found. Substituting with "any,normal,400".
Font "Frutiger LT Std,normal,400" not found. Substituting with "any,normal,400".
Font "FrutigerLTStd-Roman,normal,400" not found. Substituting with "any,normal,400".
Font "Arial-BoldMT,normal,400" not found. Substituting with "any,normal,400".
Font "ArialMT,normal,400" not found. Substituting with "any,normal,400".
SVG graphic could not be rendered. Reason: java.lang.RuntimeException: Failed to read font file file:/D:/Work/Metanorma/repositories/mn-samples-bsi/sources/pd-19650-0/arialbd.ttf D:\Work\Metanorma\repositories\mn-samples-bsi\sources\pd-19650-0\arialbd.ttf (The system cannot find the file specified)
java.lang.RuntimeException: Failed to read font file file:/D:/Work/Metanorma/repositories/mn-samples-bsi/sources/pd-19650-0/arialbd.ttf D:\Work\Metanorma\repositories\mn-samples-bsi\sources\pd-19650-0\arialbd.ttf (The system cannot find the file specified)
    at org.apache.fop.fonts.LazyFont.load(LazyFont.java:134)

The entry from Apache FOP config points to the arialbd.ttf in the current folder:

<font embed-url="file:/D:/Work/Metanorma/repositories/mn-samples-bsi/sources/pd-19650-0/arialbd.ttf" kerning="yes" mn_default="false">
    <alternate embed-url="Arial Bold.ttf"/>
    <font-triplet name="Arial" style="normal" weight="bold"/>
</font>

To do:

[x] such wrong entries should be removed (they occur due the source entries in the src/resource/pdf_fonts_config.xml)
[x] determine main font - it's the 1st item in the fo:root/@font-family

[ ] mn2pdf should parse the input XML for font-family entries and add fonts into Apache FOP config with default font file, for example:

<font embed-url="file:/C:/Users/TestUser/.fontist/fonts/NotoSans-Bold.ttf" embedding-mode="full" kerning="yes" mn_default="false">
<font-triplet name="Arial" style="normal" weight="bold"/>
</font>

Note: there isn't issue for the font Frutiger LT Std, because there isn't entry in the src/resource/pdf_fonts_config.xml for it.

.pdf.err log contains the warnings about missing fonts:

Font "FrutigerLTStd-Bold,normal,400" not found. Substituting with "any,normal,400".
Font "Frutiger LT Std,normal,400" not found. Substituting with "any,normal,400".
Font "FrutigerLTStd-Roman,normal,400" not found. Substituting with "any,normal,400".

The Apache FOP uses the font Times for non-found fonts:

Method 2: (for future) install those fonts via Fontist

I think that metanorma should parse the attributes font-family and pass the values into the fontist installation process (similar to the document attribute :fonts:).

For the experiment, I've added the fonts from SVG into processor.rb (marked >):

          "Tahoma" => nil,
          "STIX Two Math" => nil,
          "Source Han Sans" => nil,
          "Source Han Sans Normal" => nil,
          "Courier New" => nil,
          "BSI Gesta" => nil,
          "Frutiger" => nil,
          "Cambria" => nil,
          "Cambria Math" => nil,
          "Times New Roman" => nil,
          "Open Sans" => nil,
>          "FrutigerLTStd-Bold" => nil,
>          "Frutiger LT Std" => nil,
>          "FrutigerLTStd-Roman" => nil,
>          "Arial-BoldMT" => nil,
>          "ArialMT" => nil,

The process stops with the error:

[fontist] 'FrutigerLTStd-Bold' font is not supported. Please report this issue at github.com/metanorma/metanorma/issues to report this issue.

Then, remove the entry "FrutigerLTStd-Bold" => nil, and repeat the process:

[fontist] 'Frutiger LT Std' font is not supported. Please report this issue at github.com/metanorma/metanorma/issues to report this issue.

Same for FrutigerLTStd-Roman, Arial-BoldMT and ArialMT.

The result for manual fontist font installation:

fontist repo setup metanorma https://github.com/metanorma/fontist-formulas-private
fontist repo update metanorma
fontist install FrutigerLTStd-Bold

the result is:

Font "FrutigerLTStd-Bold" not found locally.                                                                                                                                                                       Font 'FrutigerLTStd-Bold' not found locally nor available in the Fontist formula repository.                                                                                                                       Perhaps it is available at the latest Fontist formula repository.                                                                                                                                                  You can update the formula repository using the command `fontist update` and try again.

fontist installs the font by name. The fragment from https://github.com/metanorma/fontist-formulas-private/blob/3428f7cf60c9de52475ba5fd2dc3cd1fb09f2854/frutiger_lt_std.yml#L45

fonts:
- name: Frutiger LT Std 45 Light
  styles:
  - family_name: Frutiger LT Std 45 Light
    type: Bold
    preferred_family_name: Frutiger LT Std

You can install the font by 'preferred family':

fontist install "Frutiger LT Std" --preferred-family

But, we don't know what is kind of name is using in font-family. The fragment from fig-2-a.svg:

    <text transform="translate(194.16 108.76)" style="fill:#231f20; font-family:FrutigerLTStd-Bold, &apos;Frutiger LT Std&apos;; font-size:8.77px; font-weight:700;">
        <tspan x="0" y="0" style="letter-spacing:.01em;">3</tspan>
        <tspan x="4.98" y="0">D</tspan>
    </text>

FrutigerLTStd-Bold is full_name and post_script_name, Frutiger LT Std is preferred_family_name:

- name: Frutiger LT Std 45 Light
  styles:
  - family_name: Frutiger LT Std 45 Light
    type: Bold
    preferred_family_name: Frutiger LT Std
    preferred_type: 65 Bold
    full_name: FrutigerLTStd-Bold
    post_script_name: FrutigerLTStd-Bold

FrutigerLTStd-Roman is post_script_name:

  - family_name: Frutiger LT Std 55 Roman
    type: Regular
    preferred_family_name: Frutiger LT Std
    preferred_type: 55 Roman
    full_name: FrutigerLTStd-Roman
    post_script_name: FrutigerLTStd-Roman
    version: 2.035;PS 002.000;hotconv 1.0.51;makeotf.lib2.0.18671
    copyright: "© 1988, 1990, 1994, 2002 Adobe Systems Incorporated. All rights reserved."
    font: FrutigerLTStd-Roman.otf

For ArialMT, also post_script_name (https://github.com/fontist/fontist/blob/2e1f15508932f9ef0f5c52c12066a0ff13699e49/spec/examples/import/webcore.yml#L119):

- family_name: Arial
    type: Regular
    full_name: Arial
    post_script_name: ArialMT

My proposal is - fontist should install the font in this sequence:

1st - find the font in family_name and install it (currently)
2nd - if font isn't found by family_name, then find it by preferred_family_name (currently we have to set the parameter --preferred-family obviously)
3rd - if font isn't found by preferred_family_name, then find it by full_name
4th - find by post_script_name

Intelligent2013 commented 1 year ago

Errors occur after update for Method 1:

WARNING: Font 'Courier New' (font name 'Courier New', font style 'normal', font weight 'normal') doesn't exist. Replaced by 'C:\Users\runneradmin\.metanorma\fonts\NotoSansMono-Regular.ttf'.
WARNING: Font 'Courier New' (font name 'Courier New', font style 'normal', font weight 'bold') doesn't exist. Replaced by 'C:\Users\runneradmin\.metanorma\fonts\NotoSansMono-Bold.ttf'.
WARNING: Font 'Courier New' (font name 'Courier New', font style 'italic', font weight 'normal') doesn't exist. Replaced by 'C:\Users\runneradmin\.metanorma\fonts\NotoSansMono-Regular.ttf'.
WARNING: Font 'Courier New' (font name 'Courier New', font style 'italic', font weight 'bold') doesn't exist. Replaced by 'C:\Users\runneradmin\.metanorma\fonts\NotoSansMono-Bold.ttf'.
WARNING: Font 'Noto Sans Mono' (font name 'Noto Sans Mono', font style 'normal', font weight 'normal') doesn't exist. Replaced by 'C:\Users\runneradmin\.metanorma\fonts\NotoSans-Regular.ttf'.
WARNING: Font 'Noto Sans Mono' (font name 'Noto Sans Mono', font style 'normal', font weight 'bold') doesn't exist. Replaced by 'C:\Users\runneradmin\.metanorma\fonts\NotoSans-Bold.ttf'.
WARNING: Font 'Noto Sans Mono' (font name 'Noto Sans Mono', font style 'italic', font weight 'normal') doesn't exist. Replaced by 'C:\Users\runneradmin\.metanorma\fonts\NotoSans-Italic.ttf'.
WARNING: Font 'Noto Sans Mono' (font name 'Noto Sans Mono', font style 'italic', font weight 'bold') doesn't exist. Replaced by 'C:\Users\runneradmin\.metanorma\fonts\NotoSans-BoldItalic.ttf'.
WARNING: Font 'Noto Sans Mono CJK HK' (font name 'Noto Sans Mono CJK HK', font style 'normal', font weight 'normal') doesn't exist. Replaced by 'C:\Users\runneradmin\.metanorma\fonts\NotoSans-Regular.ttf'.
WARNING: Font 'Noto Sans Mono CJK HK' (font name 'Noto Sans Mono CJK HK', font style 'normal', font weight 'bold') doesn't exist. Replaced by 'C:\Users\runneradmin\.metanorma\fonts\NotoSans-Bold.ttf'.
WARNING: Font 'Noto Sans Mono CJK HK' (font name 'Noto Sans Mono CJK HK', font style 'italic', font weight 'normal') doesn't exist. Replaced by 'C:\Users\runneradmin\.metanorma\fonts\NotoSans-Italic.ttf'.
WARNING: Font 'Noto Sans Mono CJK HK' (font name 'Noto Sans Mono CJK HK', font style 'italic', font weight 'bold') doesn't exist. Replaced by 'C:\Users\runneradmin\.metanorma\fonts\NotoSans-BoldItalic.ttf'.
WARNING: Font 'Noto Sans Mono CJK JP' (font name 'Noto Sans Mono CJK JP', font style 'normal', font weight 'normal') doesn't exist. Replaced by 'C:\Users\runneradmin\.metanorma\fonts\NotoSans-Regular.ttf'.
WARNING: Font 'Noto Sans Mono CJK JP' (font name 'Noto Sans Mono CJK JP', font style 'normal', font weight 'bold') doesn't exist. Replaced by 'C:\Users\runneradmin\.metanorma\fonts\NotoSans-Bold.ttf'.
WARNING: Font 'Noto Sans Mono CJK JP' (font name 'Noto Sans Mono CJK JP', font style 'italic', font weight 'normal') doesn't exist. Replaced by 'C:\Users\runneradmin\.metanorma\fonts\NotoSans-Italic.ttf'.
WARNING: Font 'Noto Sans Mono CJK JP' (font name 'Noto Sans Mono CJK JP', font style 'italic', font weight 'bold') doesn't exist. Replaced by 'C:\Users\runneradmin\.metanorma\fonts\NotoSans-BoldItalic.ttf'.
WARNING: Font 'Noto Sans Mono CJK KR' (font name 'Noto Sans Mono CJK KR', font style 'normal', font weight 'normal') doesn't exist. Replaced by 'C:\Users\runneradmin\.metanorma\fonts\NotoSans-Regular.ttf'.
WARNING: Font 'Noto Sans Mono CJK KR' (font name 'Noto Sans Mono CJK KR', font style 'normal', font weight 'bold') doesn't exist. Replaced by 'C:\Users\runneradmin\.metanorma\fonts\NotoSans-Bold.ttf'.
WARNING: Font 'Noto Sans Mono CJK KR' (font name 'Noto Sans Mono CJK KR', font style 'italic', font weight 'normal') doesn't exist. Replaced by 'C:\Users\runneradmin\.metanorma\fonts\NotoSans-Italic.ttf'.
WARNING: Font 'Noto Sans Mono CJK KR' (font name 'Noto Sans Mono CJK KR', font style 'italic', font weight 'bold') doesn't exist. Replaced by 'C:\Users\runneradmin\.metanorma\fonts\NotoSans-BoldItalic.ttf'.
WARNING: Font 'Noto Sans Mono CJK SC' (font name 'Noto Sans Mono CJK SC', font style 'normal', font weight 'normal') doesn't exist. Replaced by 'C:\Users\runneradmin\.metanorma\fonts\NotoSans-Regular.ttf'.
WARNING: Font 'Noto Sans Mono CJK SC' (font name 'Noto Sans Mono CJK SC', font style 'normal', font weight 'bold') doesn't exist. Replaced by 'C:\Users\runneradmin\.metanorma\fonts\NotoSans-Bold.ttf'.
WARNING: Font 'Noto Sans Mono CJK SC' (font name 'Noto Sans Mono CJK SC', font style 'italic', font weight 'normal') doesn't exist. Replaced by 'C:\Users\runneradmin\.metanorma\fonts\NotoSans-Italic.ttf'.
WARNING: Font 'Noto Sans Mono CJK SC' (font name 'Noto Sans Mono CJK SC', font style 'italic', font weight 'bold') doesn't exist. Replaced by 'C:\Users\runneradmin\.metanorma\fonts\NotoSans-BoldItalic.ttf'.
WARNING: Font 'Noto Sans Mono CJK TC' (font name 'Noto Sans Mono CJK TC', font style 'normal', font weight 'normal') doesn't exist. Replaced by 'C:\Users\runneradmin\.metanorma\fonts\NotoSans-Regular.ttf'.
WARNING: Font 'Noto Sans Mono CJK TC' (font name 'Noto Sans Mono CJK TC', font style 'normal', font weight 'bold') doesn't exist. Replaced by 'C:\Users\runneradmin\.metanorma\fonts\NotoSans-Bold.ttf'.
WARNING: Font 'Noto Sans Mono CJK TC' (font name 'Noto Sans Mono CJK TC', font style 'italic', font weight 'normal') doesn't exist. Replaced by 'C:\Users\runneradmin\.metanorma\fonts\NotoSans-Italic.ttf'.
WARNING: Font 'Noto Sans Mono CJK TC' (font name 'Noto Sans Mono CJK TC', font style 'italic', font weight 'bold') doesn't exist. Replaced by 'C:\Users\runneradmin\.metanorma\fonts\NotoSans-BoldItalic.ttf'.

Intelligent2013 commented 1 year ago

These errors occur on GH mn-native-pdf repository only. No errors on local machine.

Intelligent2013 commented 1 year ago

mn-native-pdf reads the XML generated 6 months ago in https://github.com/metanorma/mn-samples-iso/tree/gh-pages. These XML have a difference in sourcecode format:

                <sourcecode id="_1780a541-a39f-897a-533a-f935721eca31"
                            lang="ruby">
                    <span class="nb">puts</span>
...
                    <annotation id="_f4eaad01-1f05-7b19-91a0-ae5a8d34312f">
                        <p id="_910b9068-d8a3-ce03-b853-029f70dd6630">This is an annotation</p>
                    </annotation>
                </sourcecode>

than just generated XML:

                <sourcecode id="_1780a541-a39f-897a-533a-f935721eca31"
                            lang="ruby">
                    <span class="nb">puts</span>
...
                    <dl>
                        <name>Key</name>
                        <dt id="_f4eaad01-1f05-7b19-91a0-ae5a8d34312f">
                            <span class="c">1</span>
                        </dt>
                        <dd>
                            <p id="_910b9068-d8a3-ce03-b853-029f70dd6630">This is an annotation</p>
                        </dd>
                    </dl>
                </sourcecode>

mn2pdf on 1st step generates XSL-FO with tables only and then removes unused fonts. Because old xml doesn't have a source code in the table, then Courier New fonts removes from FOP config. Fixing.

Intelligent2013 commented 1 year ago

Method 1: (for now) just use the default fonts to render them

Method 1 realized in the mn2pdf v1.79.

metanorma / mn2pdf

Unable to render SVGs if they contained text that requires fonts #177

Method 1: (for now) just use the default fonts to render them

Method 2: (for future) install those fonts via Fontist