PDF: Add new vertical layout

[Intelligent2013]

In continue of https://github.com/metanorma/mn-native-pdf/issues/713.

As titled, based on the new JIS layout JIS_Template_traditional_blue_v2_traditional.pdf.

To do (draft list):

• [x] ~add new parameters :presentation-metadata-color-cover-page-background-1:, :presentation-metadata-color-cover-page-background-2: for vertical stripes (with opacity) on the cover page~ no need, it's part of user-provided cover page
• [x] ~add new parameters :presentation-metadata-color-cover-page-background-opacity-1:, :presentation-metadata-color-cover-page-background-opacity-2: vertical stripes opacities~ no need, it's part of user-provided cover page
• [x] ~integrate two new logos JIS and JSA into SVG.~ no need, it's part of user-provided cover page
• [x] add new parameter :presentation-metadata-color-header-footer-background: (blue bar in the reference PDF)
• [ ] use font Noto Sans JP Thin, Noto Serif JP ExtraLight and ~Source Sans~ Noto Serif JP Medium in the XSLT, and add into processor.rb metanorma-jis (https://github.com/metanorma/metanorma-jis/pull/230)
• [ ] use Japanese numerals in
  • [ ] page numbers (header/footer)
  • [x] clause numbers (https://github.com/metanorma/metanorma-jis/issues/228)
  • [x] ordered list items (https://github.com/metanorma/metanorma-jis/issues/228)
  • [x] dates (https://github.com/metanorma/metanorma-jis/issues/228)
• [x] add parameter :coverpage-image: (see https://www.metanorma.org/author/bsi/ref/document-attributes/) - image location, for image to be included on the PDF cover page. Image location is relative to the source document.
  • [x] into XSLT
  • [x] into metanorma-jis (https://github.com/metanorma/metanorma-jis/issues/227)
• [x] add parameter :backpage-image: - same, for image to be included on the PDF back page. (https://github.com/metanorma/metanorma-jis/issues/227)
  • [x] into XSLT
  • [x] into metanorma-jis (https://github.com/metanorma/metanorma-jis/issues/227)
• [x] put cover page image into the back cover page, if back cover page doesn't set
• [ ] add last section (see page 4 in the reference PDF)
• [x] don't rotate letters in web links (http;// https:// www.), no need special actions, the English text doesn't rotate by default
• [ ] some English text should not be rotated, markup with something like this span:dontrotate[text]

Known issues:

• these element didn't rotate in the vertical layout (currently I don't know how to fix it and how long it will take):
  • [ ] page numbers in ToC (https://github.com/metanorma/xmlgraphics-fop/issues/10)
  • [ ] page numbers in the footer (left side) (https://github.com/metanorma/xmlgraphics-fop/issues/10)
  • [ ] table (https://github.com/metanorma/xmlgraphics-fop/issues/8)
  • [ ] block-container (https://github.com/metanorma/xmlgraphics-fop/issues/8)

[ronaldtse]

Here's an updated file for reference:

• JIS_Template_traditional_blue_v2_traditional.pdf

use font NotoSansJP-Thin, NotoSerifJP-ExtraLight and Source Sans in the XSLT

I've used "Noto Serif JP" "Medium" to replace the Italic text, so we don't need Source Sans.

put cover page image into the back cover page

It's a mirrored front-back image of the front cover page. If possible allow using a separate front/back cover page.

Thanks!

[Intelligent2013]

add new parameters :presentation-metadata-color-cover-page-background-1:, :presentation-metadata-color-cover-page-background-2: for vertical stripes (with opacity) on the cover page

The parameter :presentation-metadata-color-cover-page-background-1: for

The parameter :presentation-metadata-color-cover-page-background-2: for

Also we need two parameters for opacities:

• :presentation-metadata-color-cover-page-background-opacity-1: 0.58
• :presentation-metadata-color-cover-page-background-opacity-2: 0.75

[Intelligent2013]

The footer contains the last on page clause title. First depth clause:

[Intelligent2013]

@ronaldtse there are question about numbers in header/footer:

• the number in header means :edition:?
• the number in footer means the page number? Just mistakenly all page number are 2?
• both numbers in header/footer ToC page should be in parenthesis?

Examples:

• ToC page

  • header - number 1 in parenthesis
  • footer - number 2 in parenthesis

• first document page

  • header - number 1
  • footer - number 2

• last document page - same

[Intelligent2013]

@ronaldtse could you check the fonts which are used on the cover page in the source Adobe Illustrator file. Adobe Acrobat shows Noto Serif Extra Light for all texts:

but the rendered fonts looks so:

I've tried to use Noto Serif JP Medium, but it looks different than in the source PDF(generated from AI):

Also, Adobe Acrobat in the document properties shows 7 fonts Noto Serif Extra Light:

Thanks.

[ronaldtse]

@Intelligent2013 I think this is because I’m using Noto Sans JP “Variable” that can realize all weights. It’s not Extra Light.

[Intelligent2013]

Actually the fonts Noto Serif JP Medium, Noto Serif JP Bold and Noto Serif Black are using on the cover page text.

[Intelligent2013]

Bad things found. Update in https://github.com/chunlinyao/fop has very limited vertical layout support! It doesn't support glyph rotation in the blocks inside fo:block-container and fo:table. From right to left: block, block-container, table. The glyphs didn't rotated in 2 and 3 blue boxes.

XSL-FO:

<fo:block border="1pt solid blue" language="JA">
                「天の人を生ずるは億兆皆同一轍にて、之に附与するに動かす可からざるの通義を以てす。即ち其通義とは人の自から生命を保し自由を求め幸福を祈るの類にて、他より之を如何ともす可らざるものなり。」
                ――『アメリカ独立宣言』
            </fo:block>
<fo:block-container>
    <fo:block border="1pt solid blue" language="JA">
                「天の人を生ずるは億兆皆同一轍にて、之に附与するに動かす可からざるの通義を以てす。即ち其通義とは人の自から生命を保し自由を求め幸福を祈るの類にて、他より之を如何ともす可らざるものなり。」
                ――『アメリカ独立宣言』
            </fo:block>
</fo:block-container>
<fo:table width="100%" table-layout="fixed">
    <fo:table-body>
        <fo:table-row>
            <fo:table-cell>
                <fo:block border="1pt solid blue" language="JA">
                                                「天の人を生ずるは億兆皆同一轍にて、之に附与するに動かす可からざるの通義を以てす。即ち其通義とは人の自から生命を保し自由を求め幸福を祈るの類にて、他より之を如何ともす可らざるものなり。」
                                                ――『アメリカ独立宣言』
                                        </fo:block>
            </fo:table-cell>
        </fo:table-row>
    </fo:table-body>
</fo:table>

[Intelligent2013]

Another issue in the vertical layout - the glyphs don't rotate in ToC (XSL-FO element fo:page-number-citation):

and in the footer (XSL-FO element fo:page-number):

[Intelligent2013]

@ronaldtse question about last page (not back page).

Is this a fixed text or it should be read from XML? It's user defined section or hard-coded boilerplate in metanorma gem?

[Intelligent2013]

The error occurs on the full document:

Rendered page fontist/fontist#1.
Rendered page fontist/fontist#2.
Index: 2, Size: 2
java.lang.IndexOutOfBoundsException: Index: 2, Size: 2
        at java.util.ArrayList.rangeCheck(ArrayList.java:659)
        at java.util.ArrayList.get(ArrayList.java:435)
        at org.apache.fop.render.intermediate.IFStructureTreeBuilder.replayEventsForPageSequence(IFStructureTreeBuilder.java:192)
        at org.apache.fop.render.intermediate.IFSerializer.startPageSequence(IFSerializer.java:278)
        at org.apache.fop.render.intermediate.IFRenderer.startPageSequence(IFRenderer.java:526)
        at org.apache.fop.area.RenderPagesModel.checkPreparedPages(RenderPagesModel.java:172)
        at org.apache.fop.area.RenderPagesModel.addPage(RenderPagesModel.java:146)
        at org.apache.fop.layoutmgr.AbstractPageSequenceLayoutManager.finishPage(AbstractPageSequenceLayoutManager.java:316)
        at org.apache.fop.layoutmgr.PageSequenceLayoutManager.finishPage(PageSequenceLayoutManager.java:243)
        at org.apache.fop.layoutmgr.PageSequenceLayoutManager.activateLayout(PageSequenceLayoutManager.java:147)
        at org.apache.fop.area.AreaTreeHandler.endPageSequence(AreaTreeHandler.java:267)
        at org.apache.fop.fo.DelegatingFOEventHandler.endPageSequence(DelegatingFOEventHandler.java:114)
        at org.apache.fop.accessibility.fo.FO2StructureTreeConverter.endPageSequence(FO2StructureTreeConverter.java:184)
        at org.apache.fop.fo.pagination.PageSequence.endOfNode(PageSequence.java:140)
        at org.apache.fop.fo.FOTreeBuilder$MainFOHandler.endElement(FOTreeBuilder.java:390)
        at org.apache.fop.fo.FOTreeBuilder$3.run(FOTreeBuilder.java:207)
        at org.apache.fop.fo.FOTreeBuilder$3.run(FOTreeBuilder.java:204)
        at java.security.AccessController.doPrivileged(Native Method)
        at org.apache.fop.fo.FOTreeBuilder.endElement(FOTreeBuilder.java:203)
        at org.apache.xalan.transformer.TransformerIdentityImpl.endElement(TransformerIdentityImpl.java:1149)
        at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.endElement(AbstractSAXParser.java:610)
        at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanEndElement(XMLDocumentFragmentScannerImpl.java:1784)
        at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl$FragmentContentDriver.next(XMLDocumentFragmentScannerImpl.java:2969)
        at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(XMLDocumentScannerImpl.java:605)
        at com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.next(XMLNSDocumentScannerImpl.java:113)
        at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:507)
        at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:867)
        at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:796)
        at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:142)
        at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(AbstractSAXParser.java:1216)
        at org.apache.xalan.transformer.TransformerIdentityImpl.transform(TransformerIdentityImpl.java:519)
        at org.metanorma.fop.PDFGenerator.generateFOPIntermediateFormat(PDFGenerator.java:860)
        at org.metanorma.fop.PDFGenerator.runFOP(PDFGenerator.java:564)
        at org.metanorma.fop.PDFGenerator.convertmn2pdf(PDFGenerator.java:512)
        at org.metanorma.fop.PDFGenerator.process(PDFGenerator.java:326)
        at org.metanorma.fop.mn2pdf.main(mn2pdf.java:350)

[Intelligent2013]

The error occurs due the block with mathml:

<UnitsML xmlns="https://schema.unitsml.org/unitsml/1.0">
            <UnitSet>
...
<math xmlns="http://www.w3.org/1998/Math/MathML">
    <mi mathvariant="normal">µF</mi>
</math>

mn2pdf can't process it, because the element math isn't in the text. I'll omit it in the preprocess phase.

[Intelligent2013]

No, without Unitsml there is the error...

[Intelligent2013]

The issue in fop-core\src\main\java\org\apache\fop\render\intermediate\IFStructureTreeBuilder.java

    public void replayEventsForPageSequence(ContentHandler handler,
            int pageSequenceIndex) throws SAXException {
            pageSequenceEventRecorders.get(pageSequenceIndex).replay(handler);
    }

The value pageSequenceIndex is great than pageSequenceEventRecorders size.

[Intelligent2013]

I've applied very rough workaround solution:

        try {
            pageSequenceEventRecorders.get(pageSequenceIndex).replay(handler);
        } catch (IndexOutOfBoundsException ex) {
            System.out.println("Error on pageSequenceIndex=" + pageSequenceIndex);
            pageSequenceEventRecorders.get(pageSequenceEventRecorders.size() - 1).replay(handler);
        }

and PDF generated.

@ronaldtse here is first PDF for JIZ Z 8301: document_vertical.presentation.zip

The PDF doesn't contain the clause numbers in Japanese, because still I'm not able to generate XML on my machine.

Known issues:

• these element didn't rotate in the vertical layout (currently I don't know how to fix it and how long it will take):
  • page numbers in ToC
  • page numbers in the footer (left side)
  • table
  • block-container
• wrong parenthesis rotation around English text UPDATE 2024-10-27: fixed
• wrong footer for annexes UPDATE 2024-10-27: fixed
• issue java.lang.IndexOutOfBoundsException: Index: 2, Size: 2 UPDATE 2024-10-27: fixed
• repeated last text page after each section (will fixed), UPDATE 2024-10-26: fixed

Vertical layout introduced in https://github.com/chunlinyao/fop is non-production feature, and time consuming to finding workaround solution.

[opoudjis]

Removing support for Noto Fonts in JIS until Fontist is made to support them: https://github.com/fontist/formulas/issues/206

[Intelligent2013]

Issue java.lang.IndexOutOfBoundsException: Index: 2, Size: 2 occurs due the attribute role="artifact" in

<fo:static-content flow-name="left-region" role="artifact">
...
<fo:retrieve-marker retrieve-class-name="section_title"

fo:retrieve-marker and role="artifact" in one element occurs this error. Looks like it's a bug in Apache FOP. I'll add the ticket on Apache tracker.

[ronaldtse]

This is the latest example from @Intelligent2013 20241106 vertical jis example.pdf

Metanorma issues:

• clause numbers in Japanese
• Japanese numbering of autonumbering items
• distinction encoding syntax for horizontal vs horizontal in vertical
• subfigures now in Japanese but mislabeled

What is to be done in FOP:

• clause numbers rotation
• punctuation rotation (see docs/110)
• table cell text rotation
• rotation of text in examples
• image/figure/formula rotation
• vertical line overflow for English

[Intelligent2013]

Metanorma issues:

• clause numbers in Japanese

@opoudjis does the current version of metanorma-jis support the clause numbers in Japanese?

[opoudjis]

Yes, with :presentation-metadata-autonumbering-style: japanese

https://github.com/metanorma/metanorma.org/pull/832/commits/b905b8722205b74d5335444806dcd06e6ce1152d

[Intelligent2013]

@opoudjis thanks!

[Intelligent2013]

This is the latest PDF with clause numbers in Japanese: 20241110.vertical.jis.example.pdf

[Intelligent2013]

• rotation of text in examples
• vertical line overflow for English

Fixed.

• image/figure rotation

I've added the rotation for the image, but now the image breaks the page:

I'll try to find the workaround solution.

The latest PDF: 20241111.vertical.jis.example.pdf

[Intelligent2013]

For the image rotation need:

• add reference-orientation="90" for the block-container
• add attributes width/height in mm for the block-container

Note: fox:transform="rotate(90)" (https://xmlgraphics.apache.org/fop/2.10/extensions.html#transform) is working for fo:block-container with position="absolute" only.

[Intelligent2013]

• image/figure rotation

  the image breaks the page:

The issue with page breaks fixed, but there are two issues with:

• sub-figures
• big height images

I'll need to fix it.

Latest PDF: 20241113.vertical.jis.example.pdf

[Intelligent2013]

Image scaling issue in the vertical layout fixed.

Next issue - each reference sub-section starts in a new page:

[Intelligent2013]

Next issue - each reference sub-section starts in a new page:

Fixed.

[Intelligent2013]

Latest PDF: 20241120.vertical.jis.example.pdf

Next issues:

• ordered list item label rotation (a) b) etc.)
• punctuation rotation