PDF `--theme`: Character replacement

[infotexture]

When building PDF output with --theme, different fonts appear to be used when compared to default PDF2 output — even if the theme file does not override any font-family settings.

When building the DITA-OT docs PDF, certain characters are replaced with #, for example:

• In the Supported languages topic, all language names in the table are represented correctly when building with PDF2 — such as 日本語 (Japanese) — but output generated with --theme shows ### (Japanese).

• In codeblocks such as those in PDF plug-in structure, directory listing characters such as├ are replaced with #.

  • PDF2 output

    .
    ├── build.properties.orig
    ├── catalog.xml.orig
    └── fo/
    ├── attrs/
    │   └── custom.xsl.orig
    └── xsl/
        └── custom.xsl.orig

  • --theme output

    .
    ### build.properties.orig
    ### catalog.xml.orig
    ### fo/
    ### attrs/
    #   ### custom.xsl.orig
    ### xsl/
        ### custom.xsl.orig

[jelovirt]

Theme plugin disables I18N processing. This may be the cause.

[infotexture]

Tried re-enabling I18N processing with --org.dita.pdf2.i18n.enabled=true, but that didn't change anything.

I thought parameter values passed via the CLI would take precedence over those set in plug-ins, but either that's not the case, or I18N processing isn't the culprit here.

[jelovirt]

Tested this and the theme file and generation works as expected.

If you only specify e.g. font-family: Helvetica, there is no fall-back font to use for characters not in Helvetica. If you specify font-family: Helvetica, Arial Unicode MS, then e.g. Japanese will work with the fall-back font.

If you don't specify a font for style-base key, the default PDF2 configuration of serif and that will be something that's configured in FOP. Same goes for codeblock that PDF2 defaults to monospace font; that might not have ASCII art characters, it all depends on the configuration of FOP.

[infotexture]

ℹ️ For future reference

Resolved per https://github.com/jelovirt/pdf-generator/issues/53#issuecomment-1284390737 by adding custom variables that declare the default PDF2 font stacks and referencing those later in the theme:

# Default font stacks from `org.dita.pdf2`
pdf2:
  font:
    sans: 'Helvetica, Arial Unicode MS, Tahoma, Batang, SimSun'
    serif: 'Times New Roman, Times, Arial Unicode MS, Tahoma, Batang, SimSun'
    monospaced: 'Courier New, Courier, Arial Unicode MS, Tahoma, Batang, SimSun'

style:
  codeblock:
    font-family: $pdf2-font-monospaced
  table:
    font-family: $pdf2-font-sans