[TeX] Switch mono font to JuliaMono and Emoji font

inkydragon commented 2 years ago

There are still many Unicode symbols that cannot be displayed in the DejaVu Sans Mono font. According to tests, JuliaMono is currently the best choice.

⊼ ⊽ ⊻ ≡ ≤ ⧐
ℯ π ௰ ℵ Θ α β
⛵ 🚀
⋮ ⠀ ⠈ ⡀ ⠁ ⠠
∤ ⅋ ≀ ⊼ ⋇ ⋉ ⋊ ⋋ ⋌ ⋒ ⟑ ⦸ ⦼ ⦾ ⦿ ⧶ ⧷ ⨇ ⨰ ⨱ ⨲ ⨳ ⨴ ⨵ ⨶ ⨷ ⨸ ⨻ ⨼ ⨽ ⩀ ⩃ ⩄ ⩋ ⩍ ⩎ ⩑ ⩓ ⩕
⩘ ⩚ ⩜ ⩞ ⩟ ⩠ ⫛ ⨝ ⟕ ⟖ ⟗ ∔ ⊽ ⋓ ⨈ ⨢ ⨣ ⨤ ⨥ ⨦ ⨧ ⨨ ⨩ ⨪ ⨫ ⨬ ⨭ ⨮ ⨹ ⨺ ⩁ ⩂ ⩅ ⩊ ⩌ ⩏ ⩐ ⩒ ⩔ ⩖ ⩗ ⩛ ⩝ ⩡ ⩢ ⩣

`DejaVu Sans Mono` vs. `JuliaMono` Result

Self-contained TeX test case: minted-math-sym

Note: JuliaMono don't support emoji.

Font comparators: Monospace fonts for math

mortenpi commented 2 years ago

I think it would be awesome to use JuliaMono in the PDF. Are there any technical difficulties with switching it over.

As for symbols (including emojis) not supported by JuliaMono -- how doable you think would be a fallback to DejaVu Sans Mono in those cases? We have a fallback hack for one symbol: https://github.com/JuliaDocs/Documenter.jl/blob/cd49b078a8ce6e1ff17f56d8b98f3f92e93b963f/assets/latex/documenter.sty#L7-L11 But I am not sure how feasible it would be to it for a whole big set of symbols.

inkydragon commented 2 years ago

Are there any technical difficulties with switching it over.

As of now, no.

diff --git a/documenter.sty b/documenter.sty     
index 1bccfff..ace21d9 100644
--- a/documenter.sty
+++ b/documenter.sty
@@ -2,13 +2,9 @@
 \usepackage{fontspec, newunicodechar, polyglossia}

 \setsansfont{DejaVu Sans}[Scale=MatchLowercase, Ligatures=TeX]
-\setmonofont{DejaVu Sans Mono}[Scale=MatchLowercase]
+\setmonofont{JuliaMono Regular}[Scale=MatchLowercase]
+\newcommand\unicodeveebar{ ⊻ }
 \renewcommand{\familydefault}{\sfdefault}
-% DejaVu Sans Mono does not have the xor symbol. So we hack around that by replacing all
-% instances of it with calls to \unicodeveebar, which prints this single character as with
-% DejaVu Sans instead.
-\newfontfamily\unicodeveebarfont{DejaVu Sans}[Scale=MatchLowercase]
-\newcommand\unicodeveebar{{\unicodeveebarfont ⊻}}
 %

how doable you think would be a fallback to DejaVu Sans Mono in those cases?

First, let's be clear that we cannot expect one font to be used in all scenarios.

After some exploration, I found that we probably need at least 5 kinds of fonts.

[x] Choosing JuliaMono as the mono font is at least as good as the DejaVu Sans Mono currently in use
[x] We need Chinese and Korean fonts: CJK
[ ] emoji needs new fonts: Emoji font; May needs to switch to use lualatex.
[ ] Mathematical symbols require careful font selection, to be explored further: Math font, unimath-symbols.pdf; Some tests: source code unicode-table/math-sym; font comparison results PDFs

mortenpi commented 2 years ago

It would also be necessary to make the Julia Mono font files available to the user somehow. And the same applies to other fonts not easily available in Linux distros. But I think we can bake them into the Docker image.

May needs to switch to use lualatex.

I believe we already build with LuaTex, as we pass -lualatex to latexmk?

cormullion commented 2 years ago

As for symbols (including emojis) not supported by JuliaMono

I'm currently preparing another release of JuliaMono, so if there are any additional Unicode characters you need me to add, now's a good time to add them. (Emojis, though, are usually handled by the OS, and I'm more than happy to not do any of them.😀)

ViralBShah commented 2 years ago

Should we transition Documenter.jl to default to JuliaMono?

cormullion commented 2 years ago

I don't know LaTeX, but does this address the emoji/font switching issue:

https://github.com/Pomax/ucharclasses

inkydragon commented 2 years ago

emoji/font switching issue

I choose \usepackage{emoji}. And I'm looking for a good emoji font. Some choices: https://github.com/inkydragon/Julia-LaTeX-doc/blob/master/LaTeX-issues/unicode-table/memo.md

cormullion commented 2 years ago

fonts provided through google-fonts (Noto) probably safer in terms of support and maintenance

ViralBShah commented 2 years ago

At the very least we should get an emoji font in place for the manual.

mortenpi commented 2 years ago

Would be happy to look at the PR if someone puts a proof of concept together. We now have a small LaTeX showcase PDF being built as part of the test suite (see test/examples/src.latex_showcase and in make.jl), and the resulting PDF also gets uploaded as an artifact. So any examples demonstrating the font support can go in there.

Also, there was a recent related issue about Chinese characters: #1918

ViralBShah commented 2 years ago

To use emoji, I believe you need to use lualatex. Does that work with tectonic? Also is just loading the package sufficient, or do we need to annotate some font changes too where the emoji are used?

viral-laptop:build viral$ tectonic -X compile --keep-intermediates --keep-logs -Z shell-escape  TheJuliaLanguage.tex
note: "version 2" Tectonic command-line interface activated
Running TeX ...
error: emoji.sty:24: Critical Package emoji Error: The 'emoji' package requires LuaTeX.

After doing usepackage{emoji} and building with lualatex, the build goes through, but I still get:

[823] [824
Missing character: There is no ⛵ (U+26F5) in font DejaVuSansMono:mode=harf;sc
ript=latn;language=dflt;!

inkydragon commented 2 years ago

I believe you need to use lualatex.

Yes. Perhaps we can build the Unicode input table separately and insert it as a PDF.

Does that work with tectonic?

I don't know, but I guess NO. https://github.com/tectonic-typesetting/tectonic/issues/158

After doing usepackage{emoji}

install https://github.com/mozilla/twemoji-colr/releases

build font information cache files: fc-cache -fv

%!TEX program = lualatex
\documentclass{ctexbook}  %% import `\newCJKfontfamily`
\usepackage{emoji}

%% set global emoji font
\setemojifont{Twemoji Mozilla}
%% use `\newCJKfontfamily` to create a font sytle
\newCJKfontfamily\EmojiFont{Twemoji Mozilla}[Renderer=HarfBuzz]
%% use `\newfontface` to ...
% \newfontface\EmojiFont{Twemoji Mozilla}[Renderer=HarfBuzz]

\begin{document}

\verb!\newCJKfontfamily\EmojiFont! \\
U+03030 | {\EmojiFont 〰}
U+0303D | {\EmojiFont 〽}
U+03297 | {\EmojiFont ㊗}
U+03299 | {\EmojiFont ㊙}
U+02194 | {\EmojiFont ↔} 
U+02195 | {\EmojiFont ↕} \\

\verb!\newCJKfontfamily\EmojiFont + \emoji{}! \\
U+03030 | \emoji{wavy-dash}
U+0303D | \emoji{part-alternation-mark}
U+03297 | \emoji{congratulations}
U+03299 | \emoji{secret}
U+02194 | \emoji{left-right-arrow} 
U+02195 | \emoji{left-right-arrow} \\

\end{document}

Note: I use ctexbook more often, so I chose it, but it's not a required package.

ViralBShah commented 2 years ago

Maybe first prirority should then be to get all these characters working with the existing system, and worry about tectonic later?

JuliaDocs / Documenter.jl