Greek unicode letters not rendered in pdf output

peteole commented 2 years ago

Bug description

Using unicode letters and rendering to pdf results in the letters bein omitted (on manjaro linux, vscode). Setting

mainfont: Times New Roman

lets me render unicode in the main text flow, but not in code blocks. Also this option results in an error when run on my system because the font is not installed. This is not the best UX. This should just work without user interaction. See this example:

---
title: "test"
jupyter: julia-1.8
---

```{julia}
α=3

α is a great constant.



------------------------------------------------------------------------------
The output is
![image](https://user-images.githubusercontent.com/56505957/197622722-1b744bd7-3769-4e33-a8da-4b44722a5e7a.png)

### Checklist

- [X] Please include a minimal, fully reproducible example in a single .qmd file? Please provide the whole file rather than the snippet you believe is causing the issue.
- [X] Please [format your issue](https://quarto.org/bug-reports.html#formatting-make-githubs-markdown-work-for-us) so it is easier for us to read the bug report.
- [X] Please document the RStudio IDE version you're running (if applicable), by providing the value displayed in the "About RStudio" main menu dialog?
- [X] Please document the operating system you're running. If on Linux, please provide the specific distribution.

cscheid commented 2 years ago

Thanks for the report. I don't know how we could proceed without user interaction when fonts is not installed. When latex packages are missing, we automatically install them. But in the case of missing fonts, there's no standard repository.

cscheid commented 2 years ago

In the meantime, I've found that the following front matter will make it so that Unicode symbols such as alpha, etc will generally work on PDF output:

format:
  pdf:
    monofont: "DejaVu Sans Mono"
    mainfont: "Times New Roman"

Naturally, you'll need to install DejaVu Sans Mono.

peteole commented 2 years ago

One part of the issue is handling missing fonts. Maybe it makes sense to just package the default font with quarto? The other part of the issue is that by default greek letters are not displayed. Should we maybe change the default font or at least provide a more clear error/warning? And what can I do to display greek letters in code blocks?

cscheid commented 2 years ago

The default font is installed, but Computer Modern does not have unicode symbols. We could try to scan the document for code points that are missing, but coming up with a font that handles those correctly is a generally hard problem (especially since Unicode continues to add new code points over time). I think this is a limitation of LaTeX in general that we won't be able to solve with quarto.

I do agree that we should better document this Unicode limitation!

capnrefsmmat commented 2 years ago

Would it be worth bundling a font like STIX Two, which is built by technical publishers to include all the characters and symbols they might conceivably need?

It probably doesn't include a full set of emoji, for instance, but it includes Latin, Greek, and Cyrillic, plus diacritics and technical symbols.

allenmanning commented 1 year ago

Pushing out to 1.4 and re-assigning to @cwickham to consider a better way to document this limitation.

gorkamunoz commented 1 year ago

Hi, I think I am facing a similar problem building the documentation via docments (through nbdev). Changing the font as commented above did not work for me. What I see is that some of the Greek letters actually work, see:

Is there any workaround to make this work? Thanks!

AlloriMD commented 1 year ago

Hello everyone. I have additional details on this error (missing unicode symbols), where it seems to be related to use of the knitr engine but not the jupyter engine.

What I did:

Created a .qmd file with Latin alphanumerics, Greek letters, and some unicode/emoji in Markdown sections. Additionally, the file contained two code chunks, one using R and the other using Python, each generating a figure, to verify whether code was being executed as I modified the engine (see below).
First, I allowed the engine to be auto-detected by Quarto. This was presumably knitr, as the R code chunk appeared first in the file.
I rendered the file: quarto render my_file.qmd --to=pdf
As separate tests, I manually specified the engines as knitr or as jupyter and re-rendered the PDF.

What I observed:

When I let quarto auto-detect the engine (presumably knitr), the Greek letters and unicode/emoji symbols did not render.
When I manually set the engine to jupyter, all Greek letters and unicode/emoji symbols rendered perfectly; however, the plot designated by R code did not generate. (The Python figure still generated.)
When I manually set the engine to knitr, the Greek letters and unicode/emoji symbols again did not render. (Both R and Python figures were generated.)

What I expected:

All Greek letters and unicode/emoji should render for any of the engine choices (if the font allows, which is the case in my example)
[As a distinct issue, I am confused why R and Python code both execute with the knitr engine but not with the jupyter engine (since Jupyter should be capable of running all these languages).]

System Specs: macOS Ventura 13.4.1(c) Quarto v1.3.433

$ quarto check

[✓] Checking versions of quarto binary dependencies...
      Pandoc version 3.1.1: OK
      Dart Sass version 1.55.0: OK
[✓] Checking versions of quarto dependencies......OK
[✓] Checking Quarto installation......OK
      Version: 1.3.433
      Path: /Applications/quarto/bin

[✓] Checking basic markdown render....OK

[✓] Checking Python 3 installation....OK
      Version: 3.11.4 (Conda)
      Path: /Applications/miniconda3/envs/quarto/bin/python
      Jupyter: 5.3.1
      Kernels: python3

(/) Checking Jupyter engine render....0.00s - Debugger warning: It seems that frozen modules are being used, which may
0.00s - make the debugger miss breakpoints. Please pass -Xfrozen_modules=off
0.00s - to python to disable frozen modules.
0.00s - Note: Debugging will proceed. Set PYDEVD_DISABLE_FILE_VALIDATION=1 to disable this validation.
0.00s - Debugger warning: It seems that frozen modules are being used, which may
0.00s - make the debugger miss breakpoints. Please pass -Xfrozen_modules=off
0.00s - to python to disable frozen modules.
0.00s - Note: Debugging will proceed. Set PYDEVD_DISABLE_FILE_VALIDATION=1 to disable this validation.
[✓] Checking Jupyter engine render....OK

[✓] Checking R installation...........OK
      Version: 4.2.3
      Path: /Library/Frameworks/R.framework/Resources
      LibPaths:
        - /Library/Frameworks/R.framework/Versions/4.2-arm64/Resources/library
      knitr: 1.42
      rmarkdown: 2.21

[✓] Checking Knitr engine render......OK

Any help would be greatly appreciated!!!

Thank you for making Quarto such a great program.

sje30 commented 1 year ago

CC: @jmb280cam

cwickham commented 8 months ago

(Adding a note for the documentation task)

Mostly covered in https://quarto.org/docs/output-formats/pdf-basics.html#unicode-characters but this section could use a comment on unicode characters in code cells and setting monofont.

Also see https://github.com/quarto-dev/quarto-cli/issues/5185

quarto-dev / quarto-cli

Greek unicode letters not rendered in pdf output #3013

Bug description