search

quarto-dev / quarto-cli

Open-source scientific and technical publishing system built on Pandoc.
https://quarto.org
Other
3.69k stars 298 forks source link

Quarto v1.5 installs a nonexistent package `hyphen-chinese-hans` when compiling PDF #10291

Open leovan opened 1 month ago

leovan commented 1 month ago

Bug description

For a book type project with lang: zh, Quarto v1.5 installs a nonexistent package hyphen-chinese-hans when compiling PDF. hyphen-chinese is a valid package, but hyphen-chinese-hans is not. Quarto v1.4 works well. Error output is shown as below:

Rendering PDF
running xelatex - 1
  This is XeTeX, Version 3.141592653-2.6-0.999996 (TeX Live 2024) (preloaded format=xelatex)
   restricted \write18 enabled.
  entering extended mode

> 1 package to install
> installing hyphen-chinese-hans (1 of 1)
ERROR: tlmgr returned a non zero status code
tlmgr install: package hyphen-chinese-hans not present in repository.
tlmgr: action install returned an error; continuing.
tlmgr: An error has occurred. See above messages. Exiting.

Stack trace:
tlmgr install: package hyphen-chinese-hans not present in repository.
tlmgr: action install returned an error; continuing.
tlmgr: An error has occurred. See above messages. Exiting.

    at Object.onComplete (file:///Applications/quarto/bin/quarto.js:92847:64)
    at eventLoopTick (ext:core/01_core.js:153:7)
    at async renderFiles (file:///Applications/quarto/bin/quarto.js:78077:16)
    at async renderProject (file:///Applications/quarto/bin/quarto.js:78479:25)
    at async Command.actionHandler (file:///Applications/quarto/bin/quarto.js:83077:32)
    at async Command.execute (file:///Applications/quarto/bin/quarto.js:8017:13)
    at async Command.parseCommand (file:///Applications/quarto/bin/quarto.js:7907:20)
    at async quarto (file:///Applications/quarto/bin/quarto.js:118219:9)
    at async file:///Applications/quarto/bin/quarto.js:118238:9
    at async mainRunner (file:///Applications/quarto/bin/quarto.js:118123:9)

Steps to reproduce

Repo: https://github.com/leovan/quarto-issue

Expected behavior

No response

Actual behavior

No response

Your environment

No response

Quarto check output

Quarto 1.5.54
[✓] Checking versions of quarto binary dependencies...
      Pandoc version 3.2.0: OK
      Dart Sass version 1.70.0: OK
      Deno version 1.41.0: OK
      Typst version 0.11.0: OK
[✓] Checking versions of quarto dependencies......OK
[✓] Checking Quarto installation......OK
      Version: 1.5.54
      Path: /Applications/quarto/bin

[✓] Checking tools....................OK
      TinyTeX: (not installed)
      Chromium: (not installed)

[✓] Checking LaTeX....................OK
      Using: Installation From Path
      Path: /Library/TeX/texbin
      Version: 2024

[✓] Checking basic markdown render....OK

[✓] Checking Python 3 installation....OK
      Version: 3.10.14
      Path: /Users/leo/Applications/Python/python310-dev/bin/python
      Jupyter: (None)

      Jupyter is not available in this Python installation.
      Install with python3 -m pip install jupyter

[✓] Checking R installation...........OK
      Version: 4.4.1
      Path: /Library/Frameworks/R.framework/Resources
      LibPaths:
        - /Library/Frameworks/R.framework/Versions/4.4-arm64/Resources/library
      knitr: 1.47
      rmarkdown: 2.27

[✓] Checking Knitr engine render......OK
mcanouil commented 1 month ago

@leovan Thanks for the report. unfortunately we currently can't use your report as you did not provide us with the means to reproduce the issue as requested in the template. Please update your report (by editing it) with the required information. This includes your environment. Thank you for your cooperation.

leovan commented 1 month ago

I've added a minimum repo to reproduce the issue: https://github.com/leovan/quarto-issue

mcanouil commented 1 month ago

Thanks!

I can even reproduce with the following:

---
tile: "范叶亮"
format: pdf
lang: zh
---

# 范叶亮

The issue is not in the LaTeX part but in the processing after that to render the document, in particular the automatic package discovery/install.

The change comes from an attempt to fix issues with hyphenation:

The following might need another exception for chinese (in addition to german):

leovan commented 1 month ago

Thanks for checking the issue, look forward to the fix.

Jay-kkk commented 1 month ago

I had the same error, look forward to the fix

leovan commented 1 month ago

I think the quick fix is to make latex-auto-install option (https://quarto.org/docs/output-formats/pdf-engine.html#package-installation) works for install hyphen package too.

https://github.com/quarto-dev/quarto-cli/blob/9724c3dd403d4e723bcb0a7e7bef6e49d23a353a/src/command/render/latexmk/pdf.ts#L177-L191

For line 182, it can be revised to:

   if (missingHyphenationFile && pkgMgr.autoInstall) {
mcanouil commented 1 month ago

The suggested change won't change the fact that Quarto tries to determine the hyphen file and will try to install it. Making the auto install will only make sure the correct package is installed but won't correct the bad name so it should lead to the exact same error.

leovan commented 1 month ago

OK, it seems adding another exception for chinese is a better way to fix this.

cderv commented 1 month ago

This is indeed a problem of Quarto trying to guess which package needs to be installed when the babel warning is encounter

Package babel Info: Hyphen rules for 'chinese-hans' set to \l@nil
(babel)             (\language10). Reported on input line 143.

We build the package name based on value from the log.

hyphen-chinese is a valid package, but hyphen-chinese-hans is not.

Installing hyphen-chinese does not solve the problem as the warning message is still present. So it seems to be another package, or a different file / fonts that would need to be installed when not present already. This is required to add the correct exception.

leovan commented 1 month ago

According to https://ctan.math.utah.edu/ctan/tex-archive/macros/latex/required/babel/base/babel.pdf, in section List of locales available in \babelprovide, chinese (zh) is used for:

chinese-simplified (zh-Hans)
chinese-hans (zh-Hans)
chinese-traditional (zh-Hant)
chinese-hant (zh-Hant)
chinese-simplified-hongkongsarchina (zh-Hans-HK)
chinese-hans-hk (zh-Hans-HK)
chinese-simplified-macausarchina (zh-Hans-MO)
chinese-hans-mo (zh-Hans-MO)
chinese-simplified-singapore (zh-Hans-SG)
chinese-hans-sg (zh-Hans-SG)
chinese-hant-hk (zh-Hant-HK)
chinese-traditional-hongkongsarchina (zh-Hant-HK)
chinese-hant-mo (zh-Hant-MO)
chinese-traditional-macausarchina (zh-Hant-MO)

Will the following code works?

if (lang === "ngerman") {
    return "hyphen-german"; 
} else if (lang.startsWith("chinese") {
    return "hyphen-chinese";
}

return `hyphen-${lang.toLowerCase()}`;

I think this will install hyphen-chinese everytime, becase the error message Package babel Info: Hyphen rules for 'chinese-hans' set to \l@nil still exists after installing hyphen-chinese. But it can continue the compling and produce the final PDF.

Really need a quick fix, otherwise Quarto v1.5 won't compile doc with lang: zh to PDF correctly.

cderv commented 1 month ago

I did try that. But installing hyphen-chinese does not prevent the warning to show in the document, at least in my test. You could try on your side, by installing the package from CTAN and re-render. If the warning from babel is still in log file, Quarto will still attempt to install it.

leovan commented 1 month ago

OK, I will try later to see whether we can get a quick fix for this issue.

cderv commented 1 month ago

hyphen-chinese is a valid package

I can't find it on CTAN. Are you sure this is the name of the package ?

leovan commented 1 month ago

Yes, cmd output:

❯ sudo tlmgr install hyphen-chinese
Password:
tlmgr: package repository https://mirrors.pku.edu.cn/ctan/systems/texlive/tlnet (verified)
tlmgr install: package already present: hyphen-chinese

And in TeX Live Utility, it is shown installed too.

SCR-20240715-pnbm
cderv commented 1 month ago

Oh great thanks. This is a sytem package then. It does not solve the warning though, so not sure if we should ask for installation for this warning problem.

Another problem we have is that if this is already install the rendering should not stop... 🤔

cderv commented 1 month ago

I think the quick fix is to make latex-auto-install option (quarto.org/docs/output-formats/pdf-engine.html) works for install hyphen package too.

@leovan I do think this is the right thing to do here. At least to unblock. Quarto LaTeX engine does auto install missing hyphenation package based on log warnings, but the latex-auto-install configuration does not applied. IMO it should. Thanks for pointing that out !

Quarto v1.5 installs a nonexistent package hyphen-chinese-hans when compiling PDF

Regarding this problem, Quarto does have a recipe to guess which package to install based on log warning. Here, it builds a wrong name so we need an exception. Though even when hyphen-chinese is present, the warning is still there. So we keep trying to install, and at the second time we fail.

And we do prevent PDF rendering for a hyphenation warning in the log, not resolved. https://github.com/quarto-dev/quarto-cli/blob/1c1609c4de06f4d530e448731cca165ee0121e98/src/command/render/latexmk/pdf.ts#L178-L190

We could also relax this constraint, and continue the rendering anyway... 🤔

leovan commented 1 month ago

I made a test, code changes:

diff --git a/src/command/render/latexmk/pdf.ts b/src/command/render/latexmk/pdf.ts
index 52062aa3f..ec9eb0554 100644
--- a/src/command/render/latexmk/pdf.ts
+++ b/src/command/render/latexmk/pdf.ts
@@ -179,7 +179,7 @@ async function initialCompileLatex(
       // See (https://github.com/rstudio/tinytex/commit/0f2007426f730a6ed9d45369233c1349a69ddd29)
       const logText = Deno.readTextFileSync(response.log);
       const missingHyphenationFile = findMissingHyphenationFiles(logText);
-      if (missingHyphenationFile) {
+      if (pkgMgr.autoInstall && missingHyphenationFile) {
         if (await pkgMgr.installPackages([missingHyphenationFile])) {
           // We installed hyphenation files, retry
           continue;

qmd example:

---
tile: "范叶亮"
format: pdf
lang: zh
keep-tex: true
latex-auto-install: false
---

# 范叶亮

Without latex-auto-install: false:

Quarto version: 99.9.9
pandoc --verbose --trace
  to: latex
  output-file: zh.tex
  standalone: true
  pdf-engine: xelatex
  variables:
    graphics: true
    tables: true
  default-image-extension: pdf

metadata
  documentclass: scrartcl
  classoption:
    - DIV=11
    - numbers=noendperiod
  papersize: letter
  header-includes:
    - '\KOMAoption{captions}{tableheading}'
  block-headings: true
  tile: 范叶亮
  lang: zh

[execProcess] /Users/leo/Documents/Projects/TypeScript/quarto-cli/package/dist/bin/tools/aarch64/pandoc +RTS -K512m -RTS --defaults /var/folders/6j/_g2nn8gj1b5_c0kw6c48_4fr0000gn/T/quarto-sessionaa09b94676a0f145/5539a4b7f6934776/quarto-defaults3307782351324eff.yml /var/folders/6j/_g2nn8gj1b5_c0kw6c48_4fr0000gn/T/quarto-sessionaa09b94676a0f145/5539a4b7f6934776/quarto-inputf75c4f14f272bf5c.md --metadata-file /var/folders/6j/_g2nn8gj1b5_c0kw6c48_4fr0000gn/T/quarto-sessionaa09b94676a0f145/5539a4b7f6934776/quarto-metadata5f0552045cd04d13.yml --verbose --trace --data-dir /Users/leo/Documents/Projects/TypeScript/quarto-cli/src/resources/pandoc/datadir
[INFO] Loaded /var/folders/6j/_g2nn8gj1b5_c0kw6c48_4fr0000gn/T/quarto-sessionaa09b94676a0f145/5539a4b7f6934776/643eacc3cb17c578/template.patched from /var/folders/6j/_g2nn8gj1b5_c0kw6c48_4fr0000gn/T/quarto-sessionaa09b94676a0f145/5539a4b7f6934776/643eacc3cb17c578/template.patched
[INFO] Loaded /var/folders/6j/_g2nn8gj1b5_c0kw6c48_4fr0000gn/T/quarto-sessionaa09b94676a0f145/5539a4b7f6934776/643eacc3cb17c578/doc-class.tex from /var/folders/6j/_g2nn8gj1b5_c0kw6c48_4fr0000gn/T/quarto-sessionaa09b94676a0f145/5539a4b7f6934776/643eacc3cb17c578/doc-class.tex
[INFO] Loaded /var/folders/6j/_g2nn8gj1b5_c0kw6c48_4fr0000gn/T/quarto-sessionaa09b94676a0f145/5539a4b7f6934776/643eacc3cb17c578/pandoc.tex from /var/folders/6j/_g2nn8gj1b5_c0kw6c48_4fr0000gn/T/quarto-sessionaa09b94676a0f145/5539a4b7f6934776/643eacc3cb17c578/pandoc.tex
[INFO] Loaded /var/folders/6j/_g2nn8gj1b5_c0kw6c48_4fr0000gn/T/quarto-sessionaa09b94676a0f145/5539a4b7f6934776/643eacc3cb17c578/tightlist.tex from /var/folders/6j/_g2nn8gj1b5_c0kw6c48_4fr0000gn/T/quarto-sessionaa09b94676a0f145/5539a4b7f6934776/643eacc3cb17c578/tightlist.tex
[INFO] Loaded /var/folders/6j/_g2nn8gj1b5_c0kw6c48_4fr0000gn/T/quarto-sessionaa09b94676a0f145/5539a4b7f6934776/643eacc3cb17c578/tables.tex from /var/folders/6j/_g2nn8gj1b5_c0kw6c48_4fr0000gn/T/quarto-sessionaa09b94676a0f145/5539a4b7f6934776/643eacc3cb17c578/tables.tex
[INFO] Loaded /var/folders/6j/_g2nn8gj1b5_c0kw6c48_4fr0000gn/T/quarto-sessionaa09b94676a0f145/5539a4b7f6934776/643eacc3cb17c578/graphics.tex from /var/folders/6j/_g2nn8gj1b5_c0kw6c48_4fr0000gn/T/quarto-sessionaa09b94676a0f145/5539a4b7f6934776/643eacc3cb17c578/graphics.tex
[INFO] Loaded /var/folders/6j/_g2nn8gj1b5_c0kw6c48_4fr0000gn/T/quarto-sessionaa09b94676a0f145/5539a4b7f6934776/643eacc3cb17c578/citations.tex from /var/folders/6j/_g2nn8gj1b5_c0kw6c48_4fr0000gn/T/quarto-sessionaa09b94676a0f145/5539a4b7f6934776/643eacc3cb17c578/citations.tex
[INFO] Loaded /var/folders/6j/_g2nn8gj1b5_c0kw6c48_4fr0000gn/T/quarto-sessionaa09b94676a0f145/5539a4b7f6934776/643eacc3cb17c578/before-title.tex from /var/folders/6j/_g2nn8gj1b5_c0kw6c48_4fr0000gn/T/quarto-sessionaa09b94676a0f145/5539a4b7f6934776/643eacc3cb17c578/before-title.tex
[INFO] Loaded /var/folders/6j/_g2nn8gj1b5_c0kw6c48_4fr0000gn/T/quarto-sessionaa09b94676a0f145/5539a4b7f6934776/643eacc3cb17c578/title.tex from /var/folders/6j/_g2nn8gj1b5_c0kw6c48_4fr0000gn/T/quarto-sessionaa09b94676a0f145/5539a4b7f6934776/643eacc3cb17c578/title.tex
[INFO] Loaded /var/folders/6j/_g2nn8gj1b5_c0kw6c48_4fr0000gn/T/quarto-sessionaa09b94676a0f145/5539a4b7f6934776/643eacc3cb17c578/before-body.tex from /var/folders/6j/_g2nn8gj1b5_c0kw6c48_4fr0000gn/T/quarto-sessionaa09b94676a0f145/5539a4b7f6934776/643eacc3cb17c578/before-body.tex
[INFO] Loaded /var/folders/6j/_g2nn8gj1b5_c0kw6c48_4fr0000gn/T/quarto-sessionaa09b94676a0f145/5539a4b7f6934776/643eacc3cb17c578/toc.tex from /var/folders/6j/_g2nn8gj1b5_c0kw6c48_4fr0000gn/T/quarto-sessionaa09b94676a0f145/5539a4b7f6934776/643eacc3cb17c578/toc.tex
[INFO] Loaded /var/folders/6j/_g2nn8gj1b5_c0kw6c48_4fr0000gn/T/quarto-sessionaa09b94676a0f145/5539a4b7f6934776/643eacc3cb17c578/before-bib.tex from /var/folders/6j/_g2nn8gj1b5_c0kw6c48_4fr0000gn/T/quarto-sessionaa09b94676a0f145/5539a4b7f6934776/643eacc3cb17c578/before-bib.tex
[INFO] Loaded /var/folders/6j/_g2nn8gj1b5_c0kw6c48_4fr0000gn/T/quarto-sessionaa09b94676a0f145/5539a4b7f6934776/643eacc3cb17c578/biblio.tex from /var/folders/6j/_g2nn8gj1b5_c0kw6c48_4fr0000gn/T/quarto-sessionaa09b94676a0f145/5539a4b7f6934776/643eacc3cb17c578/biblio.tex
[INFO] Loaded /var/folders/6j/_g2nn8gj1b5_c0kw6c48_4fr0000gn/T/quarto-sessionaa09b94676a0f145/5539a4b7f6934776/643eacc3cb17c578/after-body.tex from /var/folders/6j/_g2nn8gj1b5_c0kw6c48_4fr0000gn/T/quarto-sessionaa09b94676a0f145/5539a4b7f6934776/643eacc3cb17c578/after-body.tex
[trace] Parsed [Plain [Str "DIV=11"]] at line 1
[trace] Parsed [Plain [Str "numbers=noendperiod"]] at line 1
[trace] Parsed [Plain [Str "scrartcl"]] at line 1
[trace] Parsed [Plain [Str "zh-Hans"]] at line 1
[trace] Parsed [Plain [Str "blue"]] at line 1
[trace] Parsed [Plain [Str "letter"]] at line 1
[trace] Parsed [Plain [Str "\33539\21494\20142"]] at line 1
[trace] Parsed [Plain [Str "\30446\24405"]] at line 1
[trace] Parsed [] at line 6
[trace] Parsed [Header 1 ("\33539\21494\20142",[],[]) [Str "\33539\21494\20 at line 9
[INFO] Running filter /Users/leo/Documents/Projects/TypeScript/quarto-cli/src/resources/filters/main.lua
[INFO] Completed filter /Users/leo/Documents/Projects/TypeScript/quarto-cli/src/resources/filters/main.lua in 34 ms
[execProcess] Success: true, code: 0

Rendering PDF
running xelatex - 1
[execProcess] tlmgr --version
[execProcess] Success: true, code: 0
[execProcess] tlmgr --version
[execProcess] Success: true, code: 0
[execProcess] tlmgr --version
[execProcess] Success: true, code: 0
[execProcess] xelatex -interaction=batchmode -halt-on-error zh.tex
[execProcess] Success: true, code: 0
  This is XeTeX, Version 3.141592653-2.6-0.999996 (TeX Live 2024) (preloaded format=xelatex)
   restricted \write18 enabled.
  entering extended mode

> 1 package to install
> installing hyphen-chinese-hans (1 of 1)
[execProcess] tlmgr info --list --only-installed --data name hyphen-chinese-hans
[execProcess] Success: false, code: 1
[execProcess] tlmgr install hyphen-chinese-hans
[execProcess] Success: false, code: 1
[NotebookContext]: Starting Cleanup
ERROR: [non-error-thrown] tlmgr returned a non zero status code
tlmgr install: package hyphen-chinese-hans not present in repository.
tlmgr: action install returned an error; continuing.
tlmgr: An error has occurred. See above messages. Exiting.

Stack trace:
tlmgr install: package hyphen-chinese-hans not present in repository.
tlmgr: action install returned an error; continuing.
tlmgr: An error has occurred. See above messages. Exiting.

    at Command.handleError (file:///Users/leo/Documents/Projects/TypeScript/quarto-cli/src/vendor/deno.land/x/cliffy@v1.0.0-rc.3/command/command.ts:2202:11)
    at Command.parseCommand (file:///Users/leo/Documents/Projects/TypeScript/quarto-cli/src/vendor/deno.land/x/cliffy@v1.0.0-rc.3/command/command.ts:1782:12)
    at eventLoopTick (ext:core/01_core.js:153:7)
    at async quarto (file:///Users/leo/Documents/Projects/TypeScript/quarto-cli/src/quarto.ts:159:5)
    at async file:///Users/leo/Documents/Projects/TypeScript/quarto-cli/src/quarto.ts:180:5
    at async mainRunner (file:///Users/leo/Documents/Projects/TypeScript/quarto-cli/src/core/main.ts:35:5)
    at async file:///Users/leo/Documents/Projects/TypeScript/quarto-cli/src/quarto.ts:170:3

With latex-auto-install: false, it compiles the PDF successfully.

Quarto version: 99.9.9
pandoc --verbose --trace
  to: latex
  output-file: zh.tex
  standalone: true
  pdf-engine: xelatex
  variables:
    graphics: true
    tables: true
  default-image-extension: pdf

metadata
  documentclass: scrartcl
  classoption:
    - DIV=11
    - numbers=noendperiod
  papersize: letter
  header-includes:
    - '\KOMAoption{captions}{tableheading}'
  block-headings: true
  tile: 范叶亮
  lang: zh

[execProcess] /Users/leo/Documents/Projects/TypeScript/quarto-cli/package/dist/bin/tools/aarch64/pandoc +RTS -K512m -RTS --defaults /var/folders/6j/_g2nn8gj1b5_c0kw6c48_4fr0000gn/T/quarto-sessionae6c4759c2f1ea77/ed5f845b90d10d67/quarto-defaults5f94d52e3d71252c.yml /var/folders/6j/_g2nn8gj1b5_c0kw6c48_4fr0000gn/T/quarto-sessionae6c4759c2f1ea77/ed5f845b90d10d67/quarto-input30f068c454c9ee7.md --metadata-file /var/folders/6j/_g2nn8gj1b5_c0kw6c48_4fr0000gn/T/quarto-sessionae6c4759c2f1ea77/ed5f845b90d10d67/quarto-metadata95f92bc2a715e2bf.yml --verbose --trace --data-dir /Users/leo/Documents/Projects/TypeScript/quarto-cli/src/resources/pandoc/datadir
[INFO] Loaded /var/folders/6j/_g2nn8gj1b5_c0kw6c48_4fr0000gn/T/quarto-sessionae6c4759c2f1ea77/ed5f845b90d10d67/6dbf47c10b3ea69c/template.patched from /var/folders/6j/_g2nn8gj1b5_c0kw6c48_4fr0000gn/T/quarto-sessionae6c4759c2f1ea77/ed5f845b90d10d67/6dbf47c10b3ea69c/template.patched
[INFO] Loaded /var/folders/6j/_g2nn8gj1b5_c0kw6c48_4fr0000gn/T/quarto-sessionae6c4759c2f1ea77/ed5f845b90d10d67/6dbf47c10b3ea69c/doc-class.tex from /var/folders/6j/_g2nn8gj1b5_c0kw6c48_4fr0000gn/T/quarto-sessionae6c4759c2f1ea77/ed5f845b90d10d67/6dbf47c10b3ea69c/doc-class.tex
[INFO] Loaded /var/folders/6j/_g2nn8gj1b5_c0kw6c48_4fr0000gn/T/quarto-sessionae6c4759c2f1ea77/ed5f845b90d10d67/6dbf47c10b3ea69c/pandoc.tex from /var/folders/6j/_g2nn8gj1b5_c0kw6c48_4fr0000gn/T/quarto-sessionae6c4759c2f1ea77/ed5f845b90d10d67/6dbf47c10b3ea69c/pandoc.tex
[INFO] Loaded /var/folders/6j/_g2nn8gj1b5_c0kw6c48_4fr0000gn/T/quarto-sessionae6c4759c2f1ea77/ed5f845b90d10d67/6dbf47c10b3ea69c/tightlist.tex from /var/folders/6j/_g2nn8gj1b5_c0kw6c48_4fr0000gn/T/quarto-sessionae6c4759c2f1ea77/ed5f845b90d10d67/6dbf47c10b3ea69c/tightlist.tex
[INFO] Loaded /var/folders/6j/_g2nn8gj1b5_c0kw6c48_4fr0000gn/T/quarto-sessionae6c4759c2f1ea77/ed5f845b90d10d67/6dbf47c10b3ea69c/tables.tex from /var/folders/6j/_g2nn8gj1b5_c0kw6c48_4fr0000gn/T/quarto-sessionae6c4759c2f1ea77/ed5f845b90d10d67/6dbf47c10b3ea69c/tables.tex
[INFO] Loaded /var/folders/6j/_g2nn8gj1b5_c0kw6c48_4fr0000gn/T/quarto-sessionae6c4759c2f1ea77/ed5f845b90d10d67/6dbf47c10b3ea69c/graphics.tex from /var/folders/6j/_g2nn8gj1b5_c0kw6c48_4fr0000gn/T/quarto-sessionae6c4759c2f1ea77/ed5f845b90d10d67/6dbf47c10b3ea69c/graphics.tex
[INFO] Loaded /var/folders/6j/_g2nn8gj1b5_c0kw6c48_4fr0000gn/T/quarto-sessionae6c4759c2f1ea77/ed5f845b90d10d67/6dbf47c10b3ea69c/citations.tex from /var/folders/6j/_g2nn8gj1b5_c0kw6c48_4fr0000gn/T/quarto-sessionae6c4759c2f1ea77/ed5f845b90d10d67/6dbf47c10b3ea69c/citations.tex
[INFO] Loaded /var/folders/6j/_g2nn8gj1b5_c0kw6c48_4fr0000gn/T/quarto-sessionae6c4759c2f1ea77/ed5f845b90d10d67/6dbf47c10b3ea69c/before-title.tex from /var/folders/6j/_g2nn8gj1b5_c0kw6c48_4fr0000gn/T/quarto-sessionae6c4759c2f1ea77/ed5f845b90d10d67/6dbf47c10b3ea69c/before-title.tex
[INFO] Loaded /var/folders/6j/_g2nn8gj1b5_c0kw6c48_4fr0000gn/T/quarto-sessionae6c4759c2f1ea77/ed5f845b90d10d67/6dbf47c10b3ea69c/title.tex from /var/folders/6j/_g2nn8gj1b5_c0kw6c48_4fr0000gn/T/quarto-sessionae6c4759c2f1ea77/ed5f845b90d10d67/6dbf47c10b3ea69c/title.tex
[INFO] Loaded /var/folders/6j/_g2nn8gj1b5_c0kw6c48_4fr0000gn/T/quarto-sessionae6c4759c2f1ea77/ed5f845b90d10d67/6dbf47c10b3ea69c/before-body.tex from /var/folders/6j/_g2nn8gj1b5_c0kw6c48_4fr0000gn/T/quarto-sessionae6c4759c2f1ea77/ed5f845b90d10d67/6dbf47c10b3ea69c/before-body.tex
[INFO] Loaded /var/folders/6j/_g2nn8gj1b5_c0kw6c48_4fr0000gn/T/quarto-sessionae6c4759c2f1ea77/ed5f845b90d10d67/6dbf47c10b3ea69c/toc.tex from /var/folders/6j/_g2nn8gj1b5_c0kw6c48_4fr0000gn/T/quarto-sessionae6c4759c2f1ea77/ed5f845b90d10d67/6dbf47c10b3ea69c/toc.tex
[INFO] Loaded /var/folders/6j/_g2nn8gj1b5_c0kw6c48_4fr0000gn/T/quarto-sessionae6c4759c2f1ea77/ed5f845b90d10d67/6dbf47c10b3ea69c/before-bib.tex from /var/folders/6j/_g2nn8gj1b5_c0kw6c48_4fr0000gn/T/quarto-sessionae6c4759c2f1ea77/ed5f845b90d10d67/6dbf47c10b3ea69c/before-bib.tex
[INFO] Loaded /var/folders/6j/_g2nn8gj1b5_c0kw6c48_4fr0000gn/T/quarto-sessionae6c4759c2f1ea77/ed5f845b90d10d67/6dbf47c10b3ea69c/biblio.tex from /var/folders/6j/_g2nn8gj1b5_c0kw6c48_4fr0000gn/T/quarto-sessionae6c4759c2f1ea77/ed5f845b90d10d67/6dbf47c10b3ea69c/biblio.tex
[INFO] Loaded /var/folders/6j/_g2nn8gj1b5_c0kw6c48_4fr0000gn/T/quarto-sessionae6c4759c2f1ea77/ed5f845b90d10d67/6dbf47c10b3ea69c/after-body.tex from /var/folders/6j/_g2nn8gj1b5_c0kw6c48_4fr0000gn/T/quarto-sessionae6c4759c2f1ea77/ed5f845b90d10d67/6dbf47c10b3ea69c/after-body.tex
[trace] Parsed [Plain [Str "DIV=11"]] at line 1
[trace] Parsed [Plain [Str "numbers=noendperiod"]] at line 1
[trace] Parsed [Plain [Str "scrartcl"]] at line 1
[trace] Parsed [Plain [Str "zh-Hans"]] at line 1
[trace] Parsed [Plain [Str "blue"]] at line 1
[trace] Parsed [Plain [Str "letter"]] at line 1
[trace] Parsed [Plain [Str "\33539\21494\20142"]] at line 1
[trace] Parsed [Plain [Str "\30446\24405"]] at line 1
[trace] Parsed [] at line 6
[trace] Parsed [Header 1 ("\33539\21494\20142",[],[]) [Str "\33539\21494\20 at line 9
[INFO] Running filter /Users/leo/Documents/Projects/TypeScript/quarto-cli/src/resources/filters/main.lua
[INFO] Completed filter /Users/leo/Documents/Projects/TypeScript/quarto-cli/src/resources/filters/main.lua in 33 ms
[execProcess] Success: true, code: 0

Rendering PDF
running xelatex - 1
[execProcess] tlmgr --version
[execProcess] Success: true, code: 0
[execProcess] tlmgr --version
[execProcess] Success: true, code: 0
[execProcess] tlmgr --version
[execProcess] Success: true, code: 0
[execProcess] xelatex -interaction=batchmode -halt-on-error zh.tex
[execProcess] Success: true, code: 0
  This is XeTeX, Version 3.141592653-2.6-0.999996 (TeX Live 2024) (preloaded format=xelatex)
   restricted \write18 enabled.
  entering extended mode

running xelatex - 2
[execProcess] xelatex -interaction=batchmode -halt-on-error zh.tex
[execProcess] Success: true, code: 0
  This is XeTeX, Version 3.141592653-2.6-0.999996 (TeX Live 2024) (preloaded format=xelatex)
   restricted \write18 enabled.
  entering extended mode

[NotebookContext]: Starting Cleanup
Output created: zh.pdf

Thus, would it be possible to make a quick fix with if (pkgMgr.autoInstall && missingHyphenationFile) to make Quarto v1.5 works with chinese language document.

leovan commented 1 month ago

By the way, the fix won't change the defaut behavior since latex-auto-install is true by default. But it can give us a way to fix this problem by setting latex-auto-install to false.

cderv commented 1 month ago

Thus, would it be possible to make a quick fix with if (pkgMgr.autoInstall && missingHyphenationFile) to make Quarto v1.5 works with chinese language document.

Yes we do agree on this. Definitely something we would need to improve, i.e. not blocking the pdf rendering because of a warning in the log file.

Though, I would still like to understand what needs to be done for this specific warning 

Package babel Info: Hyphen rules for 'chinese-hans' set to \l@nil

This triggers our "package to install" detection, but wrongly as we don't know which is the required package to solve it. If we can't find it, I wonder if we should make an exception to skip this warning completely.

Do you have insight on this ? or does the PDF produced anyway is useful and you stop there ?

Any insight would be helpful !

leovan commented 1 month ago

I guess it's a upstream problem. In babel package, for English:

https://github.com/latex3/babel/blob/dcd551427599879303b7a047b6fd3aef0a77120e/locale/en/babel-en.ini#L147-L149

hyphenrules = english

for Chinese:

https://github.com/latex3/babel/blob/dcd551427599879303b7a047b6fd3aef0a77120e/locale/zh/babel-zh.ini#L196-L198

hyphenrules =

hyphenrules is set to empty for all babel-zh-Hans-*.ini. I think this leads to the warning message:

Package babel Info: Hyphen rules for 'chinese-hans' set to \l@nil

Should we just ignore the warning message from babel package about hyphen rules?

leovan commented 1 month ago

Also there is indeed no hyphen for Chinese. For a Chinese word like 你好, line breaking is like:

xxx xxx xxx xxx 你
好 xxx xxx

No need for a hyphen like:

xxx xxx xxx xxx 你-
好 xxx xxx

Why hyphen-chinese package still exits. According to https://github.com/hyphenation/tex-hyphen/blob/master/hyph-utf8/tex/generic/hyph-utf8/patterns/tex/hyph-zh-latn-pinyin.tex, hyphen is only used for Chinese Pinyin https://en.wikipedia.org/wiki/Pinyin.

cderv commented 1 month ago

So you mean we should ignore the Hyphen rules warning for chinese-hans ?

Seems like the right think to do, if there is no hyphen expected anyway

leovan commented 1 month ago

Yes, but not only for Chinese. More precisely, I think we should check all languages with empty value for hyphenrules in bable package https://github.com/latex3/babel/tree/main/locale, at least for CJK languages.

cderv commented 1 month ago

Oh interesting. I don't know very well in detail how babel works and it seems you know better. Can you detail you idea ? Empty hyphenrules fields in the .ini means that no package should try to be installed and warning is expected in log file ? Is that what you mean ?

Thanks for your help !

leovan commented 1 month ago

Not quite sure, just guessed from its document and source code.

cderv commented 1 month ago

ok thanks. I'll search about this. If we do code some logic about this, it needs to be relevant to expected babel behavior

cscheid commented 1 month ago

As an immediate workaround, I think we should add a special-case rule to not attempt to install the bad hyphenation package in this specific case.

cderv commented 1 month ago

As an immediate workaround, I think we should add a special-case rule to not attempt to install the bad hyphenation package in this specific case.

💯 Agreed. This is exactly what I was going to do if not further information on babel package.

With adding support also for latex-auto-install: false for those installed package.

cderv commented 1 month ago

Just for reference, related notes I made in the past about hyphenation:

leovan commented 2 weeks ago

Hi, @cderv any updates with this issue? Would it be possible to make a quick fix with latex-auto-install: false? It truely influence many Chinese language projects.

mcanouil commented 2 weeks ago

Asking for updates and pressuring with things like "It truely influence many Chinese language projects." does not really help. This issue is not the only one opened. There are more than 1200 issues and only a few humans to investigate and fix the issues properly (while also adding new features). Note also that it's summer holidays.

The team is pretty much transparent and communicates/replies more than average projects (see the thread itself), so if you don't see any update, it means there is nothing to communicate about.

Thanks for your interest in this issue and Quarto but please be patient.

leovan commented 2 weeks ago

Here is a quick fix like:

diff --git a/src/command/render/latexmk/pdf.ts b/src/command/render/latexmk/pdf.ts
index 52062aa3f..ec9eb0554 100644
--- a/src/command/render/latexmk/pdf.ts
+++ b/src/command/render/latexmk/pdf.ts
@@ -179,7 +179,7 @@ async function initialCompileLatex(
       // See (https://github.com/rstudio/tinytex/commit/0f2007426f730a6ed9d45369233c1349a69ddd29)
       const logText = Deno.readTextFileSync(response.log);
       const missingHyphenationFile = findMissingHyphenationFiles(logText);
-      if (missingHyphenationFile) {
+      if (pkgMgr.autoInstall && missingHyphenationFile) {
         if (await pkgMgr.installPackages([missingHyphenationFile])) {
           // We installed hyphenation files, retry
           continue;

I'm not sure will you accept this as a fix. If do, I can make the PR.

MaxforCherubim commented 1 week ago

My problem seems like closely to this.

Rendering PDF
running xelatex - 1
  This is XeTeX, Version 3.141592653-2.6-0.999996 (TeX Live 2024) (preloaded format=xelatex)
   restricted \write18 enabled.
  entering extended mode

> 1 package to install
> installing hyphen-chinese (1 of 1)

And loading forever, no returning the trace of error. But it is hyphen-chinese not hyphen-chinese-hans, is there someing difference between both? So, i also render my another previous good project, found it is the same question.

Here i found a workaround:

  1. Disable the yaml lang: zh-CN
  2. add the ctex 
    # lang: zh-CN
format:
pdf:
include-in-header:
  text: |
    \usepackage{ctex}

    But using this method, I can not custom Chinese fonts.

mcanouil commented 1 week ago

@MaxforCherubim Your issue is unrelated. The issue here causes a complete crash. Please open a separate GitHub discussion with a fully reproducible example, see see https://quarto.org/bug-reports.html.

MaxforCherubim commented 1 week ago

OK. Let me try