google / comprehensive-rust

This is the Rust course used by the Android team at Google. It provides you the material to quickly teach Rust.
https://google.github.io/comprehensive-rust/
Apache License 2.0
28k stars 1.68k forks source link

Pandoc failure during local rendering #1911

Closed henrif75 closed 3 months ago

henrif75 commented 8 months ago

Following the build procedures listed in the README.md page results in pandoc failure:

mdbook serve
2024-03-10 09:56:02 [INFO] (mdbook::book): Book building has started
2024-03-10 09:56:02 [INFO] (mdbook::book): Running the exerciser backend
2024-03-10 09:56:02 [INFO] (mdbook::renderer): Invoking the "exerciser" renderer
2024-03-10 09:56:03 [INFO] (mdbook::book): Running the html backend
2024-03-10 09:56:04 [INFO] (mdbook::book): Running the pandoc backend
2024-03-10 09:56:04 [INFO] (mdbook::renderer): Invoking the "pandoc" renderer
[2024-03-10T16:56:07Z INFO  mdbook_pandoc] Processing redirects in [output.html.redirect]
[2024-03-10T16:56:07Z WARN  mdbook_pandoc::preprocess] Failed to resolve redirect: why-rust/an-example-in-c.html => ../hello-world/example.html: Unable to normalize redirect destination: Unable to canonicalize path: /Users/henrif/Documents/comprehensive-rust/book/pandoc/pdf/src/why-rust/../hello-world/example.html: No such file or directory (os error 2)
[2024-03-10T16:56:07Z WARN  mdbook_pandoc::preprocess] Failed to resolve redirect: exercises/day-2/soluções-tarde.html => solutions-afternoon.html: Unable to normalize redirect destination: Unable to canonicalize path: /Users/henrif/Documents/comprehensive-rust/book/pandoc/pdf/src/exercises/day-2/solutions-afternoon.html: No such file or directory (os error 2)
[2024-03-10T16:56:07Z WARN  mdbook_pandoc::preprocess] Failed to resolve redirect: exercises/day-3/soluções-tarde.html => solutions-afternoon.html: Unable to normalize redirect destination: Unable to canonicalize path: /Users/henrif/Documents/comprehensive-rust/book/pandoc/pdf/src/exercises/day-3/solutions-afternoon.html: No such file or directory (os error 2)
[2024-03-10T16:56:07Z WARN  mdbook_pandoc::preprocess] Failed to resolve redirect: testing/mockall.html => ../android/testing/mockall.html: Unable to normalize redirect destination: Unable to canonicalize path: /Users/henrif/Documents/comprehensive-rust/book/pandoc/pdf/src/testing/../android/testing/mockall.html: No such file or directory (os error 2)
[2024-03-10T16:56:07Z WARN  mdbook_pandoc::preprocess] Failed to resolve redirect: traits/iterator.html => ../iterators/iterators.html: Unable to normalize redirect destination: Unable to canonicalize path: /Users/henrif/Documents/comprehensive-rust/book/pandoc/pdf/src/traits/../iterators/iterators.html: No such file or directory (os error 2)
[2024-03-10T16:56:07Z WARN  mdbook_pandoc::preprocess] Failed to resolve redirect: exercises/day-1/soluções-tarde.html => solutions-afternoon.html: Unable to normalize redirect destination: Unable to canonicalize path: /Users/henrif/Documents/comprehensive-rust/book/pandoc/pdf/src/exercises/day-1/solutions-afternoon.html: No such file or directory (os error 2)
[2024-03-10T16:56:07Z WARN  mdbook_pandoc::preprocess] Failed to resolve redirect: basic-syntax/static-and-const.html => ../unsafe-rust/static-and-const.html: Unable to normalize redirect destination: Unable to canonicalize path: /Users/henrif/Documents/comprehensive-rust/book/pandoc/pdf/src/basic-syntax/../unsafe-rust/static-and-const.html: No such file or directory (os error 2)
[2024-03-10T16:56:07Z WARN  mdbook_pandoc::preprocess] Unable to normalize link 'comprehensive-rust.pdf' in chapter 'Welcome to Comprehensive Rust 🦀': Unable to canonicalize path: /Users/henrif/Documents/comprehensive-rust/src/comprehensive-rust.pdf: No such file or directory (os error 2)
[2024-03-10T16:56:07Z WARN  mdbook_pandoc::preprocess] Unable to normalize link '../../comprehensive-rust-exercises.zip' in chapter 'Compass': Unable to canonicalize path: /Users/henrif/Documents/comprehensive-rust/src/exercises/bare-metal/../../comprehensive-rust-exercises.zip: No such file or directory (os error 2)
[2024-03-10T16:56:07Z WARN  mdbook_pandoc::preprocess] Unable to normalize link '../../comprehensive-rust-exercises.zip' in chapter 'RTC Driver': Unable to canonicalize path: /Users/henrif/Documents/comprehensive-rust/src/exercises/bare-metal/../../comprehensive-rust-exercises.zip: No such file or directory (os error 2)
[2024-03-10T16:56:07Z WARN  mdbook_pandoc::preprocess] Unable to normalize link '../../borrowing/interior-mutability' in chapter 'Mutex': Unable to canonicalize path: /Users/henrif/Documents/comprehensive-rust/src/concurrency/shared_state/../../borrowing/interior-mutability: No such file or directory (os error 2)
[WARNING] Could not convert image /private/var/folders/nk/38dlc4z527143grzs9b12cnw0034vl/T/tex2pdf.-6bf89262faae817a/d907026615712cd13ce7e4e5023eb47711d8122f.svg: check that rsvg-convert is in path.
  rsvg-convert: createProcess: posix_spawnp: does not exist (No such file or directory)
[WARNING] Could not convert image /private/var/folders/nk/38dlc4z527143grzs9b12cnw0034vl/T/tex2pdf.-6bf89262faae817a/5180f1968cb395680acc12835345595cdedd9550.svgz: check that rsvg-convert is in path.
  rsvg-convert: createProcess: posix_spawnp: does not exist (No such file or directory)
[WARNING] Could not convert image /private/var/folders/nk/38dlc4z527143grzs9b12cnw0034vl/T/tex2pdf.-6bf89262faae817a/aedd0ef191cf2e1f37d81b92916a869810082d7e.svgz: check that rsvg-convert is in path.
  rsvg-convert: createProcess: posix_spawnp: does not exist (No such file or directory)
lualatex not found. Please select a different --pdf-engine or install lualatex
pandoc exited unsuccessfully
2024-03-10 09:56:10 [ERROR] (mdbook::renderer): Renderer exited with non-zero return code.
2024-03-10 09:56:10 [ERROR] (mdbook::utils): Error: Rendering failed
2024-03-10 09:56:10 [ERROR] (mdbook::utils):    Caused By: The "pandoc" renderer failed
djmitche commented 8 months ago

It looks like the issue is that rsvg-convert is not installed? I suspect installing that will fix the immediate issue, and perhaps an README update or the like is required to help the next person who encounters this.

I wonder if pandoc just has too many dependencies to expect folks to build locally. Maybe we should be suggesting building the PDF using a docker invocation of some sort?

henrif75 commented 8 months ago

I had to install librsvg and lualatex on Mac, but I'm getting another error now:

2024-03-11 14:27:54 [INFO] (mdbook::book): Book building has started
2024-03-11 14:27:55 [INFO] (mdbook::book): Running the exerciser backend
2024-03-11 14:27:55 [INFO] (mdbook::renderer): Invoking the "exerciser" renderer
2024-03-11 14:27:55 [INFO] (mdbook::book): Running the html backend
2024-03-11 14:27:56 [INFO] (mdbook::book): Running the pandoc backend
2024-03-11 14:27:56 [INFO] (mdbook::renderer): Invoking the "pandoc" renderer
[2024-03-11T21:27:57Z INFO  mdbook_pandoc] Processing redirects in [output.html.redirect]
[2024-03-11T21:27:57Z WARN  mdbook_pandoc::preprocess] Failed to resolve redirect: exercises/day-1/soluções-tarde.html => solutions-afternoon.html: Unable to normalize redirect destination: Unable to canonicalize path: /Users/henrif/Documents/comprehensive-rust/book/pandoc/pdf/src/exercises/day-1/solutions-afternoon.html: No such file or directory (os error 2)
[2024-03-11T21:27:57Z WARN  mdbook_pandoc::preprocess] Failed to resolve redirect: exercises/day-3/soluções-tarde.html => solutions-afternoon.html: Unable to normalize redirect destination: Unable to canonicalize path: /Users/henrif/Documents/comprehensive-rust/book/pandoc/pdf/src/exercises/day-3/solutions-afternoon.html: No such file or directory (os error 2)
[2024-03-11T21:27:57Z WARN  mdbook_pandoc::preprocess] Failed to resolve redirect: why-rust/an-example-in-c.html => ../hello-world/example.html: Unable to normalize redirect destination: Unable to canonicalize path: /Users/henrif/Documents/comprehensive-rust/book/pandoc/pdf/src/why-rust/../hello-world/example.html: No such file or directory (os error 2)
[2024-03-11T21:27:57Z WARN  mdbook_pandoc::preprocess] Failed to resolve redirect: testing/mockall.html => ../android/testing/mockall.html: Unable to normalize redirect destination: Unable to canonicalize path: /Users/henrif/Documents/comprehensive-rust/book/pandoc/pdf/src/testing/../android/testing/mockall.html: No such file or directory (os error 2)
[2024-03-11T21:27:57Z WARN  mdbook_pandoc::preprocess] Failed to resolve redirect: basic-syntax/static-and-const.html => ../unsafe-rust/static-and-const.html: Unable to normalize redirect destination: Unable to canonicalize path: /Users/henrif/Documents/comprehensive-rust/book/pandoc/pdf/src/basic-syntax/../unsafe-rust/static-and-const.html: No such file or directory (os error 2)
[2024-03-11T21:27:57Z WARN  mdbook_pandoc::preprocess] Failed to resolve redirect: traits/iterator.html => ../iterators/iterators.html: Unable to normalize redirect destination: Unable to canonicalize path: /Users/henrif/Documents/comprehensive-rust/book/pandoc/pdf/src/traits/../iterators/iterators.html: No such file or directory (os error 2)
[2024-03-11T21:27:57Z WARN  mdbook_pandoc::preprocess] Failed to resolve redirect: exercises/day-2/soluções-tarde.html => solutions-afternoon.html: Unable to normalize redirect destination: Unable to canonicalize path: /Users/henrif/Documents/comprehensive-rust/book/pandoc/pdf/src/exercises/day-2/solutions-afternoon.html: No such file or directory (os error 2)
[2024-03-11T21:28:00Z WARN  mdbook_pandoc::preprocess] Unable to normalize link 'comprehensive-rust.pdf' in chapter 'Welcome to Comprehensive Rust 🦀': Unable to canonicalize path: /Users/henrif/Documents/comprehensive-rust/src/comprehensive-rust.pdf: No such file or directory (os error 2)
[2024-03-11T21:28:00Z WARN  mdbook_pandoc::preprocess] Unable to normalize link '../../comprehensive-rust-exercises.zip' in chapter 'Compass': Unable to canonicalize path: /Users/henrif/Documents/comprehensive-rust/src/exercises/bare-metal/../../comprehensive-rust-exercises.zip: No such file or directory (os error 2)
[2024-03-11T21:28:00Z WARN  mdbook_pandoc::preprocess] Unable to normalize link '../../comprehensive-rust-exercises.zip' in chapter 'RTC Driver': Unable to canonicalize path: /Users/henrif/Documents/comprehensive-rust/src/exercises/bare-metal/../../comprehensive-rust-exercises.zip: No such file or directory (os error 2)
[2024-03-11T21:28:00Z WARN  mdbook_pandoc::preprocess] Unable to normalize link '../../borrowing/interior-mutability' in chapter 'Mutex': Unable to canonicalize path: /Users/henrif/Documents/comprehensive-rust/src/concurrency/shared_state/../../borrowing/interior-mutability: No such file or directory (os error 2)
    Error producing PDF.
!  ==> Fatal error occurred, no output PDF file produced!

pandoc exited unsuccessfully
2024-03-11 14:28:27 [ERROR] (mdbook::renderer): Renderer exited with non-zero return code.
2024-03-11 14:28:27 [ERROR] (mdbook::utils): Error: Rendering failed
2024-03-11 14:28:27 [ERROR] (mdbook::utils):    Caused By: The "pandoc" renderer failed

It worked once, then no more. Could we have some sort of config to turn off the PDF generation?

mgeisler commented 8 months ago

@max-heller, do you have experience with building the course on Mac?

@henrif75, you should be able to turn this off by removing the mdbook-pandoc binary you have installed. I get

2024-03-24 21:04:55 [WARN] (mdbook::renderer): The command `mdbook-pandoc` for backend `pandoc` was not found, but was marked as optional.

on my machine because I haven't installed it here.

max-heller commented 7 months ago

@max-heller, do you have experience with building the course on Mac?

Yes, I primarily develop on a Mac.

@henrif75 could you try enabling verbose logging for Pandoc in book.toml and see what it spits out?

[output.pandoc.profile.pdf]
output-file = "comprehensive-rust.pdf"
pdf-engine = "lualatex"
verbosity = "INFO" # ADD THIS LINE
henrif75 commented 7 months ago

Here is the output: mdbook.log.gz

max-heller commented 7 months ago

Here is the output: mdbook.log.gz

Do you have Noto fonts installed? The end of the log suggests missing fonts might be the fatal error:

  luaotfload | db : Reload initiated (formats: otf,ttf,ttc); reason: Font "NotoSansMonoCJKSC" not found.
  luaotfload | resolve : sequence of 3 lookups yielded nothing appropriate....texmf-dist/tex/luatex/luaotfload/luaotfload-fallback.lua:50: attempt to inde
  x a nil value (local 'f').
  <to be read again> 
  relax 
  l.154 \begin{document}

   515 words of node memory still in use:
     5 hlist, 1 vlist, 2 rule, 1 dir, 4 kern, 1 glyph, 7 attribute, 60 glue_spec,
   7 attribute_list, 3 if_stack, 1 write, 1 user_defined, 1 pdf_colorstack nodes
     avail lists: 1:1,2:10,3:3,4:1,5:7,7:2,9:4
  !  ==> Fatal error occurred, no output PDF file produced!
  Transcript written on input.log.
Error producing PDF.
!  ==> Fatal error occurred, no output PDF file produced!

pandoc exited unsuccessfully
2024-04-07 17:51:17 [ERROR] (mdbook::renderer): Renderer exited with non-zero return code.
2024-04-07 17:51:17 [ERROR] (mdbook::utils): Error: Rendering failed
2024-04-07 17:51:17 [ERROR] (mdbook::utils):    Caused By: The "pandoc" renderer failed

I have the following installed through homebrew, although I'm not sure they're all needed:

$ brew list --casks | grep noto
font-noto-color-emoji
font-noto-emoji
font-noto-sans
font-noto-sans-cjk
font-noto-sans-math
font-noto-sans-symbols
font-noto-serif
font-noto-serif-cjk
font-noto-serif-cjk-tc
henrif75 commented 7 months ago

It worked when I installed the fonts, but I couldn't find a casket so I manually installed them. Would you have the brew command to fetch these fonts? I'll update the documentation.

max-heller commented 7 months ago

It worked when I installed the fonts, but I couldn't find a casket so I manually installed them. Would you have the brew command to fetch these fonts? I'll update the documentation.

brew tap homebrew/cask-fonts (see here)

kashopi commented 6 months ago

In my Silicon laptop I fixed it by using "tectonic" as a PDF renderer (changing it in the book.toml file) and installing the Noto fonts

max-heller commented 6 months ago

In my Silicon laptop I fixed it by using "tectonic" as a PDF renderer (changing it in the book.toml file) and installing the Noto fonts

tectonic simplifies a lot of dependency wrangling, but If I'm remembering correctly it doesn't support all of the font shenanigans necessary to render multilingual books, so unfortunately comprehensive rust probably can't switch over to it.

https://github.com/google/mdbook-i18n-helpers/issues/200 is trying to figure out a way to skip running optional = true renderers when they aren't needed, which will prevent failures like these

letesdev commented 5 months ago

Hi, I hope you can help me 😄

I am trying to build the book in order to contribute with translating to spanish, but I am facing some errors... I am using a Windows 10 machine and I followed your README.md, CONTRIBUTING.md and TRANSLATIONS.md guides, but still face some errors.

Here are all my steps :

  1. I installed Rust by following the instructions on https://rustup.rs/.
  2. I cloned this repository
  3. I installed the necessary tools (mdbook, mdbook-svgbob, mdbook-i18n-helpers, mdbook-exerciser, and mdbook-course) using cargo.
  4. I installed pandoc from Pandoc
  5. I installed mdbook-pandoc using cargo, which is not mentioned in the README.md guide.
  6. I installed the Noto Fonts from Google Fonts.
  7. I installed MikTex to get lualatex from MiKTeX.

Now I am able to build the english version of book with mdbook build, but I can't build the spanish version with MDBOOK_BOOK__LANGUAGE=es mdbook build -d book/es. Here's the output: mdbook_build_es_output1.txt

pandoc: C:\Users\s023793\letesdev\comprehensive-rust\book\es\pandoc\pdf\comprehensive-rust.pdf: withBinaryFile: permission denied (Permission denied)
pandoc exited unsuccessfully
2024-06-04 14:50:33 [ERROR] (mdbook::renderer): Renderer exited with non-zero return code.
2024-06-04 14:50:33 [ERROR] (mdbook::utils): Error: Rendering failed
2024-06-04 14:50:33 [ERROR] (mdbook::utils):    Caused By: The "pandoc" renderer failed
max-heller commented 5 months ago

5. I installed mdbook-pandoc using cargo, which is not mentioned in the README.md guide.

mdbook-pandoc is marked as optional, so if you uninstall it you should be able to build the book without it. https://github.com/max-heller/mdbook-pandoc/pull/93 will soon provide a way to disable mdbook-pandoc outside of CI to avoid having to install its dependencies locally.

pandoc: C:\Users\s023793\letesdev\comprehensive-rust\book\es\pandoc\pdf\comprehensive-rust.pdf: withBinaryFile: permission denied (Permission denied)
pandoc exited unsuccessfully

This is strange, are there restrictive permissions on any of the parent directories of this path?

letesdev commented 5 months ago

Hi max-heller, thanks for the answer 😄

max-heller commented 5 months ago
2024-06-04 11:35:24 [WARN] (mdbook::renderer): The command `mdbook-pandoc` for backend `pandoc` was not found, but was marked as optional.

This is a warning, not an error, and shouldn't impact the rest of the build.

If I am able to build the english version, shouldn't I be able to build the spanish one ?

You should be able to, yes. I'll look into it some more

letesdev commented 5 months ago

This is a warning, not an error, and shouldn't impact the rest of the build.

Well, as you can see in the output file (having stderr and stout outputs), the build stops there without any other message... Let me know if you need more details 😄

If I am able to build the english version, shouldn't I be able to build the spanish one ?

You should be able to, yes. I'll look into it some more

Great, thank you so much! 😄

max-heller commented 5 months ago

Well, as you can see in the output file (having stderr and stout outputs), the build stops there without any other message... Let me know if you need more details 😄

At that point, the build has completed successfully. mdbook-pandoc is the last renderer that gets run, so if it's skipped, nothing else is expected. You can verify this by e.g. commenting all of the pandoc configuration in book.toml.

max-heller commented 5 months ago

If I am able to build the english version, shouldn't I be able to build the spanish one ?

You should be able to, yes. I'll look into it some more

Great, thank you so much! 😄

Seems like the error can occur for a variety of reasons (see e.g. here). Do you happen to have the Spanish PDF open while building? An antivirus that might be blocking Pandoc? A network drive/OneDrive setup? Have you tried wiping out the book directory and trying again?

letesdev commented 5 months ago

I am finally able to build the english and the spanish book! Thank you so much @max-heller ! I tried commenting all of the pandoc configuration in the book.toml file as you mentioned, and, just in case, I wiped out the book/ directory and closed OneDrive...

However, I found many of the paragraphs in english in the spanish version are already translated... for instance, after formatting the po/es.po file with dprint fmt po/es.po command, I am able to modify the paragraph in the index.html (I added several 's' to 'Basándonos' just for showing) but not the list :

image

Do you have a clue in this, by the way ?

max-heller commented 5 months ago

I am finally able to build the english and the spanish book! Thank you so much @max-heller ! I tried commenting all of the pandoc configuration in the book.toml file as you mentioned, and, just in case, I wiped out the book/ directory and closed OneDrive...

Happy to hear it! Just curious, if you reenable the pandoc configuration in book.toml, do you get the same error as before?

However, I found many of the paragraphs in english in the spanish version are already translated... for instance, after formatting the po/es.po file with dprint fmt po/es.po command, I am able to modify the paragraph in the index.html (I added several 's' to 'Basándonos' just for showing) but not the list : Do you have a clue in this, by the way ?

I'm not familiar with the translation infrastructure, this seems like a good question for a separate issue to keep this one on topic

letesdev commented 5 months ago

Just curious, if you reenable the pandoc configuration in book.toml, do you get the same error as before? I tried several builds:

max-heller commented 5 months ago

Thanks for testing! This is a specific issue that should be easy to fix, I've opened https://github.com/max-heller/mdbook-pandoc/issues/96 to track it

mgeisler commented 3 months ago

I think we can close this now": We no longer run mdbook-pandoc by default and so people should not see local errors unless they opt-in.