quarto-dev / quarto-cli

Open-source scientific and technical publishing system built on Pandoc.
https://quarto.org
Other
3.95k stars 326 forks source link

Various mermaid rendering issues #5416

Closed rcannood closed 4 months ago

rcannood commented 1 year ago

Bug description

Throughout the different releases, there have been various rendering issues with Mermaid in Quarto. I understand that there's a lot of moving parts to keep track of, so no judgement there.

I just noticed a new one with Quarto 1.3, so I thought I'd do a more thorough investigation. I created this repo which gets rendered to this site, which should be a mirror of the mermaidjs syntax documentation.

Since Quarto 1.3 uses mermaid 9.1.1, I went back to the earliest version of mermaid js which had the syntax examples in the docs, which happened to be version 9.3.0. I used this script to translate the md files in the mermaid repository to qmd files in my repository.

Issues in Quarto v1.3

(See comment below for issues in Quarto v1.4, since some issues have already been fixed by upgrading to mermaid 10)

Update: issues that have already been solved in Quarto v1.4 have been ticked!

Flow charts:

Screenshot from 2023-05-04 14-20-00

Sequence diagrams:

Class diagrams:

This was all rendered with quarto 1.3.340.

Output of quarto check [✓] Checking versions of quarto binary dependencies... Pandoc version 3.1.1: OK Dart Sass version 1.55.0: OK [✓] Checking versions of quarto dependencies......OK [✓] Checking Quarto installation......OK Version: 1.3.340 Path: /home/rcannood/opt/quarto-1.3.340/bin [✓] Checking basic markdown render....OK [✓] Checking Python 3 installation....OK Version: 3.11.3 Path: /usr/bin/python3 Jupyter: 5.2.0 Kernels: python3, julia-1.7, ir (/) Checking Jupyter engine render....0.00s - Debugger warning: It seems that frozen modules are being used, which may 0.00s - make the debugger miss breakpoints. Please pass -Xfrozen_modules=off 0.00s - to python to disable frozen modules. 0.00s - Note: Debugging will proceed. Set PYDEVD_DISABLE_FILE_VALIDATION=1 to disable this validation. 0.00s - Debugger warning: It seems that frozen modules are being used, which may 0.00s - make the debugger miss breakpoints. Please pass -Xfrozen_modules=off 0.00s - to python to disable frozen modules. 0.00s - Note: Debugging will proceed. Set PYDEVD_DISABLE_FILE_VALIDATION=1 to disable this validation. [✓] Checking Jupyter engine render....OK [✓] Checking R installation...........OK Version: 4.2.3 Path: /usr/lib64/R LibPaths: - /home/rcannood/R/x86_64-redhat-linux-gnu-library/4.2 - /usr/local/lib/R/library - /usr/lib64/R/library - /usr/share/R/library knitr: 1.42 rmarkdown: 2.20 [✓] Checking Knitr engine render......OK

Checklist

rcannood commented 1 year ago

Additional questions:

mcanouil commented 1 year ago

Thanks for this pretty comprehensive report, it will help a lot.

No worries, if needed, it's easy for maintainers to turn this list items into list of issues, which would turn this one into an "epic".

cscheid commented 1 year ago

Thanks for the report, @rcannood . I hate to do this to you, but could you please check these issues on the latest 1.4 version? We just upgraded to mermaid 10 and I expect a number of these problems as well. The issue is that I don't know what you expect the output to be (since I'm not a mermaid user), so I could use help there: https://github.com/quarto-dev/quarto-cli/releases/tag/v1.4.36

rcannood commented 1 year ago

I added a CI script, regenerated the /syntax/*.qmd files using the mermaid v10.1.0 release, and bumped the quarto version used by the CI to 1.4.36, resulting in this website render.

The issue is that I don't know what you expect the output to be

That's why I copied the diagrams that are already part of mermaid.js.org. That way, you can compare the output of the same figures rendered using quarto 1.4.36 and the official mermaid.js documentation side by side. For each of the items listed above and below, I provided a link to the section rendered by quarto where the issue occurs, and the expected output as rendered at mermaid.js.org.

Quarto 1.4.34

Some problems seem to have been fixed, others still persist:

Flow charts:

Sequence diagrams:

State diagram:

rcannood commented 1 year ago

TBH most of the remaining issues are nice-to-haves, except for the backgrounds of subgraphs in flow charts (and the concurrency view in the state diagrams) are quite.

If the background issue could be fixed in quarto 1.3, that'd be fantastic, because it is something that used to work in quarto 1.2 and earlier :relaxed:

cscheid commented 1 year ago

If the background issue could be fixed in quarto 1.3, that'd be fantastic

Is there a specific reason you'd prefer to use quarto 1.3 rather than the early 1.4 releases? In terms of backports, we tend to prefer bugfixes that solve issues in both 1.4 and 1.3. In this case, the 1.3->1.4 transition means I would have to duplicate my work (mermaid 9 and 10 are not really compatible with each other).

rcannood commented 1 year ago

The reason why it'd be fantastic if the fix for the subgraph background fills could still be included in v1.3 is because it used to work fine with Quarto v1.2. We just updated one of our sites (openproblems-bio/website) to Quarto v1.3 to get access to some other bug fixes, but it seems some diagrams were broken in the process.

We can always add a workaround to our CSS, but I can imagine other people might also encounter the same issue when upgrading from v1.2 to v1.3.

At any rate, I'm already happy if we do manage to find a bug fix, even if it's included in v1.4 only.

jmcvw commented 1 year ago

Ok if I add another bug to this list? I can start a new issue if you prefer, but this seems like a nice catch-all.

I am currently using Quarto 1.4.58, but I installed back to 1.4.37 to see when this first occurs - there was no problem with 1.3.

If I try to render to png usng the YAML

---
title: "Untitled"
format:
  html:
    mermaid-format: png
---

```{mermaid}
flowchart LR
A --> B

 I get the error:

ERROR: TypeError: Relative import path "puppeteer/mod.ts" not prefixed with / or ./ or ../

TypeError: Relative import path "puppeteer/mod.ts" not prefixed with / or ./ or ../ at async getPuppeteer (file:///Applications/quarto/bin/quarto.js:35208:24) at async fetcher (file:///Applications/quarto/bin/quarto.js:35337:22) at async getBrowserExecutablePath (file:///Applications/quarto/bin/quarto.js:35341:28) at async criClient (file:///Applications/quarto/bin/quarto.js:35727:28) at async file:///Applications/quarto/bin/quarto.js:35710:22 at async Semaphore.runExclusive (file:///Applications/quarto/bin/quarto.js:35394:29) at async Object.createPngsFromHtml (file:///Applications/quarto/bin/quarto.js:35891:44) at async makePng (file:///Applications/quarto/bin/quarto.js:63036:72) at async Object.cell (file:///Applications/quarto/bin/quarto.js:63101:20) at async Promise.all (index 0)



If you remove the png line, it renders, but all the base text is blue, whereas it used to be black - don't know if that is intended, but it is different than I expected

<img width="108" alt="image" src="https://github.com/quarto-dev/quarto-cli/assets/43386915/9987a8b6-638b-461f-91b4-0f1b6cff3436">

************************

**`quarto check` output**

[✓] Checking versions of quarto binary dependencies...
      Pandoc version 3.1.2: OK
      Dart Sass version 1.55.0: OK
      NOTE: Deno version 1.33.2 is too old. Please upgrade to 1.33.1 or later.
[✓] Checking versions of quarto dependencies......OK
[✓] Checking Quarto installation......OK
      Version: 1.4.58
      Path: /Applications/quarto/bin

[✓] Checking basic markdown render....OK

[✓] Checking Python 3 installation....OK
      Version: 3.9.13 (Conda)
      Path: /usr/local/anaconda3/bin/python
      Jupyter: 4.11.1
      Kernels: python3

[✓] Checking Jupyter engine render....OK

[✓] Checking R installation...........OK
      Version: 4.3.0
      Path: /Library/Frameworks/R.framework/Resources
      LibPaths:
        - /Library/Frameworks/R.framework/Versions/4.3-x86_64/Resources/library
      knitr: 1.42
      rmarkdown: 2.21

[✓] Checking Knitr engine render......OK
davidwilby commented 1 year ago

At least one of these issues was not present 3 months ago, specifically:

Fontawesome icons are not rendered.

I have a revealjs slide deck rendered by quarto (https://github.com/davidwilby/ResearchSoftwareMethods) in which FontAwesome icons were rendered correctly in a mermaidjs flowchart.

The slides were rendered on GitHub actions using quarto but unfortunately I don't know exactly which version as the runner logs have expired, but this shows that this feature was working correctly in a previous version of quarto.

cscheid commented 1 year ago

At least one of these issues was not present 3 months ago, specifically:

We are aware, that's why this issue is open.

The slides were rendered on GitHub actions using quarto but unfortunately I don't know exactly which version

You can inspect the generated HTML output's meta tags for that information.

ForBo7 commented 1 year ago

Is there any timeline when we could expect Mermaid rendering to be fixed? My site uses Quarto 1.4.294 — in hindsight, I probably shouldn't have moved to 1.4 yet as it's prerelease heh.

image
erikaduan commented 1 year ago

Rendering of mermaid diagrams from Quarto notebooks into the gfm (GitHub flavoured markdown) output is currently producing errors.

For example, the following mermaid code chunk renders nicely when I click render from the RStudio IDE.

flowchart LR

A("Clean and 
analyse data") --> B("Draft report")  

B --> C("Change report parameters,
update data or
fix mistakes")

C -. Rerun code .-> A
image

However, when I navigate to my github markdown document, the following error message is displayed instead of a diagram.

image

It is possible that GitHub is already aware of this issue as their help page contains the statement:

Note: You may observe errors if you run a third-party Mermaid plugin when using Mermaid syntax on GitHub.

My Quarto notebook global code chunk options are also listed below.

---
execute:
  echo: true
  output: true
  message: false
  warning: false

format:
  gfm:
    toc: true
---
rcannood commented 1 year ago

@erikaduan this is a duplicate of #5319 and is already fixed in quarto 1.4.

cscheid commented 1 year ago

This does indeed look to be fixed. When using prerelease versions, please make sure to update to the latest prerelease;

Version: 1.4.58

We're on https://github.com/quarto-dev/quarto-cli/releases/tag/v1.4.315 right now.

ForBo7 commented 1 year ago

I installed 1.4.315, previewed my site locally, and I'm still having the same rendering issues.

image
flowchart LR
  subgraph A [Layer 1]
    direction LR
    id1[Affine Function] --> id2[ReLU]
  end

  subgraph B [Layer 2]
    direction LR
    id2 --> id3[Affine Function]
  end

  subgraph C [Loss Function]
    direction LR
    id3 --> id4[MSE]
  end
mcanouil commented 1 year ago

There is confusing post here.

https://github.com/quarto-dev/quarto-cli/issues/5319 fixed an issue regarding GFM, not the background issue referenced here.

@ForBo7 Showing only mermaid code is not helpful as it does not show format, etc.

rcannood commented 1 year ago

Apologies for any confusion caused. I meant the issue mentioned by @erikaduan was already solved, but it's indeed not related to the issue discussed in the original post. I'll update the repository using the latest development version of quarto to verify whether this issue still persists.

If the issue still persists, should I create a new issue just to be able to make this thread less confusing?

On Sat, 12 Aug 2023, 10:55 Mickaël Canouil, @.***> wrote:

There is confusing post here.

5319 https://github.com/quarto-dev/quarto-cli/issues/5319 fixed an

issue regarding GFM, not the background issue referenced here.

@ForBo7 https://github.com/ForBo7 Showing only mermaid code is not helpful as it does not show format, etc.

— Reply to this email directly, view it on GitHub https://github.com/quarto-dev/quarto-cli/issues/5416#issuecomment-1675799807, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAEHFKTT5MRJPH6RL35X32TXU5AHVANCNFSM6AAAAAAXVWG4RE . You are receiving this because you were mentioned.Message ID: @.***>

mcanouil commented 1 year ago

@rcannood We'll just hide the comments as "off topic".

ForBo7 commented 1 year ago

There is confusing post here.

5319 fixed an issue regarding GFM, not the background issue referenced here.

@ForBo7 Showing only mermaid code is not helpful as it does not show format, etc.

I'm not using any formatting or additional config/theming files or anything of the sort. When I write my Mermaid Diagram, Quarto renders it like that, instead of correctly rendering it like the following.

flowchart LR
  subgraph A [Layer 1]
    direction LR
    id1[Affine Function] --> id2[ReLU]
  end

  subgraph B [Layer 2]
    direction LR
    id2 --> id3[Affine Function]
  end

  subgraph C [Loss Function]
    direction LR
    id3 --> id4[MSE]
  end

Let me know if you would like any other piece of information in particular that would help narrow down the issue.

rcannood commented 1 year ago

Just to verify, I updated the test website containing all mermaidjs documentation to quarto 1.4.322, and I can still reproduce each of the previously mentioned issues.

I hope that each of the issues become apparent by clicking 'Example' and comparing to 'Expected'. Please let me know if something is not clear.

Quarto 1.4.322

Flow charts:

Sequence diagrams:

State diagram:

catanzaromj commented 7 months ago

In case anyone comes across this trying to change the dark background in a subgraph, here's a fix.

Modify the variable mermaid-fg-color--lightest in your CSS/SCSS and it will update the subgraph background. For example, to set it to white:

/*-- scss:defaults --*/
//other stuff
$mermaid-fg-color--lightest:#FFFFFF;

In case it helps anyone debug this, this line sets the subgraph background color to mermaid-fg-color--lightest, since subgraphs are cluster elements. This defaults to $mermaid-fg-color--lightest: lighten($body-color, 20%); according to the docs. Should it default to a background color instead of a foreground color?

cscheid commented 7 months ago

@catanzaromj We'd have to test that with a number of mermaid charts to be sure, but I think you're right and it was a typo when I wrote that. Can you open a new issue? Thanks.

NiccoMlt commented 7 months ago

Hi, not sure if this is the right issue, but before opening a new one I try here.

Last week I noticed that in a Reaveal.js presentations written in Quarto, the Mermaid diagrams rendered correctly while on browser, but when generating a PDF with Decktape the boxes had a wrong size and the text overlapped with the border:

immagine

immagine

Initially I though that the problem was on the Decktape side, and I opened astefanutti/decktape#328, but after that I discovered that the same problem happens with RStudio:

Screenshot 2024-04-15 102605

Screenshot 2024-04-15 102533

I'm using Quarto 1.4.553 with RStudio 26.2.4

See https://github.com/NiccoMlt/quarto-mermaid-decktape-issue-demo for a working example of the problem

cscheid commented 7 months ago

I imagine that if you use decktape and revealjs outside of quarto you'll see the same problem (so this isn't something Quarto can do. And while we try not to break decktape, it's not something we officially support either.)

The problem is likely that decktape is confusing mermaid's window size determination code. You'll need to generate the mermaid graphics at rendering time: https://quarto.org/docs/authoring/diagrams.html#mermaid-formats

NiccoMlt commented 7 months ago

Yes, initially I assumed it was something decktape-related because it worked like a charm in any browser, but when I also saw the problem on RStudio's own preview I thought it might be something related to the code that was output by Quarto. Still, it makes sense that it's not the case and RStudio preview panel just messes up with the window size in a similar way decktape does.

Thank you for your suggestion by the way, I'll give it a try tonight and see if it can serve as a workaround

NiccoMlt commented 7 months ago

I confirm that using mermaid-format: png works as a workaround, thanks 👍🏻

Sadly, using SVG format (instead of PNG) still presents some problems:

Schermata del 2024-04-15 23-15-41

in this case, the problem can be experienced even with browser and not only inside RStudio o with decktape:

Screenshot 2024-04-15 at 23-17-01 Test

It seems to be rendered too widely, because adding a magic comment to force the figure to use the 100% of the available width (%%| fig-width: 100%) fixes the problem for the SVGs

jorishijmans commented 5 months ago

While rendering the mermaid diagram (i.e. timeline) in html is possible, previewing results in error: "undefined"

image

Tested with Quarto v1.4.554 in VS Code on Windows (see ### context for more info)

timeline
    title MermaidChart 2023 Timeline
    section 2023 Q1 <br> Release Personal Tier
       Bullet 1 : sub-point 1a : sub-point 1b
                : sub-point 1c
       Bullet 2 : sub-point 2a : sub-point 2b
    section 2023 Q2 <br> Release XYZ Tier
       Bullet 3 : sub-point <br> 3a : sub-point 3b
                : sub-point 3c
            Bullet 4 : sub-point 4a : sub-point 4b

Rendering in html produces:

image

Context

Quarto v1.4.554 on Windows 11

VS Code

Version: 1.89.1 (user setup)
Date: 2024-05-07T05:13:33.891Z
Electron: 28.2.8
ElectronBuildId: 27744544
Chromium: 120.0.6099.291
Node.js: 18.18.2
V8: 12.0.267.19-electron.0
OS: Windows_NT x64 10.0.22631

Google Chrome

Versie 125.0.6422.61 (Officiële build) (64-bits)
rcannood commented 4 months ago

Hi @cscheid !

First of all, I'd like to apologize. I didn't intend for this GitHub issue to become a catch-all for any mermaid-related issues in quarto.

I just updated the demo website ( https://mermaid-demos-in-quarto.netlify.app/ ) to quarto 1.5.54 and mermaid 10.2.4. In comparison to my original post, a lot of problems have been fixed, and some still persist.

Would it make sense to close this GitHub issue and for me to create issues for each of the (smaller) remaining issues? In this way, the issue itself is less daunting and other (unrelated) issues do not get added onto this one. No offence to anyone who added other issues to this one, I just think that your issue will get solved quicker if you create a separate issue for it.

Let me know what you think :)

cscheid commented 4 months ago

Yes, please. Two features of good issues are:

Both of these make it easy for us to get to the bottom of it.

rcannood commented 4 months ago

Sounds good!

I added a comment or created new issues for the most important problems.

When these are fixed, I'll update the mermaid-demos-in-quarto page and cross-check for additional issues.

Happy for this issue to be closed in favour of #9178, #10303 and #10304!