rigtorp commented 4 years ago

When writing technical documentation and similar it's useful to include figures and diagrams. Currently they need to be generated externally as static content and referenced in the markdown. It's common to generate figures and diagrams from textual representations using for example plantuml and ditaa. It would be useful if this generation step could be done by hugo itself.

Issue https://github.com/gohugoio/hugo/issues/796 proposes to add a short code that can execute any external program. This was deemed problematic for security reasons. Instead I propose to add support for specific useful external generators. There is already precedent for this in the hugo support for AsciiDoc, RST and pandoc.

I propose extending the goldmark integration so that specific fenced code blocks would be replaced by generated svg images.

Security considerations

Since this is a feature that needs to be enabled for the goldmark generator in the hugo config similar to unsafe HTML, a malicious theme or content creator would not be able to execute these generators without the site admin enabling them in the hugo config.

Examples

The following fenced code block would be replaced by a SVG representation of the diagram as output by the svgbob tool:

                                   .--->  F
          A       B      C  D     /
          *-------*-----*---*----*----->  E
                   \            ^ \
                    v          /   '--->  G
                     B --> C -'

The following fenced code block would be replaced by a SVG representation of the plot as generated by gnuplot:

set key right nobox
set samples 100
plot [-pi/2:pi] cos(x),-(sin(x) > sin(x+1) ? sin(x) : sin(x+1))

Integration options

Simplest solution is to launch the external tool and grab it's output from stdout. An option that might be viable for picchr is to use https://github.com/elliotchance/c2go to translate the single C file to Go and bundle with hugo. For svgbob it might be possible to use https://github.com/wasmerio/wasmer-go to run svgbob compiled to webassembly.

List of useful image generators

rlcomte commented 3 years ago

I have integrated mermaid into hugo using a shortcode and a mermain script include. The basics work like a charm. Styling is a bit more work, but could be done as well.

rigtorp commented 3 years ago

Using mermaid requires client side java script. Which I try to avoid.

On Tue, Oct 13, 2020, 05:28 Rine le Comte notifications@github.com wrote:

I have integrated mermaid https://mermaid-js.github.io/mermaid/#/into hugo using a shortcode and a mermain script include. The basics work like a charm. Styling is a bit more work, but could be done as well.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/gohugoio/hugo/issues/7765#issuecomment-707704780, or unsubscribe https://github.com/notifications/unsubscribe-auth/AABLO2YV6SW6Q7WOZBQDTN3SKRBWJANCNFSM4SCEWBJQ .

pdugas commented 3 years ago

When I write pages in Hugo, I really miss the ability to embed PlantUML diagrams like I can in GitLab's markdown. Would it be kosher to look into using PlantUML's public server to generate diagram images at build time and stashing them in resources/_gen? If so, and if someone familiar with the codebase could point me to a starting point, I'll go learn Go and see what I can figure out. Been coding for 30 years - how hard can it be? lol

rigtorp commented 3 years ago

@pdugas I propose that hugo would call the plantuml executable and retrieve the svg from stdout. No network access needed.

pdugas commented 3 years ago

GitLab replaces triple-backtick-plantuml blocks in Markdown like the example below with the resulting image.

```plantuml
Alice -> Bob: Authentication Request
Bob --> Alice: Authentication Response

Alice -> Bob: Another authentication Request
Alice <-- Bob: Another authentication Response
```

Is there some other logic already in Hugo that gets called for portions of Markdown content like this that I could look at replicating? #7921 seems to suggest there isn't but that there is interest in supporting something along these lines, if I'm reading thing correctly.

rigtorp commented 3 years ago

If you load this extension for goldmark it will work exactly like that: https://github.com/OhYee/goldmark-plantuml

gadams999 commented 3 years ago

@rigtorp Is this loaded in the same way that some of the other extensions are? That is, add the repo to .go.mod, add references to the config.go and convert.go files, then compile Hugo? Based on your link (thanks) and what others did with MathJax, I was able to get working. Now trying to determine if I should submit a PR to Hugo.

rigtorp commented 3 years ago

@gadams999 Yes please submit a PR. There was very little interest from the maintainers here so I don't know how well it will go over. I've submitted PRs to hugo before with no response.

pdugas commented 3 years ago

I spun up a shortcode that works with PlantUML. See https://gitlab.com/-/snippets/2108575. YMMV

rigtorp commented 3 years ago

@pdugas Nice, but I would prefer to have the svg statically generated by hugo.

pdugas commented 3 years ago

No argument. That would definitely be preferable. My Go foo is lacking currently...

pdugas commented 3 years ago

8398 looks promising.

stevegt commented 3 years ago

This looks like a special case of #7921, which @bep has currently targeted for milestone v0.84. Not sure if he's working on code for it yet, but a PR might be welcome over there.

christophcemper commented 3 years ago

@bep any chance we can get plantUML with this soon?

arturdryomov commented 2 years ago

FYI — GitHub shipped the Mermaid support.

ondrejtomcik commented 2 years ago

@arturdryomov how this helps gohugo templating? Is there support for mermaid in gohugo?

stevegt commented 2 years ago

I may be missing something, but I think what github's new mermaid support means is that more folks may become comfortable with mermaid versus other diagram generation tools, for better or worse.

Mermaid support for goldmark: https://pkg.go.dev/github.com/abhinav/goldmark-mermaid

stevegt commented 2 years ago

It looks like security concerns here and in #796, #7921, and #8398 are slowing or blocking getting any diagramming support into hugo.

I could argue it like this:

Pros of using e.g. https://pkg.go.dev/github.com/abhinav/goldmark-mermaid:

Mermaid doesn't require that hugo execute any external processes.
- (Are there any server-side SVG generators yet that are written in Go and have the functionality of e.g. plantuml or mermaid? It's been a long time since I looked.)
Generating the diagram in client-side js moves the execution to a sandboxed environment in the client's browser, removing the server-side security concerns.

Cons:

A disadvantage of client-side js is that archiving or rendering in other media is harder.

pintergreg commented 2 years ago

There is a https://github.com/jfeliu007/goplantuml project, that seems just the perfect solution to integrate (into the extended version maybe). Also seems to be actively developed, but I've never used it and don't know how complete the implementation is. PlantUML supports several non-UML diagrams now, it should be thoroughly compared.

bep commented 2 years ago

@stevegt

It looks like security concerns here and in https://github.com/gohugoio/hugo/issues/796, https://github.com/gohugoio/hugo/issues/7921, and https://github.com/gohugoio/hugo/issues/8398 are slowing or blocking getti

Yea, well, but an os.Exec template function would also be ... very slow for big sites.

A disadvantage of client-side js is that archiving or rendering in other media is harder.

Yea, but I have always have an idea that the Markdown rendering would eventually follow the output format and that, if Mermaid keep its traction, someone will eventually implement a Go version ...

I may be missing something, but I think what github's new mermaid support means is that more folks may become comfortable with mermaid versus other diagram generation tools, for better or worse.

I agree about that.

, that seems just the perfect solution to integrate (into the extended version maybe).

@pintergreg is goplantuml CGO?

GitHub's use of iframe for this looks interesting (but that will have to wait). I looked at the Goldmark plugin linked to above, and that looks fine, but a little heavy-handed. I'm not sure we would want Hugo to include script blocks with CDN includes as part of the Markdown render.

I have looked at one of the Mermaid examples:

<!DOCTYPE html>
<html lang="en">
<head>
  <meta charset="utf-8">
</head>
<body>
  <div class="mermaid">
  graph LR
      A --- B
      B-->C[fa:fa-ban forbidden]
      B-->D(fa:fa-spinner);
  </div>
 <script src="https://cdn.jsdelivr.net/npm/mermaid/dist/mermaid.min.js"></script>
 <script>mermaid.initialize({startOnLoad:true});
</script>
</body>
</html>

In my head:

If Hugo rendered the div above, that should solve the "bulk of the problem".
I think we should make it up to the user to load the mermaid JS script (I assume it can be loaded once at the bottom). We should probably add a way to figure out if Mermaid is in use, but I'm not sure that will be a blocker for the first take.

gwatts commented 2 years ago

If anyone is interested in seeing how we use Mermaid for the docsy theme, most of the heavy lifting is done by a snippet of js: https://github.com/google/docsy/blob/master/assets/js/mermaid.js - mermaid itself is loaded from the cdn

Of course, docsy has a more constrained problem in so far as users of the theme should only enable it if they trust the content creator.

bep commented 2 years ago

This looks pretty cool: https://github.com/blampe/goat

jmooring commented 2 years ago

A few weeks ago someone asked about using an external service to generate and then embed mermaid, plantuml, etc. It seemed to work pretty well:

git clone --single-branch -b hugo-forum-topic-36924 https://github.com/jmooring/hugo-testing hugo-forum-topic-36924
cd hugo-forum-topic-36924
hugo server

Ref https://discourse.gohugo.io/t/is-there-some-pre-packaged-way-to-use-kroki-from-markdown/36924/5

bep commented 2 years ago

@jmooring that's pretty impressive.

davidsneighbour commented 2 years ago

Gists should support Mermaid.

https://docs.github.com/en/get-started/writing-on-github/working-with-advanced-formatting/creating-diagrams

https://github.blog/2022-02-14-include-diagrams-markdown-files-mermaid/

McShelby commented 2 years ago

Without having any knowledge about Hugo's architecture, just as an idea, that builds on top of @jmooring: Start up a local Mermaid server during generation and request the diagrams via GET request.

This one is written in Go https://github.com/TomWright/mermaid-server

pintergreg commented 2 years ago

is goplantuml CGO?

It is not, I guess. But I'm not a Go programmer, just a Hugo user. The repo practically contain only Go code. PlantUml is written in Java, btw.

On the other hand, I could not even install it, as the the provided command does not work (https://github.com/jfeliu007/goplantuml/issues/119).

The missing module support may cause problems and there are also some issues about rendering. Furthermore, it seems that only the basic uml diagrams are implemented.

freylax commented 2 years ago

Thanks for this great step for using ascii sketches as codeblocks. I did not found a way how to use this from org-mode files. Is there a way to use goat from go-org or how could this be achieved. I'm using v0.96.

bep commented 2 years ago

I suspect you would get better answers at https://discourse.gohugo.io/

github-actions[bot] commented 2 years ago

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.

gohugoio / hugo

Add support for generating figures and diagrams #7765

Security considerations

Examples

Integration options

List of useful image generators

8398 looks promising.