srid / neuron

Future-proof note-taking and publishing based on Zettelkasten (superseded by Emanote: https://github.com/srid/emanote)
https://neuron.zettel.page
GNU Affero General Public License v3.0
1.51k stars 149 forks source link

Embed Graphviz graphs #228

Open srid opened 4 years ago

srid commented 4 years ago

DOT language code blocks should render using d3-graphviz.

Similar to: https://forum.graphviz.org/t/render-graphviz-graphs-directly-in-your-posts/125

NullSense commented 4 years ago

Perhaps it would be a good idea to look into what d3.js has to offer, with something like pixi.js.

srid commented 4 years ago

Roam supports mermaid diagrams: https://roamresearch.com/#/app/help/page/kw78QlSZ6

b0o commented 3 years ago

I've been working on a tool to 'transform' code blocks by piping their content through CLI utilities like graphviz, graph-easy, svgbob, ditaa, or any other program that accepts stdin and outputs to stdout (or even a JavaScript function).

2020-08-06_19-11-58_102:nx:2

2020-08-06_19-51-29_DP-3

I think it's an interesting approach as it enables all sorts of cool things beyond diagram generation. For example, you can execute Python code blocks and put the output into another code block, basically re-creating the basic functionality of a Jupyter notebook:

2020-08-06_19-40-16_102:nx:2

Maybe a general-purpose approach like this would be a good way to implement this functionality? There are definitely security concerns when it comes to executing arbitrary code, but I can imagine a number of ways to mitigate them (e.g. a configurable list of allowed executables).

srid commented 3 years ago

@b0o That's interesting.

There are two classes of code evaluation here:

A is lightweight and safer (and can be integrated with Cerveau), whereas B is flexible - allowing one to use any tool installed.

A can be implemented in neuron itself (as a builtin plugin); and B can perhaps be done as some sort of general Pandoc filter? For example, you specify the executables to use as filter in neuron.dhall, and then neuron passes the AST to those programs, expecting them to output the final AST that gets used in generated html files.

b0o commented 3 years ago

I think an integrated solution like mermaid.js would suffice most use cases and be much more portable, lightweight, and secure. I think that's the way to go for the most part. I do think an additional 'class B' solution would be quite valuable for the flexibility it offers, though.

I didn't know about Pandoc filters – I think that would be a great way to accomplish a 'class B' solution. Having a way to 'tap into' the rendering pipeline between Neuron and the output HTML would open up a lot of possibilities.

On a related note: since the tool I'm working on operates on Neuron's rendered HTML files and not the source markdown files, I have run into some issues dealing with Neuron/Rib/Shake thinking that the HTML files need to be re-generated after my script runs.

My guess (without investigating further) is that Neuron/Rib/Shake is caching the hashes or modification times of the HTML files (possibly in .shake/.shake.database?), which my program then invalidates by modifying those HTML files. The next time Neuron runs, the files my transfomer program modified are overwritten. Then, my program to needs to re-transform all of the files again, even if the change that triggered the Neuron run happened in an unrelated file.

I can think of a few ways to work around this, but if I could just hook into the rendering pipeline, e.g. with a Pandoc filter, I would assume that Neuron/Rib/Shake would cease to be confused about the modification made by my script and everyone would live happily ever after 😉.

srid commented 3 years ago

I came across https://excalidraw.com - which produces SVG of hand drawn diagrams. It is pretty cool. The web app has a "Copy to clipboard as SVG" feature, which you can past directly in a neuron markdown file and it will work. However, another option is to just paste the URL and let neuron do the replacement for us, kind of like: https://twitter.com/Vjeux/status/1257906664239333376

To make it self-contained, the diagram has to be put in the zettelkasten, perhaps as a excalidraw file (which apparently is JSON based).

NullSense commented 3 years ago

https://observablehq.com/@d3/force-directed-graph <- Very beautiful, interactive graphs.

srid commented 3 years ago

@b0o On my personal Zettelkasten, I hit upon a need for Pandoc filters myself (mainly to transform tasks lists based on inline tags to identify next actions), so it had me once again reflect on how this feature would look if implemented. So I've opened a feature request #363

srid commented 3 years ago

This feature can then be implemented using one of the existing Pandoc filters, i.e. https://github.com/hertogp/imagine

aca commented 3 years ago

Here's how I integrated chartjs, mermaid, graphviz. https://aca.github.io/c34ea231.html

srid commented 3 years ago

https://neuron.zettel.page/custom-head.html

0ihsan commented 3 years ago

And here's how I embedded TikZJax.

zettelkasten/head.html

<!-- tikzjax -->
<link rel="stylesheet" type="text/css" href="https://tikzjax.com/v1/fonts.css">
<script src="https://tikzjax.com/v1/tikzjax.js"></script>
<script>
window.addEventListener("load", function(){
  for (let element of document.getElementsByClassName("tikzjax")) {
    let parent = element.parentNode;
    let pparent = parent.parentNode;
    let script = document.createElement('script'); script.type = 'text/tikz';
    let box = document.createElement('div');
    script.appendChild(document.createTextNode(element.textContent));
    box.appendChild(script)
    box.setAttribute("style","display:block;width:75%;text-align:'center';margin: 5px auto;");
    pparent.replaceChild(box, parent);
  }
});
</script>

zettelkasten/a-zettel.md

```tikzjax
\begin{tikzpicture}
   \draw (0,0) circle (1in);
\end{tikzpicture}
```
Screenshot 2020-12-08 at 14 48 43
srid commented 3 years ago

@ihsanturk Cool. Perhaps we can add it to https://neuron.zettel.page/custom-head.html

srid commented 3 years ago

Relevant: https://github.com/srid/neuron/issues/531#issuecomment-763027613 (a plugin that automatically injects the JS and inlines the code when it sees ![[Example1.graphviz]])?

jaygray0919 commented 3 years ago

Would be very useful; part of a generalized approach to support external, structured data. Graphviz is the workhorse for visualizing RDF structures. Mermaid and similar visualizers are comparable (subsets). The key here - which you are doing - is generalizing service initiation. The equivalent model: the configuration.py setup in Sphinx where different Python libraries are initialized. A developer doesn't need to know the internals of the libraries; only the syntax to achieve a purpose.