dotnet / interactive

.NET Interactive combines the power of .NET with many other languages to create notebooks, REPLs, and embedded coding experiences. Share code, explore data, write, and learn across your apps in ways you couldn't before.
MIT License
2.92k stars 389 forks source link

Support text/vnd.mermaid MIME Type in Formatters #3637

Open IntegerMan opened 3 months ago

IntegerMan commented 3 months ago

Is your feature request related to a problem? Please describe. I'd like to be able to easily write formatters for built-in and custom types that use Mermaid markdown to render a custom diagram for an object. My specific case in wanting this at the moment is a Microsoft.ML.ITransformer object where I want to visualize the chain of transformers from a machine learning model.

I can generate a valid mermaid string, but outputting it from a Formatter either results in the mermaid string appearing as text if plain text or HTML mime types are used, or with my formatter not being used at all if I specify "text/markdown" or "text/vnd.mermaid" (Mermaid's official MIME type).

There are ways of manually passing things off to the Mermaid kernel via extensions, but those are a lot more work than a simple Formatter, and it'd be ideal to avoid having to write a magic command just to process something.

Describe the solution you'd like Formatters with a MIME type of text/vnd.mermaid should have their output piped to the Mermaid kernel and the result of that operation should be rendered below the cell when the formatter is in use.

You should be able to set up a formatter in a manner like this:

Formatter.Register<ITransformer>((transformer, writer) =>
{
    writer.Write(MattEland.ML.Interactive.TransformerExtensions.ToMermaid(transformer));
}, "text/vnd.mermaid");

Describe alternatives you've considered I've considered finding an external rendering library to transform mermaid to a supported MIME type, writing a custom magic command to connect things directly to the Mermaid kernel, and considered avoiding Mermaid entirely here and building something in SVG.

Additional Context This is part of a larger effort I'm making to try to improve the data analysis and data science workflow in a Polyglot Notebook based on things I've observed in book / course creation and while using this toolset pursuing my master's degree.

See https://github.com/IntegerMan/MattEland.ML/blob/main/MattEland.ML/MattEland.ML.Interactive/TransformerExtensions.cs and neighboring files for details on my early transformer mermaid visualization attempt.

Also, I mistakenly originally created this issue in the machinelearning repository as https://github.com/dotnet/machinelearning/issues/7175

jonsequitur commented 3 months ago

You can emit the desired MIME type using a formatter but unless you have a notebook renderer installed for that MIME type, it won't display correctly.

The Mermaid kernel is an example of a general design stance we've taken to make the kernels capable of handling cases where a frontend extension would otherwise be needed. It's more portable if we format directly to HTML. The problem that this design is meant to address is that the mutually incompatible frontend ecosystems across JupyterLab, Jupyter Notebook, VS Code, and others leads to a situation where you can't expect a notebook to work the same way in different frontends.