dotnet / machinelearning

ML.NET is an open source and cross-platform machine learning framework for .NET.
https://dot.net/ml
MIT License
9.04k stars 1.89k forks source link

Create Pipeline Visualizer #6326

Open luisquintanilla opened 2 years ago

luisquintanilla commented 2 years ago

Create a visualization method that when given a pipeline displays a visual of the pipeline. This can work both for interactive and standard .NET projects.

API

Define pipeline

var pipeline = 
    mlContext.Transforms.Concatenate(outputColumnName:"Features", inputColumnNames: new [] {"Col1", "Col2" })
        .Append(mlContext.Transforms.ReplaceMissingValues(outputColumnName:"Features",inputColumnName:"Features")
        .Append(mlContext.Regression.Trainers.Sdca(labelColumnName:"Label", featureColumnName:"Features");

Interactive

pipeline

.NET project

pipeline.SaveToImage("image.png");

Samples

flowchart TB;
  subgraph OutputColumns
    OutputSchema[Score]
  end
  subgraph Pipeline
    direction TB
    Concatenate--Features-->ReplaceMissingValues
    ReplaceMissingValues--Features-->Sdca
    click Concatenate "https://docs.microsoft.com/dotnet/api/microsoft.ml.transformextensionscatalog.concatenate?view=ml-dotnet#microsoft-ml-transformextensionscatalog-concatenate(microsoft-ml-transformscatalog-system-string-system-string())"
    click ReplaceMissingValues "https://docs.microsoft.com/dotnet/api/microsoft.ml.extensionscatalog.replacemissingvalues?view=ml-dotnet#microsoft-ml-extensionscatalog-replacemissingvalues(microsoft-ml-transformscatalog-microsoft-ml-inputoutputcolumnpair()-microsoft-ml-transforms-missingvaluereplacingestimator-replacementmode-system-boolean)"
    click Sdca "https://docs.microsoft.com/dotnet/api/microsoft.ml.trainers.sdcaregressiontrainer?view=ml-dotnet"
  end
  subgraph InputColumns
    InputSchema["Col1, Col2, Label"]
  end
  InputColumns-->Pipeline
  Pipeline-->OutputColumns

Proposed Implementation

flowchart LR;
    EstimatorChain --> Mermaid
    Mermaid --> Markdig
    Markdig --> HTML
    HTML --> CustomFormatter
    HTML --> Image
  1. Take an ML.NET EstimatorChain and dynamically generate Mermaid diagram.
  2. Process Mermaid diagram as Markdown using Markdig
  3. Convert Mermaid diagram to HTML
  4. Display HTML
    1. If in interactive environment, register a custom formatter.
    2. If in standard .NET application, save as image.
luisquintanilla commented 2 years ago

Related to #4275

luisquintanilla commented 1 year ago

Related to #6563