Open GeorgeS2019 opened 2 years ago
Thanks @GeorgeS2019 . It's a great idea. Did you know if Dagre.NET project has any example of Json format for models ? If so, I can take a look. I'm also thinking about getting rid of BinaryFormatter for model serialization and de-serialization. :)
Thanks Zhongkai Fu
@zhongkaifu
It adhere as close as possible to the JavaScript library version
if there is anything you need, please feedback directly here.
DagreInputGraph dg = new DagreInputGraph();
var nd1 = dg.AddNode();
var nd2 = dg.AddNode();
dg.AddEdge(nd1, nd2);
dg.Layout();
Console.WriteLine($"node1 : {nd1.X} {nd1.Y}");
Console.WriteLine($"node2 : {nd2.X} {nd2.Y}");
// Create a new directed graph
var g = new dagre.graphlib.Graph();
// Set an object for the graph label
g.setGraph({});
// Default to assigning a new object as a label for each new edge.
g.setDefaultEdgeLabel(function() { return {}; });
// Add nodes to the graph. The first argument is the node id. The second is
// metadata about the node. In this case we're going to add labels to each of
// our nodes.
g.setNode("kspacey", { label: "Kevin Spacey", width: 144, height: 100 });
g.setNode("swilliams", { label: "Saul Williams", width: 160, height: 100 });
g.setNode("bpitt", { label: "Brad Pitt", width: 108, height: 100 });
g.setNode("hford", { label: "Harrison Ford", width: 168, height: 100 });
g.setNode("lwilson", { label: "Luke Wilson", width: 144, height: 100 });
g.setNode("kbacon", { label: "Kevin Bacon", width: 121, height: 100 });
// Add edges to the graph.
g.setEdge("kspacey", "swilliams");
g.setEdge("swilliams", "kbacon");
g.setEdge("bpitt", "kbacon");
g.setEdge("hford", "lwilson");
g.setEdge("lwilson", "kbacon");
dagre.layout(g);
Did you know if Dagre.NET project has any example of Json format for models ?
Dendrite supports only ONNX so far, but I can implement any Json format that you provide.
Thanks @GeorgeS2019 and @fel88 . It's really helpful.
In Seq2SeqSharp, I used to use Microsoft.Msagl.Drawing to draw operators/layers/networks, however, since it doesn't support .NET core and .NET 5.0, so I comment it out. But I still keep the empty of it. If you look at VisualizeNodes method in https://github.com/zhongkaifu/Seq2SeqSharp/blob/master/Seq2SeqSharp/Tools/ComputeGraphTensor.cs file, you will find these code.
So, one thing we could do it to replace those old code I comment out to your code in Dagre.NET project. In addition, methods in ComputeGraphTensor.cs are operator level, so they are good entry pointers for visualization and ONNX export.
I'm glad if you could do it and let me know if you have any further question on it.
Thanks Zhongkai Fu
@zhongkaifu @fel88
Is there a graph sub patterns or regex that can be used to search the complete e.g BERT.ONNX graph to create groups or nested graphs to create boundaries around e.g. Encoder, Decoder of a transformer architecture?
@GeorgeS2019 Yes, Seq2SeqSharp does have logic to create boundaries for sub-graphs. You can check method "IComputeGraph CreateSubGraph(string name)" in https://github.com/zhongkaifu/Seq2SeqSharp/blob/master/Seq2SeqSharp/Tools/ComputeGraphTensor.cs file as well. However, I already commented it out due to the same reason in above.
@zhongkaifu thnx for the valuable tip :-)
public IComputeGraph CreateSubGraph(string name)
{
ComputeGraphTensor subGraph = new ComputeGraphTensor(m_weightTensorFactory, m_deviceId, m_needsBackprop, m_backprop, isSubGraph: true);
if (m_visNeuralNetwork)
{
// Create parameters for neural network visualization
subGraph.m_opsViz = m_opsViz;
subGraph.m_setEdges = m_setEdges;
subGraph.m_name2SubGraph = m_name2SubGraph;
if (m_name2SubGraph.ContainsKey(name) == false)
{
int index = name.LastIndexOf(".");
subGraph.m_subGraph = new Subgraph(name)
{
LabelText = name.Substring(index + 1)
};
m_name2SubGraph.Add(name, subGraph.m_subGraph);
if (m_subGraph == null)
{
m_opsViz.RootSubgraph.AddSubgraph(subGraph.m_subGraph);
}
else
{
m_subGraph.AddSubgraph(subGraph.m_subGraph);
}
}
else
{
subGraph.m_subGraph = m_name2SubGraph[name];
}
}
return subGraph;
}
private void VisualizeNodes(IEnumerable<IWeightTensor> sourceNodes, IWeightTensor targetNode)
{
if (!m_visNeuralNetwork || m_deviceId != 0)
{
return;
}
// Create node for target tensor
int index = targetNode.Name.LastIndexOf('.');
Microsoft.Msagl.Drawing.Node tgtNode = m_opsViz.AddNode(targetNode.Name);
tgtNode.LabelText = targetNode.Name.Substring(index + 1);
if (targetNode.IsTrainable)
{
tgtNode.Attr.FillColor = Microsoft.Msagl.Drawing.Color.LightSteelBlue;
}
if (m_subGraph != null)
{
// Current compute graph is a sub-graph
m_subGraph.AddNode(tgtNode);
}
// Create edges for each source node and target node
foreach (IWeightTensor sourceNode in sourceNodes)
{
if (!string.IsNullOrEmpty(sourceNode.Name) && !string.IsNullOrEmpty(targetNode.Name))
{
string key = $"{sourceNode.Name}->{targetNode.Name}";
if (m_setEdges.Contains(key))
{
continue;
}
int srcIndex = sourceNode.Name.LastIndexOf('.');
Microsoft.Msagl.Drawing.Node srcNode = m_opsViz.AddNode(sourceNode.Name);
srcNode.LabelText = sourceNode.Name.Substring(srcIndex + 1);
if (sourceNode.IsTrainable)
{
srcNode.Attr.FillColor = Microsoft.Msagl.Drawing.Color.LightSteelBlue;
if (m_subGraph != null)
{
m_subGraph.AddNode(srcNode);
}
}
Edge edge = m_opsViz.AddEdge(sourceNode.Name, targetNode.Name);
m_setEdges.Add(key);
}
}
}
Is your feature request related to a problem? Please describe. We need a way to visualize/edit/export (onnx) transformer architecture for .NET community.
Describe the solution you'd like Dagre.NET and Dendrite could evolve to support this need. Seq2SeqSharp could export the architecture in Json format for import into Dendrite or use Dagre.NET to export images.