Dependency Graph Version Clustering

[nathanhaigh]

I was thinking that it would be nice to group different version-toolchains together into a single node "cluster" if there is more than 1 version-toolchains combination for a particular tool.

Although I haven't coded this into Easybuild, I have done a quick post-dot creation manipulations to demonstrate what I would look like.

Here's an example for GNU-4.9.3-2.25

[boegel]

That looks useful... How do you achieve this in dot files?

[nathanhaigh]

The above figure was created from this dot file:

digraph graphname {
  // Set some defaults for the graph and nodes
  // To make things "neater" it's best with "record" shaped nodes.
  graph [fontsize=10 fontname="Verdana"];
  node [shape=record fontsize=10 fontname="Verdana"];

  // Subgraph definitions replace existing node definitions. We just define a cluster
  //   when multiple versions/toolchains exist for a tool.
  // Nodes for tools with a single version-toolchain in the graph are implicit in the edge
  //   definitions, which remain as unchanged.

  // The subgraph name must have "cluster" as its prefix
  subgraph "cluster_M4" {
    // Nodes within the subgraph become shaded
    node [style=filled];
    // Name the cluster according to the tool's name
    label = "M4";
    // Colour the subgrapgh boarder red
    color=red;
    // Define the nodes which are in the cluster
    // Use the version-toolchain as the label for aesthetics but the full tools/version-toolchain
    //   as the node ID. This enables existing edge definitions to work seemlessly
    "M4/1.4.17" [label="1.4.17"];
    "M4/1.4.17-GCC-4.9.3-binutils-2.25" [label="1.4.17-GCC-4.9.3-binutils-2.25"];
  }

  subgraph "cluster_binutils" {
    node [style=filled];
    label = "binutils";
    color=red;
    "binutils/2.25-GCC-4.9.3-binutils-2.25" [label="2.25-GCC-4.9.3-binutils-2.25"];
    "binutils/2.25" [label="2.25"];
  }

  subgraph "cluster_flex" {
    node [style=filled];
    label = "flex";
    color=red;
    "flex/2.5.39" [label="2.5.39"];
    "flex/2.5.39-GCC-4.9.3-binutils-2.25" [label="2.5.39-GCC-4.9.3-binutils-2.25"];
  }

  subgraph "cluster_zlib" {
    node [style=filled];
    label = "zlib";
    color=red;
    "zlib/1.2.8-GCC-4.9.3-binutils-2.25" [label="1.2.8-GCC-4.9.3-binutils-2.25"];
    "zlib/1.2.8" [label="1.2.8"];
  }

  subgraph "cluster_Bison" {
    node [style=filled];
    label = "Bison";
    color=red;
    "Bison/3.0.4-GCC-4.9.3-binutils-2.25" [label="3.0.4-GCC-4.9.3-binutils-2.25"];
    "Bison/3.0.4" [label="3.0.4"];
  }

"binutils/2.25-GCC-4.9.3-binutils-2.25" -> "flex/2.5.39-GCC-4.9.3-binutils-2.25"  [color=blue, style=dotted, arrowhead=diamond];
"binutils/2.25-GCC-4.9.3-binutils-2.25" -> "Bison/3.0.4-GCC-4.9.3-binutils-2.25"  [color=blue, style=dotted, arrowhead=diamond];
"binutils/2.25-GCC-4.9.3-binutils-2.25" -> "zlib/1.2.8-GCC-4.9.3-binutils-2.25"  [color=blue, style=dotted, arrowhead=diamond];
"binutils/2.25-GCC-4.9.3-binutils-2.25" -> "binutils/2.25"  [color=blue, style=dotted, arrowhead=diamond];
"binutils/2.25-GCC-4.9.3-binutils-2.25" -> "GCC/4.9.3-binutils-2.25";
"zlib/1.2.8-GCC-4.9.3-binutils-2.25" -> "binutils/2.25"  [color=blue, style=dotted, arrowhead=diamond];
"zlib/1.2.8-GCC-4.9.3-binutils-2.25" -> "GCC/4.9.3-binutils-2.25";
"GNU/4.9.3-2.25" -> "GCC/4.9.3-binutils-2.25";
"GNU/4.9.3-2.25" -> "binutils/2.25-GCC-4.9.3-binutils-2.25";
"Bison/3.0.4" -> "M4/1.4.17"  [color=blue, style=dotted, arrowhead=diamond];
"GCC/4.9.3-binutils-2.25" -> "binutils/2.25"  [color=blue, style=dotted, arrowhead=diamond];
"M4/1.4.17-GCC-4.9.3-binutils-2.25" -> "binutils/2.25"  [color=blue, style=dotted, arrowhead=diamond];
"M4/1.4.17-GCC-4.9.3-binutils-2.25" -> "GCC/4.9.3-binutils-2.25";
"Bison/3.0.4-GCC-4.9.3-binutils-2.25" -> "M4/1.4.17-GCC-4.9.3-binutils-2.25"  [color=blue, style=dotted, arrowhead=diamond];
"Bison/3.0.4-GCC-4.9.3-binutils-2.25" -> "binutils/2.25"  [color=blue, style=dotted, arrowhead=diamond];
"Bison/3.0.4-GCC-4.9.3-binutils-2.25" -> "GCC/4.9.3-binutils-2.25";
"binutils/2.25" -> "flex/2.5.39"  [color=blue, style=dotted, arrowhead=diamond];
"binutils/2.25" -> "Bison/3.0.4"  [color=blue, style=dotted, arrowhead=diamond];
"binutils/2.25" -> "zlib/1.2.8"  [color=blue, style=dotted, arrowhead=diamond];
"flex/2.5.39-GCC-4.9.3-binutils-2.25" -> "binutils/2.25"  [color=blue, style=dotted, arrowhead=diamond];
"flex/2.5.39-GCC-4.9.3-binutils-2.25" -> "GCC/4.9.3-binutils-2.25";
}

[nathanhaigh]

This is a Perl script to convert dot files currently output by EasyBuild. It reads and writes dot on STDIN/STDOUT:

#!/usr/bin/env perl
use strict;
use warnings;

my %tool_versions;
my @edges;
while (<>) {
  chomp;

  if (/^"([^\/]+)\/(\S+?)";$/) {
    # Extract tool/toolchain info from node definition lines
    $tool_versions{$1}{$2} = 1;
    next
  } elsif (/ -> /) {
    # Extract edge definition lines
    push @edges, $_;
  }
}

# Output dot file header
print <<__HEADER__;
digraph graphname {
  graph [fontsize=10 fontname="Verdana"];
  node [shape=record fontsize=10 fontname="Verdana"];

__HEADER__

# Output subgraph/clusters for tools with > 1 version-toolchain
foreach my $tool (keys %tool_versions) {
  next if scalar keys %{$tool_versions{$tool}} == 1;
  print "  subgraph \"cluster_$tool\" {\n    node [style=filled];\n    label = \"$tool\";\n    color=red;\n";
  foreach my $version (keys %{$tool_versions{$tool}}) {
    printf "    \"%s\" [label=\"%s\"];\n",
      "$tool/$version",
      $version;
  }
  print "  }\n\n";
}

# Output all encountered edge definition lines
foreach my $edge (@edges) {
  print "  $edge\n";
}

print "}\n";