Rewrite profiling graph using force-directed D3

[ezyang]

Here's a recent report I generated: http://web.mit.edu/~ezyang/Public/report.html

I wanted to make a rule graph with the filter name(/(.+).o \1.hi/)

But on both Firefox and Chrome, it seems to just hang.

[ndmitchell]

Looking at the JS console, it says:

Uncaught Assertion: Cannot enlarge memory arrays. Either (1) compile with -s TOTAL_MEMORY=X with X higher than the current value ( 10485760), (2) compile with ALLOW_MEMORY_GROWTH which adjusts the size at runtime but prevents some optimizations, or (3) set Module.TOTAL_MEMORY before the program runs.

So this is the GraphViz viewer keeling over with out of memory. It looks plausible though, so I'll check the dot output that is being generated (I might be generating something silly) and see if I can catch this error and display a nicer message.

[ezyang]

Oof! Maybe we should use force directed layout with a library like d3.js, so that you can scale to more points.

[ndmitchell]

I'm not wedded to GraphViz, it's just something I already know and managed to find. I'd rather a native Javascript solution, if someone can point me at which bit of d3 and an example which works "in the right way".

[ezyang]

• Force directed layout: http://bl.ocks.org/mbostock/4062045
• Dependency wheel: http://www.redotheweb.com/DependencyWheel/

I actually am not entirely sure what a good visualization for dependency graphs are; they tend to be quite unwieldy in practice. In practice, what is Rule Graph most useful for? I was hoping to get some insight into what modules were "blockers".

[ndmitchell]

Finding blockers is quite hard, unless you can prune the relevant dependency path quite small. I've found it useful to see how extensions depend on each other (general structure of the graph), and see the path for some particular dependency (almost zooming in, but with the filter).

[ndmitchell]

I think going force directed as per that example is the best way to go. It might even be enough to see blockers in your case.

[ndmitchell]

From #467 a note that GraphViz doesn't play nicely if you have custom fonts, as @nkaretnikov has - another point in favour of D3.

[ndmitchell]

Another point in favour of D3. The GraphViz code on a small profile accounts for 87% of the profile size. Writing out the profile seems entirely proportional to the size, and on my laptop, 1.5Mb (the "empty" profile) takes 0.6s, so making the empty profile smaller would be beneficial.

[ndmitchell]

See also https://github.com/haslab/HAAP/blob/master/packages/Hoed-extras/HoedExtrasTest/jshood.html which has a nice example of using Vis.

[ndmitchell]

https://gephi.org/developers/ is another possible tool