Plotting of traces

[mkborregaard]

I think this package is awesome, in fact I just recently looked everywhere for a package like this. One thing that could be awesome would be plotting capabilities for traces - am I right in thinking that a trace can be expressed as a graph with functions as vertices and traces as edges?

If I understand correctly what this package does, there are a lot of possibilities, e.g. making the sizes of vertices proportional to the time spent in a function (as a type of profiling), the width of edges proportional to the number of times it appears, etc.

Long story short, if you're interested in this and can help on the tracecalls side of things I'd be willing to contribute a plotting recipe. It would likely come with a dep on a graph library.

[cstjean]

Adding graph support would be cool! I'm not very familiar with the graph ecosystem in Julia. Could you point me out to a visualization example similar to what you're thinking about? I worry that function calls take a lot space to display, but maybe there are ways around that.

am I right in thinking that a trace can be expressed as a graph with functions as vertices and traces as edges?

The data structure is explained here. Each Trace object is a node, and the edges are in Trace.called

It would likely come with a dep on a graph library.

Can it be optionally loaded with @require?

[mkborregaard]

Depends what you mean by similar - I used PlotRecipes to create this graph of the dependencies and inverse dependencies of the Plots package in Julia:

It is part of an effort to try to make elements of julia code easy to visualize - another example is Julia type trees: https://github.com/JuliaPlots/PlotRecipes.jl#julia-type-trees . As you can see, the algorithms aren't publication-grade, but very useful for interactive visualization.

With regards to the dep: why don't I develop this in a PR to TraceCalls, and then we'll see whether it should end up living here, or in a small "glue" package by itself, e.g. TracePlots.jl? As long as you're happy to spar and feedback.

[cstjean]

With regards to the dep: why don't I develop this in a PR to TraceCalls, and then we'll see whether it should end up living here, or in a small "glue" package by itself, e.g. TracePlots.jl? As long as you're happy to spar and feedback.

Sure!

[mkborregaard]

This example plots a call graph, i.e. who calls who. That's useful, but I was thinking more of something like a call tree, that keeps track of the sequence and time information as well, but I'll need to define that plot myself, I think. Haven't seen those in the literature.

using TraceCalls

type MyGraph
    source::AbstractVector{Int}
    destiny::AbstractVector{Int}
end

type TraceGraph
    g::MyGraph
    linewidths::Int
    functionnames::Vector{String}
end

function make_tracegraph(trace::TraceCalls.Trace)
    function addedge!(g::MyGraph, m, n)
        for i in eachindex(g.source)
            g.source[i] == m && g.destiny[i] == n && return
        end

        push!(g.source, m)
        push!(g.destiny, n)
    end

    function add_subcalls!(trace)
         funcsym = Symbol(trace.func)
         fromvert = findfirst(functionnames, funcsym)

         subcalls = trace.called
         for subcall in subcalls
             subsym = Symbol(subcall.func)
             if subsym in functionnames
                 tovert = findfirst(functionnames, subsym)
             else
                 push!(functionnames, subsym)
                 tovert = length(functionnames)
             end

              addedge!(g, fromvert, tovert)
              add_subcalls!(subcall)
          end
      end

     functionnames = Symbol[Symbol(trace.func)]
     g = MyGraph(Int[], Int[])

     add_subcalls!(trace)
     g.source, g.destiny, functionnames
 end

## demo

using LightGraphs
graph = Graph(3)                                      # build an undirected graph with three connected vertices
add_edge!(graph, 1, 2); add_edge!(graph, 2, 3)
trace_walk = @trace LightGraphs randomwalk(graph, 2, 5)
s,d,f = make_tracegraph(trace_walk)

using PlotRecipes; pyplot()
graphplot(s,d, names = f, ms = 5)

[cstjean]

Is there any way to plot a directed graph with arrows? My traces are trees because of the arguments, but without the arguments, mutually-recursive functions will form a graph with cycles.

[mkborregaard]

Not with the current algorithms. That is one of the reasons I want to plot it like a tree instead. One possibility is to use something that looks like a phylogenetic tree, the question is just how to place functions that are called several times by functions different places in the call chain.

[mkborregaard]

Do you think the vertices should be methods rather than functions? Can that be extracted from the Trace?

[cstjean]

Do you think the vertices should be methods rather than functions?

That depends on the use case for this functionality. Do you have anything specific in mind? If it's to get an overview of a code base, then I think functions are more appropriate.

You can get the method for each trace with which(::Trace). FWIW, you could also build a call graph from the trees displayed in Profile.print

[mkborregaard]

That's right, the idea is to make it easier for people to get an overview of a code base, but also to help a more visual approach to profiling, in fact. Is that not the intended use of this package as a whole?

In my experience the Profile.print trees don't always show the sequence of calls correctly? I may be mistaken here.

[cstjean]

That's right, the idea is to make it easier for people to get an overview of a code base, but also to help a more visual approach to profiling, in fact. Is that not the intended use of this package as a whole?

Sure! I meant that if you want a call graph, then functions-as-node works well, but for profiling, maybe methods-as-node. Also might be cool to display the map(is_inferred) results with red/green method-nodes. Same for the compare_past_trace output.

Can PlotRecipes do tooltips?

[mkborregaard]

Nice idea! Tooltips at the moment only work with the PlotlyJS backend, so they can't display vital information.

[mkborregaard]

I didn't implement your colour suggestions yet, but here's my current take on the plots. The trace_walk = @trace LightGraphs randomwalk(graph, 2, 5) graph now looks like:

For a much more complex example illustrating a call to Plots.histogram(x) see https://www.dropbox.com/s/nu9i7dq4achfeu4/newesthistogram.pdf?dl=0

Comments and ideas very welcome. I think graphs like these could be very useful for someone engaging with a new code base.

[cstjean]

Looks very nice with this layout👍. I agree, it gives a better overview of the code base. I wonder what we could do about shared sub trees (when a deep function is called in several different places). Or maybe it's not a significant issue, users can always use filter-cutting.

On vacation now, so I may not reply promptly.

On Monday, August 21, 2017, Michael Krabbe Borregaard < notifications@github.com> wrote:

I didn't implement your colour suggestions yet, but here's my current take on the plots. The trace_walk = @trace LightGraphs randomwalk(graph, 2, 5) graph now looks like:

For a much more complex example illustrating a call to Plots.histogram(x) see https://drive.google.com/open?id=0BwWjA0Ez4iLhMGVkTVZMXzBUQkk

Comments and ideas very welcome. I think graphs like these could be very useful for someone engaging with a new code base.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/cstjean/TraceCalls.jl/issues/36#issuecomment-323729714, or mute the thread https://github.com/notifications/unsubscribe-auth/AIOSuHScTly3u0iBuczpi1StETQJRLpAks5saXXOgaJpZM4Oxxio .

[mkborregaard]

Great, enjoy your vacation. I've been pondering the subtrees as well. Try to think of something.