Some thoughts....

[chabb]

Thanks for letting me participating in the project :)

I have a couple of random thoughts there... that comes more from a developer perspective

It looks like the JSON format is ad-hoc/data-set oriented.

What i see there is a graph that evolves over time, so we could have an internal representation that will store the graph and its evolution. That would decouples our JSON from the chart. Right now it is very tied to the type of chart. I tried to make another type of chart with the same datas to illustrate my point.

And by having a graph representation, you can use all graph representation algorithm ( i give you a simple idea, say the graph is the representation of the interaction of the people of a book... if the graph is a spanning tree at a moment T, we know that all people interact at that 'frame' )

By having a intermediate representation that is decoupled from the type of chart can give us more ideas... if a JSON is too close from a given chart.. it closes a bit your brain.

These are just some random thoughts, so feel free to comment.. :)

[RasmusFonseca]

Hi Francois .. great you want to participate!

I agree that it'd be nice if the underlying datastructures are not too tied to our current application, but I do consider the current representation relatively generic (i.e. non-ad-hoc). You can think about the 'interactions'-section as an adjacency matrix representation where each entry in the matrix holds a list of active time-frames instead of a single bit. Each matrix-entry is a line in the 'interactions'-section. I'm open to better solutions, though.

The term 'interaction' is very data-set oriented. We could change this to 'connection' or 'edges' or something. Thoughts?

[chabb]

Hi Rasmus...

Please excuse me for my english, i am writing a bit fast.....

We can switch to edges... that will makes thing more obvious iguess.

I'd like to have your feedback for another part, i'd like to be able to 'move' the nodes in real-time.. and the use of the cluster layout makes this stuff a bit complicated 💯

I think we can get rid of that part

   var cluster = d3.layout.cluster()
  .size([360, ry - 120])
  .sort(function(a, b) { 

    var aRes = a.key.match(/[0-9]*$/);
    var bRes = b.key.match(/[0-9]*$/);
    if(aRes.length==0 || bRes.length==0){
      aRes = a.key;
      bRes = b.key;
    }else{
      aRes = parseInt(aRes[0]);
      bRes = parseInt(bRes[0]);
    }
    return d3.ascending(aRes, bRes); });
    bundle = d3.layout.bundle();

Basically, my concern is that we tie ourselves to this cluster layout (now the layout imposes its ordering)... and i think this layout is not really needed (not 100% sure about that)...my issue is that every time we need to recompute the whole layout, then make a data-join, go through the whole selection dance. Maybe we can find a way that gives us more flexibility...

We need the cluster layout only if we have some kind of tree structure... i am not sure we have a 'real' tree structure..... if this is the case.. then ok.. we need this layout...

but let's suppose we do not need a tree structure or that we can flatten the tree.. Then we have a chart in 'polar coordinate'. and then we have a better structure... nodes are the 'axis' and we have line that connect the nodes.

For drawing the nodes, we can just do that ( basic idea... ) :

var ga = svg.append("g")
 .attr("class", "a axis")
 .selectAll("g")
 .data(d3.range(0, 360, 15))   <= instead of 15, we write nodes.length 
  .enter().append("g")
  .attr("transform", function(d) { return "rotate(" + (d-90) + ")"; });

see this for example : http://bl.ocks.org/ufenegga/6f26d011aebc4c75272b ( that's simple chart in polar coordinate)

For drawing the lines we can rely to a line generator ( i need to dig that ... ).. After that, the nice thing is that changing the node order is very easy, we just need to redo a data-join, and we can use some transition for smooth... transition...

I'll take some time tomorrow to see if it works

[chabb]

Ok.. Please forget my comment.. i think a find an elegant way to move switch nodes and lines. So i'll explore a bit more

[RasmusFonseca]

"interactions" changed to "edges" ✔️

I'm working on moving nodes around as well. Ill let you know if I figure anything out. .. I'm not touching the reordering for a couple of days =)

[chabb]

Ok.. I have some ideas for reordering, so i pushed to whole stuff :

http://rasmusfonseca.github.io/EvoBundle/chabbeydrag/

There is a random reordering of nodes, and you can drag nodes around. Note that we are directly playing with the x and y coordinate.. which is not the 'best' way to do it... but it's enough to play with the charts..

I have another idea.. (See other issues...) i think we can generate a legend, that would allow the user to select a particular cluster (if the user click on the 'red' legend, we will only the show the links that comes/go to 'red' nodes)

[RasmusFonseca]

Wooow how crazy is that transition!! The reordering looks really good, though there are some control points that needs to get updated. For our applications I'm not sure manually dragging nodes would ever be a good idea.

I have an idea for clustering based on connectivity that I think will work well with these types of transitions.

[chabb]

Thanks Rasmus,

I am a bit playing and trying 'everything', that allows to me better understand how these layouts work, i know D3, but have never used cluster and bundle layout. So.. yes moving manually the points is definitively not useful... i am more thinking about some 'predefinite' clusterings that the user can switch

Ah.. the control points... this has puzzled me. and given birth to some debate between me and AJ :). I think they are right.. these knots and strange artifacts that you see, i can explain them :

if you look at the 'splines' array, what do we have ? ( i am using the mor-active-rep1-wb.json )

let's look at the 5x58 - 7x53 links, the corresponding element in the splines array is an array with 5 elements, it's basically the path from 5x58 To 5. To TreeRoot to 7. To 7x53... d3 use the coordinate of the 'intermediate' node for the control points. If I move manually 5x58 to 2x66, a knot will appear.... because d3 will use the 5 control points then the 'root' control point and then the '7' control point.

the only way to avoid that would be to say that now 5x58 below to cluster, so it will be 2.5x58.. then it will use these control points 5x58 -> 2 => TreeRoot => 7 to 7x53... and then it will be nice

The underlying 'issue' is that we break the clustering by moving these guys (see what happens when we move them randomly)... so we need a way to keep this clustering (or move to another clustering)... we just transition from a specific clustering to another specific clustering.

The trick would be to update the splines array manually (instead of having a brute force approach)... if we do that we do not have to touch the rendering code.. it should work...

So one of my idea would be to have a way to specify different clustering over the same set of nodes, and let the user switch between them..

And i am think about the ordering of the cluster... actually the clusters have a 'parent-key' ordering.. maybe we could use different ordering ( based on a given properties of a cluster )...

[chabb]

And i could not resist... i pushed a small thing... just use the 'mor-active' and wait...

the viz morph into a tree that gives you the clustering... it's quite ugly right now... but that could be intersting.. ( we could add color to three.. use the tree for some storytelling.. i think it makes the clustering more clearer )

of course, the animation is 'wrong' because it makes you thinks that the links of the tree are related to the links of the circle.. which is wrong.... the links should go in the bottom of the tree from leaves to leaves... again.. it's just there for giving ideas ( and by doing, i have clearer understanding of the underlying data structures.. )

[RasmusFonseca]

Haha, I know exactly what stackoverflow thread youre looking at =)

I'm trying to think of a good method for reordering nodes according to "connectivity". Essentially we want strongly interacting residues to be close, so my idea is to define "interaction-strenght" between two nodes as the average number of edges between the nodes over all frames (thatll be a number between 0 and 1). Now define the "interaction-distance" between two nodes as 1-interactionstrength and perform hierarchical clustering. That should group together strongly interacting clusters.

Problem is the HC approach doesnt give a fixed-depth tree with all nodes at the same depth, so things will be spaced oddly. This is what I'm working on now.

One thing that struck me about your tree-visualization is that it might be a good summary plot for the raw HC. Essentially a "Summarize strong interactions"-button that produces your tree.

[chabb]

:)

i forget that we have this evolution over time ^^..

which leads to some question/idea.. is this possible that the clustering evolves over time ?.. let's suppose that you have a clustering algorithm that iterates over all the frames... we could show the evolution of clustering... ( just another idea.. )