integrate ontology overview using force directed layout

[ghost]

the prototype that shows mappings between a set of ontologies, where dots are relative to the ontology size and arcs are relative to the number of mappings.

• pull data (fixed because this is overview data, cached)
• add a new embedded view (defer to issue #207)

NB: Check #206 before merging to issue101 branch (or others used here) into master, and before closing this issue!

[everbeek]

109 and #101 are work Elena did on this. Bo says things were nearing completion, and that we need to pull the right json data for the view.

I think that #109 is using the relevant REST service, and that #101 deals with the view. Each has a branch, but both branches are out of date. I am going to decide if each branch is in a condition where I can merge them together (into a branch named after this present case perhaps). They also need to have the master merged in. I will start by doing that for each.

[everbeek]

Will keep working on this, subsuming #101 and #109, in issue101 branch. Ignore issue109 branch, now that it is merged into issue101. Clean them all up after merging to master.

[everbeek]

I've been making progress without making notes. Here is where I am at now, and a summary of what Bo and I have discussed. The embed version isn't a priority for this feature, but there was work towards that, so I am wrapping and mothballing that for later. The UI will require an Ontology Search (the same way we currently have concept search). All nodes in that view will be created only after the user drags an ontology resource from this search results list. We will need a second graph view that handles Ontologies (rather than nodes). I am currently working on the arcs for these ontology mappings, as well as the ontology search. I also need to make the graph view for this, or extend the existing one to be specialized for receiving ontologies rather than concepts.

One barrier will be arc thickness; it looks like that is set up so that all arcs have the same thickness, given a type. I will sort that out later.

My next immediate task is the search for ontologies, then the graph, then arcs. With other things in between, surely.

[everbeek]

NB: This graph view should be in D3, along the lines of the prototype. This might get revised if we later have a wrapper for D3, but we will have more immediate results if I do this as a one-off view, and will be good experience for me.

[everbeek]

Working on ontology search. This will require GUI component, ontology models, and service classes.

[everbeek]

Completed search functionality and GUI.

I have some uncommitted work on arcs for ontologies. I will review what I have so far, and see if there's some things to finish off with that, as I begin looking at integrating the prototype D3 view, or making a new D3 graph view for this.

[everbeek]

Read about JSNI, looked at the Protovis wrapper a bit, and checked out the d3_gwt demo, and the . Starting on specialized D3 graph view.

Need to work my way from the interfaces used by SvgGraphRenderer and GraphDisplayController, to see what I need to provide from D3. The latter class is the one used by Graph (client.visualization_component.graph.Graph.java), which means I need to introduce some indirection for use of GraphDisplayController via the DefaultDisplay static class seen in Graph. Actually, I need to make a second Graph class that uses D3, specifically for the Ontology Overview, and if D3 is later used for all graphs, I will need to combine it then.

[everbeek]

Working on the D3GraphRenderer, parallel design to the SvgGraphRenderer. It felt really wrong, so I stepped back for a bit. The existing GraphDisplayController (an SVG specific version of the GraphDisplay), or rather a parallel GraphDisplayControllerD3, is the class that uses GraphRenderer->AbstractGraphRenderer->[Svg][D3]GraphRenderer interface I am trying to fulfill. I think I can create D3GraphRenderer such that it accounts for the different way that D3 can work, but I had started making it extremely parallel to the SVG implementation. I should back up and fulfill the required interface without forcing the D3 usage to look like the interface. That's the point.

To summarize, I let the SVG implementation mislead me for a while. I will try a different approach that lets D3 be itself.

[everbeek]

Backing up a bit. Talked with Lars in person, and came to understand that using D3 for any graph view means not making use of any of the work put towards graph view. Graphs are already fairly mature, and things we like in D3 graphs can be copied or emulated easily enough while still preserving all of the strengths of the existing graph system in BioMixer.

Prior to heading towards a D3 implementation of the graph, I was beginning to see how to make a new graph type. I will get back to that now.

[everbeek]

Progress made on having two graph types. Set up unit tests too, but they aren't passing yet due to minor differences in the circular and boxed label node construction (e.g. starting position). Will complete the work prior to fixing the SVG files for the unit tests.

[everbeek]

Still working through the expander system in use already, and adapting a bulk expander for situations when we have an efficient bulk REST service available (like for ontology mappings).

[everbeek]

The bulk expander system is in place.

I need JSONP for the ontology mapping service. I have begun this process; I will clean up and commit what I have and do other tasks on this issue. (I have a few too many changes in the commit! Bad practice...)

[everbeek]

Paul sorted me out; there is a generalized JSONP service, but we discovered that if it is accessed via http it double-encodes the url, but if accessed via https, it works. See this example (but add the api key): https://stage.bioontology.org/ajax/jsonp?path=%2Fmappings%2Fstats%2Fontologies%3Fontologyids%3D1032%2C1099%2C1009%2C&callback=%3f&userapikey=&apikey=

[everbeek]

Did some squashing on my accidentally bifurcated commit. Hopefully it worked. The new commit is 5bb84ffd3d83a58848115340c14b4e764fb5e571. The additions and deletions add up between the bifurcated commits of the part I 79a2acdbe29585c21d812b4f5deabed302bd13b7 and part II 105b7c6ae4cd51a67e457bfa5a9f25a440aaa3c1 commits, so I think ti worked fine. It is not yet showing up in github though, and I understand sometimes it takes time...and I also missed the '#' on the issue number, since I was editing in Vi, and it was using '#' as comment line indicators. Unpleasant overall.

Commit includes arcs being rendered, unit tests (some as broken placeholders such as the loader and expander classes). Nodes and arcs need their size and thickness manipulated, and the circular nodes need to be revised backwards towards the original rectangular label node style, in everything but shape. Did refactoring to help see the difference between concept mapping expansion and ontology mapping expansion.

Why do I make comments here and in the commit? It feels right...

[everbeek]

Have more work on edges, but am moving on to revamp the circular nodes now, then on to node sizing based on the number of concepts in the ontology.

[everbeek]

To get the number of classes (concepts) in an ontology requires a call to the metric REST service: http://www.bioontology.org/wiki/index.php/Ontology_Metrics http://rest.bioontology.org/bioportal/ontologies/metrics/40133?apikey=YourAPIKey

This suggests that we should not grab these in the search results, because those are likely to be numerous; if we grab it when the ontologies are loaded into the graph, then we can at least defer to a smaller number, even though it gives us a delay in seeing that data renderer.

I wish they had a complete batch REST service that could wrap any exterior REST service call. Would this be reasonable to ask for, at all? It could be abused and lead to service disruptions (imagine making the largest and slowest request for the most numerous item, rather than making client-dependent multiple requests).

Lazy loading ontology size (metrics) seems acceptable given the alternatives.

[everbeek]

Need to fix resource set removal, cross-graph dragging, and get all the unit tests whipped into shape.

[everbeek]

Deal with the set removal bug, and unit tests. Oh, set removal was malfunctioning as an in-progress bug I think, because it works fine now.

[everbeek]

New bug: have to test repeatability, but it seems that if I try to drop on an invalid container, the next time I select a resource set, I get an error message with no useful information. It has happened a lot, and only during this issue. So, explore it and fix it here.

[everbeek]

New issue for the drag and drop problem. See #215.

[everbeek]

215 resolved. Merged branch issue101 to master. This is wrapped up! #214 is related but entered as a separate case, so we can close this issue.

[everbeek]

236 is work on the embed version of the ontology view, made for the use case of seeing the mapping neighbourhood for a central ontology node. Does not use all mappings within that neighbourhood, because neighbourhoods can (frequently?) contain very dense interconnections.