Ultratube visualization

[richford]

This PR implements visualization of the core fiber as a tube.

Resolves #199 Partially resolves issue #163 Related to #189

Continuing the discussion from #163, please play around with this and see what you think.

[arokem]

This looks good to me. What do you think about @jyeatman's idea here?

https://github.com/yeatmanlab/AFQ-Browser/issues/163#issuecomment-346056461

As mentioned there - I think that this works as well. I'd be interested to hear what @jyeatman thinks about these core fibers - it looks very 'sparse' to me, but my prior might be off.

[richford]

Latest commit resolves #203.

[richford]

@arokem, @jyeatman: My thoughts on the independent sliders for the streamline opacity and the core fiber opacity:

We actually have four independent groups of objects to control here:

1. non-selected bundle streamlines (i.e. grey bundles)
2. selected bundle streamlines (i.e. color bundles)
3. non-selected core fiber tubes (i.e. grey tubes)
4. selected core fiber tubes (i.e. color tubes)

Currently, the bundle view decision tree is (\___ = user input):

You want cores or streamlines?
\___ Streamlines
     Okay, what do you want the grey bundles to look like?
     \___ User selects bundle opacity
\___ Core Fibers
     Okay, what do you want the grey core fibers to look like?
     \___ User selects core tube opacity

Since we only display one fiber representation at a time, we can get away with two controllers.

@jyeatman, I like your idea of allowing the user to see both streamlines and core fibers. But I also think that more gui controllers = more cognitive load for the user, reducing the overall experience.

If we want to display four groups of objects at the same time we'd need

\___ User chooses opacity for unselected bundle streamlines
\___ User chooses opacity for selected bundle streamlines
\___ User chooses opacity for unselected core fiber tubes
\___ User chooses opacity for selected core fiber tubes

Or maybe we could get it down to three controllers

\___ User chooses opacity for selected bundle streamlines
\___ User chooses opacity for selected core fiber tubes
\___ User chooses relative opacity for all unselected items (both tubes and streamlines)

So that the difference in opacity between the greyed out tubes and the greyed out streamlines would be inherited from the difference in opacity between the selected tubes and selected streamlines. This has the benefit of reducing the number of controllers, but it may be harder to communicate the effect of the controls within the limited space of the control panel.

As you can see, I'm not sure what the best approach is. It'd be nice for the user to explore the entire representation space (i.e. core fibers and streamlines) but with minimal cognitive load about how to do that. What do you all think? I'm happy to mock up various options using separate PRs.

[richford]

Commits 2268cab and fbc53ac are mostly bike-shedding commits. They also resolve #202.

[jyeatman]

I really like the last option that you proposed. I think it provides optimal functionality with minimal controls. And it's nice to maintain the same opacity ratio between core and bundle for selected and unselected.

_ User chooses opacity for selected bundle streamlines _ User chooses opacity for selected core fiber tubes ___ User chooses relative opacity for all unselected items (both tubes and streamlines)

[richford]

After discussing the "non-toggling" version, @jyeatman, @akeshavan, and I decided to go with this version of the ultratube visualization.

[arokem]

Looks good to me!

[richford]

Pushed one more commit to make this compatible with streamlines.json files that have only core fibers. In that case, it does not generate the streamline geometries and the "fiber representation" drop-down menu has only one choice. I think this is finally ready to go.

@arokem, @jyeatman, @akeshavan, @anotherjoshsmith: Could you all play with selection in the 3D view a little bit. Remember from our discussion today that the new invisible convex hulls make the frame rate much better, but introduce some occlusion in bundle selection. I argue that this is a good trade-off since the entire point of the 3D window is to be able to orbit around the anatomical view to get around occlusion (and the user can always select using the checklist panel). But if you think this change is a deal breaker, I can revert that part.

After that last check, I think we are good to go.

[jyeatman]

The convex hull operation does make it hard to select some of the fiber groups that are behind. For example the corticospinal tract behind the SLF. I forgot that a convex hull does not allow any concavities (I know that should have been obvious from the name). But I also think it is still worth it for the speedup of the visualization, unless there is an easy way to "shrinkwrap" the fiber group such that it is turned into a single object that has concavities

[richford]

I went down the alpha shape rabbit hole before. I'll look into it again a bit but my suspicion is that it will not be feasible. For one thing, the computational cost will be higher. There's this library if we want to do it on the javascript side. Or we could use plotly to pre-generate the shapes in python and then just read them into the javascript.

I'll look into it a bit more and get back to you.

[richford]

Okay, here's the deal.

There are a number of alpha shape generation libraries out there but only a few that handle 3D. There's this for javascript. You can play with an example in the browser. Just paste

var alphaShape = require('alpha-shape')

var points = []
for(var i=0; i<10; ++i) {
  points.push([Math.random(), Math.random(), Math.random()])
}

var cells = alphaShape(0.1, points)

console.log(cells)

into the console on that page. You can see the output for ten points there. But we have ~38,000 points for a single bundle. And the javascript alpha shape generator doesn't scale well enough for that. It's not feasible to make the user wait to generate alpha shapes for each bundle every time they refresh the page.

Then I thought that we could generate the alpha shapes on the python side, as part of the assemble step. Later on, when we implement different streamlines for each subject, this could take a while. But maybe we could get around that by just generating a canonical set of alpha shapes and shipping those with AFQ-Browser. @arokem, @jyeatman is the variance in bundle geometry small enough that a canonical set would do for all subjects?

For pre-generation of alpha shapes, plotly generates alpha shapes, but I can't find a way to get the Mesh3D faces out of plotly. So I think we'd have to use some other tool (e.g. dionysus or CGAL).

In any case, I think this is becoming a bigger project beyond the scope of this PR. I recommend we open a "create canonical alpha shapes" issue (not in the revision milestone) and leave this as an open project for the future.

In the meantime, I don't think the occlusion is too bad with the convex hulls.

[jyeatman]

I agree that it is fine for the meantime

[richford]

K. I think this is ready then. I'll open up a new issue to convert the convex hulls to alpha shapes.