Major WebGL refactor

[jamesgao]

I'm starting a major webview refactor right now. To give an overview of what will happen:

1. Monolithic mriview.js file will be split into three components
2. Dataset.js will now represent some volumetric data in space, regardless of the cut surface
3. A new file mriview_surface.js will represent a surface visualization
4. Additional volumetric slicing options will simply use options in dataset.js to display

This refactor will allow two important things:

1. Simple integration into jsplot: mriview axes are now just standard threejs axes, with objects simply added to the scene. This will allow additional 3D plots in the same space as the brain
2. Additional volumetric displays will be easier to implement, for example MIP and individual slice planes

Some additional points:

• LandscapeControls.js functionality will be merged into some sort of a "manipulator" object. Instead of moving the camera around the brain, it will spin the brain in place.
• MultiViews will be supported by using only a single webgl canvas with a scene graph, rather than individual canvases. This will allow proper MultiView support, since memory will not be duplicated.
• Dropping all shader responsibilities to dataset.js conveniently allows better mobile support; if float texture extension is not supported, simply prerender the colormap and send in a raw dataset
• Remove native morphtargets from the surfaces, instead wrapping the functionality into the shader code. This will hopefully make the threejs upgrade smoother

Some things I haven't worked out yet:

• Scissor test? Should graph views support the scissor test to segment the viewport like multiple canvases do right now?
• Picking -- How to modularly support additional objects, such as slice planes?

[jamesgao]

After more than a year on hiatus, I think this issue is just about ready to be closed. The new glrework branch has a completely redone UI interface making it fully extensible. Also, the overlay system has been completely reworked to allow custom styling, multiple display layers, and an improved label system (less blinky).

Some things left to do before it's merged in as master:

1. Orthographic camera
2. Better UI for the dataset and colormap selection
3. Voxel line / sampling selection

Some wish list items:

1. Multiview support
2. Picking deferral (to send pick events to all objects in scene)
3. Other volumetric displays -- I have some stub code in mriview_surface.js which I need to reimplement for slice planes, volume renderings, etc.

[chrisgorgo]

Are there any API changes I should look into?

On Thu, Jan 15, 2015 at 1:42 PM, James Gao notifications@github.com wrote:

After more than a year on hiatus, I think this issue is just about ready to be closed. The new glrework branch has a completely redone UI interface making it fully extensible. Also, the overlay system has been completely reworked to allow custom styling, multiple display layers, and an improved label system (less blinky).

Some things left to do before it's merged in as master:

1. Orthographic camera
2. Better UI for the dataset and colormap selection
3. Voxel line / sampling selection

Some wish list items:

1. Multiview support
2. Picking deferral (to send pick events to all objects in scene)
3. Other volumetric displays -- I have some stub code in mriview_surface.js which I need to reimplement for slice planes, volume renderings, etc.

— Reply to this email directly or view it on GitHub https://github.com/gallantlab/pycortex/issues/48#issuecomment-70167446.

[jamesgao]

No, there are no changes in the backend. Dataset objects still work the same way, only the generated pages look different. I'll give you a warning before I merge anything, so you can make sure your pipeline works correctly with the new code!

Oh, two new features I forgot to mention -- Oculus Rift and Leap Motion compatibility!

[chrisgorgo]

No. F**king. Way.

On Thu Jan 15 2015 at 2:31:19 PM James Gao notifications@github.com wrote:

No, there are no changes in the backend. Dataset objects still work the same way, only the generated pages look different. I'll give you a warning before I merge anything, so you can make sure your pipeline works correctly with the new code!

Oh, two new features I forgot to mention -- Oculus Rift and Leap Motion compatibility!

— Reply to this email directly or view it on GitHub https://github.com/gallantlab/pycortex/issues/48#issuecomment-70175145.