search

widgetti / ipyvolume

3d plotting for Python in the Jupyter notebook based on IPython widgets using WebGL
MIT License
1.93k stars 234 forks source link

[feature request] local maximum intensity projection `volshow` #96

Open eldad-a opened 6 years ago

eldad-a commented 6 years ago

It would be great to have volshow present Local maximum intensity projection and Maximum intensity projection, primarily for viewing 3d microscopy data. My impression is that these projections are very common in the modern Life Sciences literature, and so many in the community are used to this type of visualization.

maartenbreddels commented 6 years ago

Ok, great for letting me know. Do you want to try to implement this? Happy to help out.

eldad-a commented 6 years ago

Yes, I'm happy to contribute, if I can

How should we proceed? In which part of the code there access to the 3d array to be projected?

maartenbreddels commented 6 years ago

Some less relevant information, but useful maybe for understanding: 3d texture support does not exist for WebGL (version 1), therefore the image cube (with intensities) is tiled on a 2d png image https://github.com/maartenbreddels/ipyvolume/blob/master/ipyvolume/serialize.py#L54 In the alpha channel we put the intensity (scaled between 0 and 255), and the rgb colors contain the gradient (useful for lighting effect). This may be subject to change. When the data arrives at the browser, this code sets up the shader uniforms.

More relevant: The shaders used for this material do most of the work, actually almost all of the magic is in the fragment shader The function sample_as_3d_texture implements the 3d texturemapping in software, and line 122-125 is the actually computation of the colors. As you can see, the code is not super neat, it contains lots of commented out lines used for debugging, feel free to clean up some of that as well, or leave it as is.

I suggest you first try implementing something like LMIP by changing those mentioned lines, just hardcoding some values, feel free to make a PR (if you have questions) so I can see the code. If there are use defined values that need to be controlled, take a look how ambient_coefficient is handled in widgets.py, figure.js and the shader, that should give you some ideas. I think you can support different rendering methods in the same shaders by using the preprocessor macro's, like in the shader: https://github.com/maartenbreddels/ipyvolume/blob/595caaf5be93e9b3d21e35a2e925c1f37b1fe179/js/glsl/mesh-vertex.glsl and by making multiple materials, like i do here.

Hope this is useful, feel free to ask questions!

maartenbreddels commented 6 years ago

Just for clearity, max intensity is now in, not local max intensity.

eldad-a commented 6 years ago

@maartenbreddels Thank you so much for implementing this new feature!

Sorry for my delayed response (and not contributing yet). Recently I had to focus on 2D data and, unfortunately, haven't had the chance to explore this further.

I hope to get back to this soon and to have some useful input.