4D rendering

[tpowellmeto]

Firstly just want to say amazing project! I have been testing out the 3D rendering from dask arrays and it works awesome.

I was wondering if you thought it would be possible to add a fourth dimension to the data, say time. So the volumetric rendering could change with a time slider or something.

I have some experience doing this with three.js using a video as a texture with a custom shader.

Thanks for any guidance.

[thewtex]

@tpowellmeto thanks for the positive feedback!

Time-based visualization is a good idea. Do you have some data that we could make an example out of and can you share the three.js demo do get an idea of what you are going for?

[tpowellmeto]

@thewtex thanks for engaging on this.

Here is a demo of some volume rendering through time that we created in my team with three.js. http://demo.3dvis.informaticslab.co.uk/ng-3d-vis/apps/desktop/

I've had a further play with the widget and managed to cobble together a semi-working demo using an ipywidget to progress the fourth data dimension. One of the main sticking points I'm having here is that the color bar resets whenever the image data is changed.

I will organise some open data to share with you, in the meantime here is a demo of my progress. https://s3-eu-west-1.amazonaws.com/informatics-webimages/videos/index/jupyter_volume_render_nabro.mp4

[thewtex]

Here is a demo of some volume rendering through time that we created in my team with three.js.

Very impressive! I am surprised there were not more clouds over the UK. :partly_sunny:

I've had a further play with the widget and managed to cobble together a semi-working demo using an ipywidget to progress the fourth data dimension. One of the main sticking points I'm having here is that the color bar resets whenever the image data is changed.

I will organise some open data to share with you, in the meantime here is a demo of my progress. https://s3-eu-west-1.amazonaws.com/informatics-webimages/videos/index/jupyter_volume_render_nabro.mp4

Well done!

Yes, a smooth transition over time will be the challenge. This is a good start!

[vianamp]

Hi guys.

First congrats for the project!!

A related question/feature is the support for multichannel images. I work with multichannel microscope images and it would be very helpful if I could view many channels at once and turn some of them on and off.

Any thoughts?

[thewtex]

Hola @vianamp :wave:

We could look at adopting the same approach for efficient client-side switching of time points for switching image channels.

In the meantime, a ipywidgets could be used to select a channel to visualize. If you have some example data, we could add this to the set of examples.

What other methods do you use to view multiple channels?

[vianamp]

Hi @thewtex.

Here is a link to an example of data I work with. The data corresponds to a 2-channels 3D time-varying image that contains 19 time points. It represents the mitochondria of a yeast cell with two fluorescent-tagged mitochondrial proteins. We have a matrix marker in red and an outer membrane marked in green.

https://www.dropbox.com/s/us4s3012ndo6lj3/yeast_mito.tif?dl=0

I work with segmentation algorithms for 3D images and part of my work is to evaluate the performance different algorithms, which is very hard to be done in 3D. Therefore, I use Fiji to move between 2D slices along the x, y or z directions. Features that would be very helpful are:

1) Multichannel 3D volume viewer where I can select what channels I want to look at. 2) Option to have two images side by side such that when in 2D mode, the two images are in synch with respect to which slice is shown.

Let me know if you have any questions.

Thanks a lot.

[thewtex]

@vianamp thanks for sharing the data. I will work on an example that shown show to select channels from a multi-channel image. Issue created here: #85

A side-by-side comparison viewer is also something I would like. Issue created here: #86

[GenevieveBuckley]

It sounds like the suggestion for multi-channel data is to choose to view only one channel at a time, rather than rendering multiple channels - is my understanding correct?

Is it possible to do something like I've done here with ipyvolume and display multiple image channels with independently controllable transfer functions? Note that in the example below there are only two channels (displayed as red and green) but we could have four, five, or more channels in any given image.

[thewtex]

@GenevieveBuckley good idea! :bulb: Yes, we could select each transfer function independently per channel. And, we could provide an option to enable / disable the channels.

Depends on https://github.com/Kitware/itk-vtk-viewer/issues/122

We could also add a channels trait to select which channels to visualize.

[GenevieveBuckley]

Neat, thank you - I'll keep an eye on that Kitware issue.