Ideas about MPR and 3D

[xhf127]

I want to pass the NII file to the browser, and use xtk to display 3D.

In registerImageLoader, I want to use xtk or other to cut AXIAL, SAGITTAL, CORONAL, transform data to Uint16Array assign to getPixelData in image, and other attributes in Image, I can read NII files.

I see an example of webgl in cornerstone, but it seems to be wrong.

But I don't know much about xtk. If you have a better idea about MPR and 3D, please tell me, thank you.

[dannyrb]

Our team has managed to accomplish this by creating a tool that works with a stack. It creates a volume from that stack (not using xtk), and then generates new slices / images with IDs that correspond to the input that was required to create them. We're not using an imageLoader.

I would be very interested to see an implementation that uses xtk.js to do this. I have had a hard time finding examples / documentation specific to DICOM / MPR.

I am a web application developer, so my expertise could very well be the issue. If you were aware of more approachable xtk.js or slicer examples, I'd be happy to take a stab at a POC I could share.

[swederik]

Both will work, you can load your Nifti into an ndarray with nifti-js (https://github.com/scijs/nifti-js) and then write an image loader which takes in some volume ID and slicing information and picks out (https://github.com/scijs/ndarray#arraypickp0-p1-) the slice that you want for your 2D image.

You're right, the WebGL Example for Cornerstone only deals with accelerating the 2D rendering. It has nothing to do with this and won't help you.

[xhf127]

Thank you for your help. So about 3D, I can only show it with xtk.js or others, not based on cornerstone?

[swederik]

Not really. You can write image loaders to display any image, but Cornerstone does not include any approaches for generating 3D views of volumes.

[xhf127]

Cornerstone can only render 2D? So I need to generate 3D and render it?

[medihack]

@xhf127 Yes, that's right. Cornerstone can render 2D images (or a stack of those) using a WebGL enabled canvas (for performance improvements) but doesn't support 3D volume rendering like in this XTK example. 3D volume rendering is quite a heavy and difficult task (most often using a raytracer that scans the volume) and is not to be confused with just rendering an already given surface. If you really want to take a deeper look at it then there is also X3DOM and AMI with some support for this.

[Revln9]

Hi , i made a volume rendering plugin for cornerstone using three js , it takes a dicom stack (100 frames for min quality ) and renders a 3d representation using the ray-marching technique , i can't really drop the code because i wrote it for the company i work for, but i'll ask for permission to do so.

The main issues i had to deal with is performance and transfer functions . to render a relatively good quality 3d models you need a lot of things, shaders , trilinear texture filtering , lights , shadow casts , multidimensional transfer functions , clipping planes ...etc but before you step into that , you need to load 512x512x100 pixels using web technology (javascript and canvas) in a matter of seconds , and that's the hard part , i used a grid of 16 by 16 frames in a canvas to load the pixels into the shaders.

If you want to dig deeper, you need to know GLSL , you also need to investigate volume rendering techniques , like single pass double pass ray casting , iso surface rendering , and volumetric light scattering , shading...etc. the only good source to learn about new techniques on these fields is to read the scientific papers using google , but it takes time and a lot of trials/errors to understand and apply concepts , so patience is the name of the game .

I'm planning to make a 3d boilerplate example using cornerstone and three js, so stay tuned.

Hope it helps

[dannyrb]

@Revln9 -- Well, that's incredibly cool (: If you send me an e-mail to danny.ri.brown@gmail.com, I'd be more than happy to add you to our maintainer/contributor Slack group?

Also, if you have any reading material suggestions, I'd appreciate hearing them.

[Revln9]

thank you @dannyrb , i'll send you an email soon , you can find here a couple of readings that really helped me:

http://prideout.net/blog/

http://graphicsrunner.blogspot.com/search?q=volume+rendering

https://github.com/nopjia/WebGL-Volumetric/

ftp://ftp.informatik.uni-stuttgart.de/pub/library/medoc.ustuttgart_fi/BCLR-0006/BCLR-0006.pdf

http://www3.cs.stonybrook.edu/~igutenko/webglvolrender.html

http://nopjia.blogspot.com.ar/2012/09/webgl-volumetric-renderer.html

[dannyrb]

Before I close this, it's worth noting that we are now aware of multiple modified Cornerstone instances that support 3D and/or 2D MPR. It is unlikely that you will see 3D MPR support added; however, a basic 2D MPR implementation is a possibility.

Our hope is that with v3 and the addition of 3rd party plugs, that you will start to see advanced functionality like this made available through free/paid plugins; filling the gap between what is reasonable for us to maintain, and frequently requested advanced functionality.

[amrita-syn]

Hi @dannyrb, is there any update on the 2D MPR? I would want to contribute but have limited understanding of volumes and graphics. The feature is very much awaited.

[bang6777]

Hi , i made a volume rendering plugin for cornerstone using three js , it takes a dicom stack (100 frames for min quality ) and renders a 3d representation using the ray-marching technique , i can't really drop the code because i wrote it for the company i work for, but i'll ask for permission to do so.

The main issues i had to deal with is performance and transfer functions . to render a relatively good quality 3d models you need a lot of things, shaders , trilinear texture filtering , lights , shadow casts , multidimensional transfer functions , clipping planes ...etc but before you step into that , you need to load 512x512x100 pixels using web technology (javascript and canvas) in a matter of seconds , and that's the hard part , i used a grid of 16 by 16 frames in a canvas to load the pixels into the shaders.

If you want to dig deeper, you need to know GLSL , you also need to investigate volume rendering techniques , like single pass double pass ray casting , iso surface rendering , and volumetric light scattering , shading...etc. the only good source to learn about new techniques on these fields is to read the scientific papers using google , but it takes time and a lot of trials/errors to understand and apply concepts , so patience is the name of the game .

I'm planning to make a 3d boilerplate example using cornerstone and three js, so stay tuned.

Hope it helps

Can you send this example to me? I am using AMI and Three.js libraries. It can resolve my problem. Thanks bangb1606777@student.ctu.edu.vn

[pieper]

If you are interested in another approach, here's info on an implementation using vtk.js

https://discourse.vtk.org/t/new-free-video-tutorials-vtk-js-and-ohif-web-apps/2164

[florentinGo]

HI @Revln9, I am currently trying to implement a volume renderer using extracted data from dicom. I already implemented a mpr viewer. Can you give some tips or advice from your personal experience on the subject ? You can contact me at florentin.gonthier@gmail.com.

[getnsv]

Hi , i made a volume rendering plugin for cornerstone using three js , it takes a dicom stack (100 frames for min quality ) and renders a 3d representation using the ray-marching technique , i can't really drop the code because i wrote it for the company i work for, but i'll ask for permission to do so.

The main issues i had to deal with is performance and transfer functions . to render a relatively good quality 3d models you need a lot of things, shaders , trilinear texture filtering , lights , shadow casts , multidimensional transfer functions , clipping planes ...etc but before you step into that , you need to load 512x512x100 pixels using web technology (javascript and canvas) in a matter of seconds , and that's the hard part , i used a grid of 16 by 16 frames in a canvas to load the pixels into the shaders.

If you want to dig deeper, you need to know GLSL , you also need to investigate volume rendering techniques , like single pass double pass ray casting , iso surface rendering , and volumetric light scattering , shading...etc. the only good source to learn about new techniques on these fields is to read the scientific papers using google , but it takes time and a lot of trials/errors to understand and apply concepts , so patience is the name of the game .

I'm planning to make a 3d boilerplate example using cornerstone and three js, so stay tuned.

Hope it helps

Hi @Revln9,, I was so glad to come across this post of yours and can become a life save for cornerstone users. Currently am trying to volume render some dicom images after extraction. Please can you send your boilerplate 3d code to my email - sreevani.nagendran@gmail.com please

Very gratefully, Sreevani

[krishnauppili]

Hi , i made a volume rendering plugin for cornerstone using three js , it takes a dicom stack (100 frames for min quality ) and renders a 3d representation using the ray-marching technique , i can't really drop the code because i wrote it for the company i work for, but i'll ask for permission to do so.

The main issues i had to deal with is performance and transfer functions . to render a relatively good quality 3d models you need a lot of things, shaders , trilinear texture filtering , lights , shadow casts , multidimensional transfer functions , clipping planes ...etc but before you step into that , you need to load 512x512x100 pixels using web technology (javascript and canvas) in a matter of seconds , and that's the hard part , i used a grid of 16 by 16 frames in a canvas to load the pixels into the shaders.

If you want to dig deeper, you need to know GLSL , you also need to investigate volume rendering techniques , like single pass double pass ray casting , iso surface rendering , and volumetric light scattering , shading...etc. the only good source to learn about new techniques on these fields is to read the scientific papers using google , but it takes time and a lot of trials/errors to understand and apply concepts , so patience is the name of the game .

I'm planning to make a 3d boilerplate example using cornerstone and three js, so stay tuned.

Hope it helps

Hi @Revln9 , Currently I have used cornerstone tools for scroll, zoom and measurements and I want to integrate 2D/3D MPR in the same. Could you please send your code to krishna.uppili96@gmail.com for reference?

[longerking]

Hi , i made a volume rendering plugin for cornerstone using three js , it takes a dicom stack (100 frames for min quality ) and renders a 3d representation using the ray-marching technique , i can't really drop the code because i wrote it for the company i work for, but i'll ask for permission to do so.

The main issues i had to deal with is performance and transfer functions . to render a relatively good quality 3d models you need a lot of things, shaders , trilinear texture filtering , lights , shadow casts , multidimensional transfer functions , clipping planes ...etc but before you step into that , you need to load 512x512x100 pixels using web technology (javascript and canvas) in a matter of seconds , and that's the hard part , i used a grid of 16 by 16 frames in a canvas to load the pixels into the shaders.

If you want to dig deeper, you need to know GLSL , you also need to investigate volume rendering techniques , like single pass double pass ray casting , iso surface rendering , and volumetric light scattering , shading...etc. the only good source to learn about new techniques on these fields is to read the scientific papers using google , but it takes time and a lot of trials/errors to understand and apply concepts , so patience is the name of the game .

I'm planning to make a 3d boilerplate example using cornerstone and three js, so stay tuned.

Hope it helps

I have used cornerstone tools for scroll, zoom and measurements and I want to integrate 2D/3D MPR in the same. Could you please send your code to longerking@foxmail.com for reference? Very gratefully!

[kawthar-bensalah]

hi ! @Revln9 can you please send me code or libraries used thank you : etudianteinfo45@gmail.com

[BARREEric]

Hi, @Revln9 is it possible to exchange some messages with you too ? email me at nojikosama@gmail.com if possible ! See you.

[yangAlan]

hi, @Revln9 . Could you please send your code to yangalan0914@gmail.com for reference? Very gratefully

[zhan2016]

hi, @Revln9 , appreciate for your work. Could you please send your code to ronzhang2015@gmail.com for reference? Very gratefully

[faizan170]

Hi, @Revln9. I am tryinh to implement this using threejs with image slices. But having issues with textures. Can you share any refrence url for code for reference? faizanamin192@gmail.com

[ldminh93]

Hi , i made a volume rendering plugin for cornerstone using three js , it takes a dicom stack (100 frames for min quality ) and renders a 3d representation using the ray-marching technique , i can't really drop the code because i wrote it for the company i work for, but i'll ask for permission to do so.

The main issues i had to deal with is performance and transfer functions . to render a relatively good quality 3d models you need a lot of things, shaders , trilinear texture filtering , lights , shadow casts , multidimensional transfer functions , clipping planes ...etc but before you step into that , you need to load 512x512x100 pixels using web technology (javascript and canvas) in a matter of seconds , and that's the hard part , i used a grid of 16 by 16 frames in a canvas to load the pixels into the shaders.

If you want to dig deeper, you need to know GLSL , you also need to investigate volume rendering techniques , like single pass double pass ray casting , iso surface rendering , and volumetric light scattering , shading...etc. the only good source to learn about new techniques on these fields is to read the scientific papers using google , but it takes time and a lot of trials/errors to understand and apply concepts , so patience is the name of the game .

I'm planning to make a 3d boilerplate example using cornerstone and three js, so stay tuned.

Hope it helps

can you send me this sample code to minhld93@gmail.com? i'm a new guy to research this topic so it very hard with me

[jmhmd]

For anyone else who comes across this thread - 3D rendering is now built in to the latest version of cornerstonejs, also known as Cornerstone3D. Please see this repository: https://github.com/cornerstonejs/cornerstone3D/

All future development on 3D features will occur there.