Corresponding points in MPR

[JoyBoy-369]

Hi Nicolas,

It would be really cool if AMI were to show a point if selected on the axial plane of a slice , the corresponding point and slice would be shown on coronal and sagittal plane. Similarly, to what papaya has achieved.

[NicolasRannou]

Hi @SevenSinS02 - yes that is not hard to implement but I am not sure which is the best way to do it... (see related issue #70).

With the THREEJS raycaster we can get the intersection point in LPS space between the cursor and the image, the question is which is the best way to display it on top of the WebGL canvas: canvas 2D, dom elements, SVG...

Right now I am thinking that we can associate a "localizer" widget to the webgl canvas and this widget would be in charge of painting the lines.

Any thougts?

[JoyBoy-369]

Hi @NicolasRannou - I think that the widget is the way to go too. I feel giving the users the option to do enable/disable the localizer would be a good thing. All of the options seems viable. Canvas usually gives the best performance. SVG's would be easier to build with. Dom elements are usually not that far up in the performance scale when considering that you want to draw something(although this could be considered a simple object). These are my thoughts and I might be wrong. Please do tell me if I'm wrong and it would be great if an explanation was given as well. Always ready to learn! :) Thanks for a great library!

[JoyBoy-369]

Hi @NicolasRannou - I've achieved the part where clicking on a point switches to the corresponding slices in the other views. However, I have no clue how to go about showing the crosshair. Any ideas?

[NicolasRannou]

Awesome!

So I think the best approach right now (to best tested though) is to add an extra pass in the shaders to draw cross/box intersection if needed. That will be the fastest to implement at least, if doesn't look good, is not flexible enough or too slow we can consider other approaches.

My idea right now is that you would give the box dimensions, colors and transforms you want to draw to the shader and it would paint it as expected. and maybe allow 10 to 50 boxes to be drawn.

Drawing a slice would be the same as drawing a box with height = 0;

Some pseudo code to try to make it more clear: In Javascript:

on render, update the box center/dimensions

In the new shader:

uBoxHalfDimensions = vec3...
uBoxCenter = vec3...
uWorldToBoxTransform = matrix4...
uColor = vec4

// check if current pixel is on the box border, if so, draw it
...

Do you see what I mean?

[JoyBoy-369]

I haven't really used shaders before, so I'm still a bit confused. Are we trying to make the crosshair out of box borders?

[NicolasRannou]

Yes, because it should ideally be more that cross hairs: random orientation, boxes, etc.

See screenshot at https://github.com/FNNDSC/ami/issues/70

To make a cross I imagine would just do something like: 'hey in this renderer we are going to draw 2 lines in direction X and Y'

To draw a box, would be 4 lines in direction X, Y, Z, D...

Then we could also draw N parallel lines, etc. And should be super fast in the shaders...

Try to upload some data there: https://fnndsc.github.io/ami/#viewers_upload

Then drag the slice out of the screen, it will show a "dashed line" where the slice get cropped. The point there is that it is very fast and I'd like the cross hairs to be super responsive too, hence in the shaders.

[JoyBoy-369]

Oh ok. I didn't see the dashed line before. I must have missed it. So, the shader is responsible to draw lines or boxes and we would have to pass the stack information and also the shape(line/s or box) to draw? The shader would then draw them on the screen. Have I got it correct so far?

[NicolasRannou]

Yes the dashed line is new :)

and yes that is the big picture - I'll try to add one example to have something we can build on -

[JoyBoy-369]

Oh that would help speed the process. I'll put more effort in learning three.js to avoid asking so many questions. :p

[NicolasRannou]

It is almost there! Downside is that it needs some boilerplate code but it will be pretty flexible. For now we will just draw lines (not segments and boxes) and we can extend it when needed!

[JoyBoy-369]

Oh that's cool! Can't wait to get my hands on it. ;)

[NicolasRannou]

FYI still working on that, need to clean up the API a bit still but mostly working now. Will add it as part of the "quadview" example, plus some other cool stuff :)

[JoyBoy-369]

Oh cool! That's amazing. I love surprises :p

[NicolasRannou]

See https://fnndsc.github.io/ami/#viewers_quadview :)