Closed oeway closed 4 years ago
We can certainly overlay a new canvas
element, e.g. the scale bar that is new in https://imjoy.io/lite?plugin=oeway/ImJoy-Plugins:itk-vtk-viewer (near the bottom) is a separate canvas overlay.
That said, like you suggested are very powerful and necessary, we are developing native widgets for 3D annotation in vtk.js and itk.js, which need to be integrated. I will follow-up via email with details.
Hi @thewtex while waiting the native annotation widget, I started to hack a bit by combining itk-vtk-viewer with OpenLayers. The idea is to enable 2D annotation with OpenLayers when the viewer is in 2D mode or showing a plane. My current implementation used itk-vtk-viewer as custom layer in OpenLayers.
Here is what I got:
This is the live demo: https://imjoy.io/lite?plugin=https://gist.githubusercontent.com/oeway/531c2f72ff712414f2229f9f8b522464/raw/6adfc0639aa3ca9c08aa64f74ba6daeec36f9f15/ImageViewer.imjoy.html The polygon annotation tool is enabled, so you can click on the screen to add new points to a polygon and close it by clicking the starting point.
As you will see, the viewer is not changing when you zoom or drag the plane, because it's not synchronized with OpenLayers yet (but the polygon annotations will change when you zoom). Would be great if you can help me with that. Specifically, I have the following two questions:
Thanks in advance.
quick update: I manage to solve my second question, which is I can now forward the mouse events to the itk-vtk-viewer.
I added a button, so I can now enable and disable the annotations in OpenLayers:
To move on, I would like to have some hint on how should we manipulate the camera to make the viewer match my openlayers coordinates.
Nice!
On my system (Linux / Chrome), the viewer slides in from the left:
This can be fixed by adding position to the div as discussed in https://github.com/Kitware/itk-vtk-viewer/issues/223
To move on, I would like to have some hint on how should we manipulate the camera to make the viewer match my openlayers coordinates.
Here is an example on how to get the camera parameters:
And more info on what they mean so they could be changed:
https://kitware.github.io/vtk-js/api/Rendering_Core_Camera.html
Hey, thanks for the tips, I will take a look and see if I can get it to work.
Now fixed the sliding bug: https://imjoy.io/lite?plugin=https://gist.githubusercontent.com/oeway/531c2f72ff712414f2229f9f8b522464/raw/c782e71b9b941dff1d37f81b0507819776e13d2a/ImageViewer.imjoy.html
@thewtex BTW, could you also give me some hint on how to convert the pixel coordinates and pixel size on the screen to the coordinates in the render, and further to the settings of the camera parameters? What I would like to do is to get the coordinates from OpenLayers, convert to pixel coordinates and pixel size, and then somehow get camera settings of the itk-vtk-viewer.
Hey @thewtex I made it! A preliminary version which synchronizes the two coordinate system:
However, while I can do camera. setParallelScale
to set the scale of the viewer, I cannot find a way to shift/translate the camera, I tried:
const viewFocus = camera.getFocalPoint();
const viewPoint = camera.getPosition();
camera.setFocalPoint([viewFocus[0]-diff_x, viewFocus[1]-diff_y, viewFocus[2]]);
camera.setPosition([viewPoint[0]-diff_x, viewPoint[1]-diff_y, viewPoint[2]]);
viewProxy.getRenderer().getRenderWindow().render();
( code: https://gist.github.com/oeway/531c2f72ff712414f2229f9f8b522464/533a0226c86cc14f7995b9f0c72c71ce23a92a8e/#file-imageviewer-imjoy-html-L207-L208 )
The issue of the above code is that the entire display gone if call render()
.
Do you know why?
Another thing is for the Zplane, openlayers uses lower-left corner as it's origin, but the viewer uses top-left. I had to do:
viewer.setViewMode('ZPlane');
viewProxy.updateOrientation(2, 1, [0, 1, 0])
But I think I can fix this if I find a way to set the orientation in openlayers.
More information on the camera parameters: https://kitware.github.io/vtk-js/api/Rendering_Core_Camera.html
The issue of the above code is that the entire display gone if call render(). Do you know why?
The camera clipping range may need to be set.
A few possible helpful snippets:
Hey @thewtex Thanks for your code snippets! I finally figured it out! It was because I passed an array instead of three arguments ;(
//wrong!
camera.setFocalPoint([xf, yf, zf])
camera.setPosition([xp, yp, zp]);
//correct
camera.setFocalPoint(xf, yf, zf)
camera.setPosition(xp, yp, zp);
Now it works like a charm : https://imjoy.io/lite?plugin=https://gist.githubusercontent.com/oeway/531c2f72ff712414f2229f9f8b522464/raw/9d750604cefed030df83a86646fa537f1423915e/ImageViewer.imjoy.html
A huge thanks to you!
We have been using OpenLayers for displaying images in some of the ImJoy plugins (e.g.: https://imjoy.io/lite?plugin=oeway/ImJoy-Plugins:ImageAnnotator), and we have been benefiting from a well equipped 2D annotation features provided by the library.
Now I started to think whether we can combine the itk-vtk-viewer with OpenLayers, say, create a new CanvasLayer in OpenLayers, and use itk-vtk-viewer to manipulate that layer. Here is an example about d3 integration in OpenLayers: https://openlayers.org/en/latest/examples/d3.html Essentially, we need to provide a
render
function to get synchronized.Could you please tell me what do you think about that? Or do you have a native way of supporting annotation in the viewer?
If that would work, I think it's gonna be a very powerful viewer with annotation support. Annotating 2D images should work without much tweaking, and for 3D data, we can at least support plane based annotation.
@thewtex