Support for Image Fusion such as PET/CT or Pixel Image Overlays

[Ouwen]

I was wondering what the status of fusion support is. Noticed a stale support question (#502) but wasn't sure if there has been resolution. This feature could also be used to support heat map outputs from prediction models (#1989).

Ideally, the cornerstone layers API could be leveraged to solve this, but it does not seem like the OHIFCornerstoneViewport will play nice with the layers API.

Is there a suggested pathway to support features like #502 and #1989?

[ArturRod]

Hello @Ouwen As I have seen that it was not progressing, I decided to carry out this implementation on my own. He made a view like the one in 'CINE' which is for PET / CT fusion. And then apply the methods: https://rawgit.com/cornerstonejs/cornerstone/master/example/layers/index.html

[PattersonOfficial]

Thing is @ArturRod, following the layers example, it works quite fine but when you those images load with additional images, you can`t scroll over it. Is there a way to handle this?

[ArturRod]

Hi @PattersonOfficial
AHe is currently consulting to see if this was possible, as it is only allowed with one CT and one PET image. It does not support multiple image stacks at this time. (At least if modifying cornerstone) I'm waiting to see if someone can help me ... (https://github.com/cornerstonejs/cornerstone/issues/560)

What I did find is this, I will try to implement it if possible. But it would be better if they added something from cornertonejs. http://www.dvisionlab.com/Larvitar/examples/layers.html

[PattersonOfficial]

@ArturRod thanks for the feedback. The thing that keeps me wondering is that the second image that I add to the layer overlays quite well onto the main layer(image) except that the main image can still be scrolled through but the overlay image disappears when scrolling through the layer... Any advice?

[ArturRod]

Hi @PattersonOfficial From what you tell me, the layers are superimposed on you, but can you tell me that the main view remains as it is, being able to move, right? I think what you have to do is remove the view that is previously there and then generate the layers, since otherwise the main view remains together with the layers, so it allows you to move the image.

[PattersonOfficial]

https://user-images.githubusercontent.com/26417112/141091721-fef2cfe5-e188-4274-bc7d-992c34ed97c3.mp4

@ArturRod so basically the base image is already loaded automatically, then using cornerstone.addLayer(element, image); I add the overlay. The base layer is still scrollable but the overlay image disappears even though the layer is still there. There are 2 things I want to achieve and that is

1. Make sure that overlay image is also scrollable
2. Sync it so that once we scroll, both images scroll in a sync

[ArturRod]

@PattersonOfficial Hi again. 2 layers have to be loaded, with a 'standard' view and 1 layer does not work, since 2 layers have to be loaded. One for PET and one for CT. From what you show me in the video, load 1 layer on the main viewer, in the example that I tried (https://rawgit.com/cornerstonejs/cornerstone/master/example/layers/index.html) you have to load 2 layers, one with each image and its color and opacity options. The problem is that currently addLayer does not allow loading a stack of images. So try using this other method: (http://www.dvisionlab.com/Larvitar/examples/layers.html).

[PattersonOfficial]

@ArturRod I really appreciate your efforts and everything. Thank you so much for everything

[sedghi]

This is already supported in master in TMTV mode Try viewer-dev.ohif.org instead of viewer.ohif.org Our viewer.ohif.org is deployed from release branch while viewer-dev.ohif.org is our master branch Read more about branch explanations here https://docs.ohif.org/development/getting-started#developing