[Perf] Re-rendering multiple dicom images at once

shunia commented 4 days ago

High-level description

I'm building a filmer app that the customer would be able to interact with multiple dicom images at once for adjusting say ww/wc, pan, scale etc, and I was aiming for 60fps for smooth real-time interactions.

I'm using vtkOpenGLRenderWindow under the hood and creating multiple renderWindows and each window holds a dicom image (pixel data) for interactions, which is quite similar to the MultipleRenderWindow example, but with some modifications:

use custom managed input event handlers instead of vtk interactors
re-render by picking nessesary child renderWindows and call the .traverseAllPasses() api instead of rootWindow.render()
split render calls into batches with a size about 40 between frames (with requestAnimationFrame), and ensures each renderWindow will only be re-rendered once within the re-render calls

I'm stuck into a sutuation where in my developing machine I can only achive 60fps when re-rendering mostly 40 images at once with strategies implemented as above, when adjusting ww/wl with:

// the code is definitly much complex than this but the calling sequence is much like this
actor.getProperty().setColorWindow(ww);
actor.getProperty().setColorLevel(wl);
renderWindow.traverseAllPasses();

the hardware may be better in production machine but still the number is far less than what I would want to achive, which I would like to keep 60fps when re-rendering about 100 images at once, because the whole app would load over 2000 images at once (the loading performance and the first rendering performance is not a problem here)

With above senerios, what could I do to improve the performance? Or is there anything I do wrong? Is there ways to batch the re-rendering under the hood in vtkjs to squeeze the performance?

Steps to reproduce

It's somehow a little bit complicate to extract the logic out of my app so for now there's none re-produciable steps.

Detailed Current Behavior

Environment

vtk.js version: "@kitware/vtk.js": "^30.9.3",
Browsers: Chrome Version 129.0.6668.100 (Official Build) (64-bit)
OS: Window 10 Enterprise 10.0.19045 Build 19045

finetjul commented 4 days ago

What exactly does the screenshot represent ?

It's odd that you have to create a new texture each time you re-render. Do you confirm that updatelabelOutlineThicknessTexture() creates vtkOpenGLTexture at each render ? (fyi @sedghi)

Did you check that MultipleRenderer is any faster ? If your render windows do not need to share GPU resources, did you try to not make them shared ?

sedghi commented 4 days ago

when re-rendering mostly 40 images at once

You mean you have 40 textures and 40 render window? I'm not surprised if that does not achieved 60 fps

shunia commented 3 days ago

Hi @finetjul, @sedghi,

Thank you for your quick responses. I’d like to provide some additional context regarding our current rendering challenges.

Summary of Current Situation: Our application renders between 200 to 2000 renderWindows simultaneously, each displaying a DICOM image with adjustable parameters like window width/level, scale, pan, etc. Users can select multiple renderWindows to modify images collectively. However, achieving 60 fps with 2000 renderWindows is impractical. To address this, I am currently batching the render calls, processing 40 renderWindows per batch using the traverseAllPasses function for each renderWindow.

Questions and Clarifications:

Regarding updatelabelOutlineThicknessTexture: I have not called this API. What steps should I take to confirm this?
MultipleRenderer Approach: Implementing a MultipleRenderer is not feasible at this stage as our app is based on a MultipleRenderWindow architecture. Reworking the entire implementation is not an option right now.
Render Windows: While I mentioned having 40 renderWindows in the batch process, I need to clarify that I'm managing 200 to 2000 renderWindows simultaneously, rendering only 40 at a time to optimize performance.

I would greatly appreciate your feedback and any suggestions on alternative approaches we might explore. If MultipleRender is truly the only viable solution, I am open to considering it, despite the potential need for extensive codebase modifications. However, I hope we can identify other strategies to test before pursuing that route.

finetjul commented 3 days ago

If you have that many views in your single page application, it seems that you will get the best performances with having a unique render window with a unique renderer. You would then have many ImageResliceMappers... (or potentially a unique one with a tiled image you created on the cpu)

shunia commented 2 days ago

...with having a unique render window with a unique renderer...

Could you please elaborate more on this?

...You would then have many ImageResliceMappers...

I did not find any examples on how to use vtkImageResliceMapper, could you please direct me to resources/docs on how to use it?

finetjul commented 2 days ago

Could you please elaborate more on this?

const fullScreenRenderer = vtkFullScreenRenderWindow.newInstance();
const renderer = fullScreenRenderer.getRenderer();
const renderWindow = fullScreenRenderer.getRenderWindow();

for (let i = 0; i < 1000; ++i) {
  const mapper = vtkImageResliceMapper.newInstance();
  mapper.setSlicePlane(vtkPlane.newInstance({normal: ..., origin: ...));

  const actor = vtkImageSlice.newInstance();
  actor.setMapper(mapper);
  renderer.addActor(actor);
}

I did not find any examples on how to use vtkImageResliceMapper, could you please direct me to resources/docs on how to use it?

https://kitware.github.io/vtk-js/examples/ImageResliceMapper.html

Kitware / vtk-js