search

Kitware / vtk-js

Visualization Toolkit for the Web
https://kitware.github.io/vtk-js/
BSD 3-Clause "New" or "Revised" License
1.24k stars 380 forks source link

[Feature] Share ressources between render window contexts #2866

Open finetjul opened 1 year ago

finetjul commented 1 year ago

Feature Request

It would be great to have a mechanism to share GPU memory between contexts.

Motivation and Detailed Description

Each render window has its own OpenGL context. Resources are currently not shared between render windows. In the case of a multi render window application, displaying 2D images via vtkImageResliceMapper, the whole 3D volume is being sent to the GPU in each render window. This consumes a lot of memory.

It appears that there is no way in WebGL to share contexts.

A solution could be to "proxy" all render windows into a unique offscreen render window that gather all render windows renderers, render them, and copy/paste their result image into the proxied render windows.

A current alternative is to create a unique render window for all the views, and an offseted vtkRenderer for each view.

daker commented 1 year ago

Not an expert but maybe SharedArrayBuffer can help?

pieper commented 1 year ago

Hi @finetjul - If I'm not mistaken OHIF has implemented the off-screen proxy option on top of vtk.js to render multiple views with only one webgl context (e.g. for this example with 10 views). It would be great to see if that code could be refactored into vtk.js itself to make it more generally available and to ease maintenance in the longer term. @sedghi can you comment?

finetjul commented 1 year ago

@pieper that would be great indeed.

sedghi commented 1 year ago

Yeah here is the custom vtk classes that we have, more specifically this part . I don't think SharedArrayBuffer would help though, it would help to decode in multiple webworkers at the same time to the same buffer to enable streaming.

I would love these custom classes to go upstream in vtk.js if they seem fit

sedghi commented 1 year ago

Steve shared a video, but here is the live demo of 10 viewports using two shared textures https://viewer.ohif.org/tmtv?StudyInstanceUIDs=1.3.6.1.4.1.14519.5.2.1.7009.2403.871108593056125491804754960339

floryst commented 1 year ago

@sankhesh can mention more about work regarding shared resources between render windows.

sankhesh commented 1 year ago

Thank you @pieper and @sedghi. I've been looking at the ability to share graphics resources across mappers between viewports within the same context. That way you could have something like https://kitware.github.io/vtk-js/examples/ManyRenderers/index.html with different pipelines sharing the same data. A side-effect to this is that applications can choose between two approaches - have multiple viewports (instead of windows) or have an offscreen window with multiple viewports that are then blitted to individual render windows on the onscreen canvas (ohif approach).