[Bug] ResliceCursor and ImageResliceMapper not working on older computers

[TimDeBauwReLu]

Bug description

When running the resliceCursorWidget or ImageResliceMapper live demos on an older pc (windows 10, low end nvidia gpu, intel i7), the widgets seems crash on the vtkPolyDataVS.glsl shader due to a lack of gpu ram.

Steps to reproduce

Open https://kitware.github.io/vtk-js/examples/ResliceCursorWidget.html on an older windows computer

Detailed Behavior

Would it be possible to have a cpu fallback option so that even old computers can be supported?

Expected Behavior

The reslice mapper should work the same as the vtkImageMapper, and not crash when too little resources are available.

Environment

• vtk.js version: 28.2.6
• Browsers: Chrome
• OS: Windows 10

[finetjul]

Could you please share the console log ?

It's odd because it's already low end... reslicing is performed with CPU and not GPU (i.e. the whole volume is never sent to the GPU).

[emil-peters]

Hi, I've been testing with this as well and when loading multiple big dicoms at once it crashes every time for me. I'll share some of the weird error messages I've been getting. When I have a running instance open, and replicate it by loading multilpe dicoms at once, the running instance also crashes.

I see that my GPU get's bottlenecked and crashes at about 10GB VRAM used. The following process is the culprit (after it crashed):

1589690 ?        Sl     0:00 /opt/google/chrome/chrome --type=gpu-process --crashpad-handl

While checking with nvidia-smi I see it creeps up the VRAM:

|    0   N/A  N/A   1589690    C+G   ...357159424649057971,131072     8910MiB 

Errors:

chunk-vendors.e14b044a.js:81 Error compiling shader '#version 300 es
#define attribute in
#define textureCube texture
#define texture2D texture
#define textureCubeLod textureLod
#define texture2DLod textureLod

#ifdef GL_FRAGMENT_PRECISION_HIGH
precision highp float;
precision highp int;
#else
precision mediump float;
precision mediump int;
#endif

/*=========================================================================

  Program:   Visualization Toolkit
  Module:    vtkImageResliceMapperVS.glsl

  Copyright (c) Ken Martin, Will Schroeder, Bill Lorensen
  All rights reserved.
  See Copyright.txt or http://www.kitware.com/Copyright.htm for details.

     This software is distributed WITHOUT ANY WARRANTY; without even
     the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR
     PURPOSE.  See the above copyright notice for more information.

=========================================================================*/

// all variables that represent positions or directions have a suffix
// indicating the coordinate system they are in. The possible values are
// MC - Model coordinates
// WC - World coordinates
// VC - View coordinates
// DC - Display coordinates
// TC - Texture coordinates

// frag position in VC
attribute vec4 vertexWC;
//12198Plane1524Image1408

// Texture coordinates
uniform mat4 WCTCMatrix;
out vec3 fragTexCoord;

// picking support
//VTK::Picking::Dec

// camera and actor matrix values
uniform mat4 MCPCMatrix;
uniform mat4 MCVCMatrix;

void main()
{
  gl_Position = MCPCMatrix * vertexWC;

  fragTexCoord = (WCTCMatrix * vertexWC).xyz;

  //VTK::Picking::Impl
}

9WebGL: CONTEXT_LOST_WEBGL: loseContext: context lost
chunk-vendors.e14b044a.js:21 Uncaught TypeError: Failed to execute 'shaderSource' on 'WebGL2RenderingContext': parameter 1 is not of type 'WebGLShader'.
    at e.compile (chunk-vendors.e14b044a.js:81:471993)
    at e.compileShader (chunk-vendors.e14b044a.js:81:473033)
    at e.readyShaderProgram (chunk-vendors.e14b044a.js:21:227523)
    at e.readyShaderProgramArray (chunk-vendors.e14b044a.js:21:227437)
    at e.updateShaders (chunk-vendors.e14b044a.js:81:186572)
    at e.drawArrays (chunk-vendors.e14b044a.js:81:185518)
    at e.renderPieceDraw (chunk-vendors.e14b044a.js:81:269600)
    at e.renderPiece (chunk-vendors.e14b044a.js:81:270380)
    at e.render (chunk-vendors.e14b044a.js:81:241057)
    at e.translucentPass (chunk-vendors.e14b044a.js:81:240588)
    at e.apply (chunk-vendors.e14b044a.js:15:73325)
    at e.traverse (chunk-vendors.e14b044a.js:15:73184)
    at e.traverseTranslucentPass (chunk-vendors.e14b044a.js:81:200587)
    at e.traverse (chunk-vendors.e14b044a.js:15:73172)
    at e.traverse (chunk-vendors.e14b044a.js:15:73246)
    at e.traverse (chunk-vendors.e14b044a.js:81:145783)
    at e.traverse (chunk-vendors.e14b044a.js:81:150128)
    at M.t.traverseAllPasses (chunk-vendors.e14b044a.js:21:156613)
    at S (chunk-vendors.e14b044a.js:21:545504)
    at e.render (chunk-vendors.e14b044a.js:21:556759)
    at e.render (chunk-vendors.e14b044a.js:21:458740)

chunk-vendors.e14b044a.js:81 0: #version 300 es
1: #define attribute in
2: #define textureCube texture
3: #define texture2D texture
4: #define textureCubeLod textureLod
5: #define texture2DLod textureLod
6: 
7: 
8: #ifdef GL_FRAGMENT_PRECISION_HIGH
9: precision highp float;
10: precision highp int;
11: #else
12: precision mediump float;
13: precision mediump int;
14: #endif
15: 
16: /*=========================================================================
17: 
18:   Program:   Visualization Toolkit
19:   Module:    vtkPolyDataVS.glsl
20: 
21:   Copyright (c) Ken Martin, Will Schroeder, Bill Lorensen
22:   All rights reserved.
23:   See Copyright.txt or http://www.kitware.com/Copyright.htm for details.
24: 
25:      This software is distributed WITHOUT ANY WARRANTY; without even
26:      the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR
27:      PURPOSE.  See the above copyright notice for more information.
28: 
29: =========================================================================*/
30: 
31: attribute vec4 vertexMC;
32: 
33: // frag position in VC
34: //VTK::PositionVC::Dec
35: 
36: // optional normal declaration
37: //VTK::Normal::Dec
38: 
39: // extra lighting parameters
40: //VTK::Light::Dec
41: 
42: // Texture coordinates
43: attribute vec2 tcoordMC; out vec2 tcoordVCVSOutput;
44: 
45: // material property values
46: //VTK::Color::Dec
47: 
48: // clipping plane vars
49: //VTK::Clip::Dec
50: 
51: // camera and actor matrix values
52: uniform mat4 MCPCMatrix;
53: 
54: // Apple Bug
55: //VTK::PrimID::Dec
56: 
57: // picking support
58: //VTK::Picking::Dec
59: 
60: void main()
61: {
62:   //VTK::Color::Impl
63: 
64:   //VTK::Normal::Impl
65: 
66:   tcoordVCVSOutput = tcoordMC;
67: 
68:   //VTK::Clip::Impl
69: 
70:   //VTK::PrimID::Impl
71: 
72:     gl_Position = MCPCMatrix * vertexMC;
73: 
74:   //VTK::Light::Impl
75: 
76:   //VTK::Picking::Impl
77: }
78: 

[finetjul]

@sankhesh @bruyeret

[bruyeret]

It seems that the errors are caused by the context loss which itself can be caused by the browser taking to much VRAM on the GPU. If the data don't fit on the GPU, the context will be lost at some point, and the only solution I can think of is to reduce the size of the data (using float16 or a lower resolution).

[emil-peters]

It is not possible to run it completely on the CPU and store nothing on the GPU? As re-slicing big dicoms is a mandatory thing in what I'm doing, but we have low end pc's crashing because they run out of VRAM :sweat_smile:

[bruyeret]

If you use the ImageReslice in Imaging/Core/ImageReslice, it is fully running on the CPU and only the slice is sent to the GPU. Be carfeul not to use the ImageResliceMapper in Rendering/Core/ImageResliceMapper which runs on the GPU and send the whole ImageData on it. Do you have a RenderWindow where you display the full volume? Or only RenderWindows with the slices?

[finetjul]

Hi @emil-peters, the reslice cursor widget example is not supposed to send the whole volume onto the GPU. So I do not understand why you have high VRAM usage with the reslice cursor widget example

[emil-peters]

I do not have it with the example, but with that we are developing. Sorry if there was some miscommunication

[finetjul]

Fair enough, so can you please explain what you are doing in the code you are developping ?

Have you adapted the ResliceCursorWidget example so that it uses vtkImageResliceMapper (GPU) instead of vtkImageReslice (CPU)?

[emil-peters]

Sure, I think @TimDeBauwReLu should be able to explain, he's the one developing the reslicing feature into our application. I've mostly been testing to see where and why it crashes.

[TimDeBauwReLu]

We still have to implement the change to the vtkImageReslice, but in our application we have 3 views of a large dicom and a label map on top of it. If we understand it correctly, 6x the size of the dicom is loaded in GPU VRAM when using the vtkImageResliceMapper, leading to the poor performance on GPU's with low specs.

[finetjul]

This is a know limitation, render window contexts do not share memory.

I created a ticket to reference this limitation and track the feature request: https://github.com/Kitware/vtk-js/issues/2866

Shall you have funding, we can work on that feature.

In the mean time, you can use vtkImageReslice (CPU) instead of vtkResliceMapper.