search

Kitware / itk-vtk-viewer

2D / 3D web image, mesh, and point set viewer using itk-wasm and vtk.js
https://kitware.github.io/itk-vtk-viewer/
BSD 3-Clause "New" or "Revised" License
210 stars 64 forks source link

Calling setImage is laggy #337

Open oeway opened 4 years ago

oeway commented 4 years ago

Hi, I am trying to use itk-vtk-viewer to display images updating in real-time, and I use viewer.setImage() to update it. The issue I have is that, when the function is called, the canvas doesn't necessary updated, the display constantly freeze. See an animation below updating_image-itk-vtk-viewer

(the number show later on the right is generated by console.log called right after viewer.setImage)

Is there a way to force the canvas to update? Do you have any suggestion for improving real-time performance? Thanks in advance.

thewtex commented 4 years ago

Hi @oeway ! For performance / stability reasons, we are currently throttling setImage:

https://github.com/Kitware/itk-vtk-viewer/blob/169a5ea34f891ae61318a876a484020a0c232162/src/createViewer.js#L432

If we change this from 100 (maybe too conservative) to 10 or 20, does that serve your use case?

oeway commented 4 years ago

I would like to do as fast as possible, ideally without throttling, would that crash the viewer? Otherwise 10 might be already good enough for most cases.

thewtex commented 4 years ago

The units are in milliseconds; 10 would be 100 frames / second, which is already faster than a very smooth rendering rate, i.e. 60 fps. To avoid crashes, we should not go too high. Perceptually, it is hard to see any difference over 60 fps.

oeway commented 4 years ago

The units are in milliseconds; 10 would be 100 frames / second, which is already faster than a very smooth rendering rate, i.e. 60 fps. To avoid crashes, we should not go too high. Perceptually, it is hard to see any difference over 60 fps.

I thought this is the interval between calls, that is to say, the actually rendering/display would take more time, no? Assume we set throttling to 0, the maximum frame rate we can achieve will be determined by the rendering which won't be very high for larger images.

Can we do throttling on the actual frame rate rather than the call to set_image?

Also does itk-vtk-viewer always rendering it immediately after calling set_image?

thewtex commented 4 years ago

Also does itk-vtk-viewer always rendering it immediately after calling set_image?

Yes -- render is performed after new data is added, so we need to take care to avoid trying to render at too rapid of a rate.

oeway commented 4 years ago

Also does itk-vtk-viewer always rendering it immediately after calling set_image?

Yes -- render is performed after new data is added, so we need to take care to avoid trying to render at too rapid of a rate.

Sounds good.

thewtex commented 2 years ago

Theories discussed with @PaulHax include cloning to create a transferable when passing to worker threads when SharedArrayBuffer is not available. Architecture improvements could include keeping source data in a dedicated web worker.