[Bug] All individual volume slices must be preloaded to prevent "Cannot destructure property 'pixelRepresentation'" error

sgielen commented 1 year ago

Describe the Bug

Possibly the same as #836.

If the individual slices of a volume are not preloaded before the volume is loaded, volume loading fails with the following error:

Uncaught TypeError: Cannot destructure property 'pixelRepresentation' of 'metaData_exports.get(...)' as it is undefined.
    at makeVolumeMetadata (makeVolumeMetadata.ts:17:5)
    at getStreamingImageVolume (cornerstoneStreamingImageVolumeLoader.ts:98:28)
    at cornerstoneStreamingImageVolumeLoader (cornerstoneStreamingImageVolumeLoader.ts:263:39)
    at loadVolumeFromVolumeLoader (volumeLoader.ts:151:28)
    at Object.createAndCacheVolume (volumeLoader.ts:227:22)
    at main.ts:166:37

Steps to Reproduce

A reproduction scenario is available at: https://sjorsgielen.nl/cornerstone3d/mandatory-load-slices.zip

Unpack this .zip file, run yarn install and yarn dev, then browse to the indicated port.

The current behavior

The volume does not load. The console shows this error:

Uncaught TypeError: Cannot destructure property 'pixelRepresentation' of 'metaData_exports.get(...)' as it is undefined.
    at makeVolumeMetadata (makeVolumeMetadata.ts:17:5)
    at getStreamingImageVolume (cornerstoneStreamingImageVolumeLoader.ts:98:28)
    at cornerstoneStreamingImageVolumeLoader (cornerstoneStreamingImageVolumeLoader.ts:263:39)
    at loadVolumeFromVolumeLoader (volumeLoader.ts:151:28)
    at Object.createAndCacheVolume (volumeLoader.ts:227:22)
    at main.ts:166:37

Now, open src/main.ts and change loadSlices to true, then refresh the page. The volume loads normally.

The expected behavior

Since we tell cornerstone3d to display a volume with a set of image IDs, it should start the loading of those image IDs normally. It should not matter whether the image IDs are preloaded in order to display the volume correctly.

OS

macOS 14.0

Node version

20.8.1

Browser

Safari 17.0, Chrome 119.0.6045.123

html5shiv commented 11 months ago

i load dicom file, This error also appears, Did you solve it?

sgielen commented 11 months ago

i load dicom file, This error also appears, Did you solve it?

Yes, there is a workaround in the code in the .zip file

html5shiv commented 11 months ago

i run your code,but the error also appears this is devtolls console

[vite] connecting...
client.ts:150 [vite] connected.
main.ts:61 Downloading slices.txt...
main.ts:66 Init cornerstone...
chunk-E4KJPFQM.js?v=47981a66:11468 CornerstoneRender: Using detect-gpu to get the GPU benchmark: {device: undefined, fps: 50, gpu: 'intel uhd graphics 620', isMobile: false, tier: 2, …}
chunk-E4KJPFQM.js?v=47981a66:11473 CornerstoneRender: using GPU rendering
main.ts:121 Initializing viewport...
main.ts:151 Rendering...
@cornerstonejs_streaming-image-volume-loader.js?v=5dcec5f3:21 Uncaught TypeError: Cannot destructure property 'pixelRepresentation' of 'metaData_exports.get(...)' as it is undefined.
    at makeVolumeMetadata (@cornerstonejs_streaming-image-volume-loader.js?v=5dcec5f3:21:11)
    at getStreamingImageVolume (@cornerstonejs_streaming-image-volume-loader.js?v=5dcec5f3:859:28)
    at cornerstoneStreamingImageVolumeLoader (@cornerstonejs_streaming-image-volume-loader.js?v=5dcec5f3:947:39)
    at loadVolumeFromVolumeLoader (chunk-E4KJPFQM.js?v=47981a66:40154:28)
    at Object.createAndCacheVolume (chunk-E4KJPFQM.js?v=47981a66:40188:22)
    at main.ts:168:37
makeVolumeMetadata @ @cornerstonejs_streaming-image-volume-loader.js?v=5dcec5f3:21
getStreamingImageVolume @ @cornerstonejs_streaming-image-volume-loader.js?v=5dcec5f3:859
cornerstoneStreamingImageVolumeLoader @ @cornerstonejs_streaming-image-volume-loader.js?v=5dcec5f3:947
loadVolumeFromVolumeLoader @ chunk-E4KJPFQM.js?v=47981a66:40154
createAndCacheVolume @ chunk-E4KJPFQM.js?v=47981a66:40188
(anonymous) @ main.ts:168
Show 5 more frames
Show less
@cornerstonejs_streaming-image-volume-loader.js?v=5dcec5f3:21 Uncaught (in promise) TypeError: Cannot destructure property 'pixelRepresentation' of 'metaData_exports.get(...)' as it is undefined.
    at makeVolumeMetadata (@cornerstonejs_streaming-image-volume-loader.js?v=5dcec5f3:21:11)
    at getStreamingImageVolume (@cornerstonejs_streaming-image-volume-loader.js?v=5dcec5f3:859:28)
    at cornerstoneStreamingImageVolumeLoader (@cornerstonejs_streaming-image-volume-loader.js?v=5dcec5f3:947:39)
    at loadVolumeFromVolumeLoader (chunk-E4KJPFQM.js?v=47981a66:40154:28)
    at Object.createAndCacheVolume (chunk-E4KJPFQM.js?v=47981a66:40188:22)
    at main.ts:168:37
makeVolumeMetadata @ @cornerstonejs_streaming-image-volume-loader.js?v=5dcec5f3:21
getStreamingImageVolume @ @cornerstonejs_streaming-image-volume-loader.js?v=5dcec5f3:859
cornerstoneStreamingImageVolumeLoader @ @cornerstonejs_streaming-image-volume-loader.js?v=5dcec5f3:947
loadVolumeFromVolumeLoader @ chunk-E4KJPFQM.js?v=47981a66:40154
createAndCacheVolume @ chunk-E4KJPFQM.js?v=47981a66:40188
(anonymous) @ main.ts:168
Promise.then (async)
createAndCacheVolume @ chunk-E4KJPFQM.js?v=47981a66:40189
(anonymous) @ main.ts:168
Show 6 more frames
Show less
@cornerstonejs_streaming-image-volume-loader.js?v=5dcec5f3:21 Uncaught (in promise) TypeError: Cannot destructure property 'pixelRepresentation' of 'metaData_exports.get(...)' as it is undefined.
    at makeVolumeMetadata (@cornerstonejs_streaming-image-volume-loader.js?v=5dcec5f3:21:11)
    at getStreamingImageVolume (@cornerstonejs_streaming-image-volume-loader.js?v=5dcec5f3:859:28)
    at cornerstoneStreamingImageVolumeLoader (@cornerstonejs_streaming-image-volume-loader.js?v=5dcec5f3:947:39)
    at loadVolumeFromVolumeLoader (chunk-E4KJPFQM.js?v=47981a66:40154:28)
    at Object.createAndCacheVolume (chunk-E4KJPFQM.js?v=47981a66:40188:22)
    at main.ts:168:37

sgielen commented 11 months ago

i run your code,but the error also appears this is devtolls console

What does it do when you set loadSlices to true as indicated in the "current behavior" section?

sgielen commented 3 weeks ago

I'll pick this issue up soon. The root cause here is that the volume loader expects metadata to exist for all image IDs. Especially in the case of WADOURI, metadata is only available when the image ID is loaded.

Outside of WADOURI, this can be resolved by ensuring metadata is available. Unfortunately, the metadata providers cannot return Promises but have to return their metadata immediately, so you'll need some way to make metadata available before you setVolume on the viewport. (I'm not sure how WADO-RS typically does this?)

With WADOURI I'd like to attempt a fix by doing the following:

Create a specific error type for "metadata missing for image ID". Throw this error whenever missing metadata is a fatal error, such as when a volume is being loaded, i.e. this issue.
An application could then catch that error, ensure metadata exists for that image ID (e.g. by downloading the DICOM file from WADOURI so its metadata is available in cache), and then perform the setVolume again.
I'll implement that in the custom DICOM loader example application for reference. Also, I'll try to see where it makes sense to write this in documentation.

@sedghi do you agree with that approach?

sedghi commented 3 weeks ago

Yes in case of wadoURI it is a must. I thought this piece of code handles it

if (options.imageIds[0].split(':')[0] === 'wadouri') {
      const [middleImageIndex, lastImageIndex] = [
        Math.floor(options.imageIds.length / 2),
        options.imageIds.length - 1,
      ];
      const indexesToPrefetch = [0, middleImageIndex, lastImageIndex];
      await Promise.all(
        indexesToPrefetch.map((index) => {
          return new Promise((resolve, reject) => {
            const imageId = options.imageIds[index];
            imageLoadPoolManager.addRequest(
              async () => {
                loadImage(imageId)
                  .then(() => {
                    console.log(`Prefetched imageId: ${imageId}`);
                    resolve(true);
                  })
                  .catch((err) => {
                    reject(err);
                  });
              },
              RequestType.Prefetch,
              { volumeId },
              1 // priority
            );
          });
        })
      ).catch(console.error);
    }

sgielen commented 3 weeks ago

Yes in case of wadoURI it is a must. I thought this piece of code handles it
if (options.imageIds[0].split(':')[0] === 'wadouri') {

That is true, but only if the imageId scheme is wadouri, which is not the case if a custom image loader is used :-(

sedghi commented 3 weeks ago

Your custom image loader was also dicom based right ? so maybe we can expand on that approach, it should be like

const imageLoader = findImageLoaderFromImageLoaderRegisteries(imageId)
const canAffordMetdataLoading = imageLoader.can_provide_metadata_by_loading_dicom
if (canAffordMetdataLoading){
 // get into that loop
}

sgielen commented 3 weeks ago

Your custom image loader was also dicom based right ? so maybe we can expand on that approach, it should be like
const imageLoader = findImageLoaderFromImageLoaderRegisteries(imageId)
const canAffordMetdataLoading = imageLoader.can_provide_metadata_by_loading_dicom
if (canAffordMetdataLoading){
 // get into that loop
}

That is also a good alternative! My approach was to just let the metadata fail further on, and then catch the exception and resolve it then. But it could be more elegant to call into the imageLoader which image IDs are necessary. I can write the PR according to your vision of what would be best.

sedghi commented 1 week ago

Can we close this after Cornerstone3D 2.0?

sgielen commented 1 week ago

Can we close this after Cornerstone3D 2.0?

No, not yet - I’m still planning to make a PR for this soonish

cornerstonejs / cornerstone3D