Some 3D DICOM files fail to load

timmevandermeer commented 2 years ago

I use itk.wasm to load DICOM files into a browser application. This typically concerns 3D DICOM files of between 100 and 400 mb in size. For most files this works fine using readImageBlob. However, for some files the following error occurs:

Aborted() error in GDCMImageIOReadImage.umd.js:9

```js Aborted() Module.printErr | @ | GDCMImageIOReadImage.umd.js:9 -- | -- | -- | abort | @ | GDCMImageIOReadImage.umd.js:9 | _abort | @ | GDCMImageIOReadImage.umd.js:9 | $func42 | @ | 00ad2fc6:0x2715 | $func374 | @ | 00ad2fc6:0x22cc3 | $func99 | @ | 00ad2fc6:0x6629 | $func2036 | @ | 00ad2fc6:0x127c84 | $func2042 | @ | 00ad2fc6:0x1280db | $func54 | @ | 00ad2fc6:0x4787 | $func444 | @ | 00ad2fc6:0x35331 | $func1265 | @ | 00ad2fc6:0xcf792 | $func576 | @ | 00ad2fc6:0x469c1 | $func260 | @ | 00ad2fc6:0x144b6 | $func940 | @ | 00ad2fc6:0x8a904 | $func1730 | @ | 00ad2fc6:0x113e7c | $func703 | @ | 00ad2fc6:0x605e0 | $R | @ | 00ad2fc6:0x1651ad | Module._main | @ | GDCMImageIOReadImage.umd.js:9 | callMain | @ | GDCMImageIOReadImage.umd.js:9 | M_ | @ | 6012291f-afcb-4805-8260-6c8b6a589776:3 | (anonymous) | @ | 6012291f-afcb-4805-8260-6c8b6a589776:3 | p | @ | 6012291f-afcb-4805-8260-6c8b6a589776:3 | (anonymous) | @ | 6012291f-afcb-4805-8260-6c8b6a589776:3 | (anonymous) | @ | 6012291f-afcb-4805-8260-6c8b6a589776:3 | e | @ | 6012291f-afcb-4805-8260-6c8b6a589776:1 | u | @ | 6012291f-afcb-4805-8260-6c8b6a589776:1 | (anonymous) | @ | 6012291f-afcb-4805-8260-6c8b6a589776:1 | (anonymous) | @ | 6012291f-afcb-4805-8260-6c8b6a589776:1 | Z_ | @ | 6012291f-afcb-4805-8260-6c8b6a589776:3 | Q_ | @ | 6012291f-afcb-4805-8260-6c8b6a589776:3 | (anonymous) | @ | 6012291f-afcb-4805-8260-6c8b6a589776:3 | p | @ | 6012291f-afcb-4805-8260-6c8b6a589776:3 | (anonymous) | @ | 6012291f-afcb-4805-8260-6c8b6a589776:3 | (anonymous) | @ | 6012291f-afcb-4805-8260-6c8b6a589776:3 | e | @ | 6012291f-afcb-4805-8260-6c8b6a589776:1 | u | @ | 6012291f-afcb-4805-8260-6c8b6a589776:1 ```

When loading the same file in the Node.js implementation using readImageLocalFile the same error occurs:

Aborted() error in GDCMImageIOReadImage.js:9

```js RuntimeError: Aborted(). Build with -s ASSERTIONS=1 for more info. at abort (file:///Users/timmevandermeer/Development/Test/itk-test/node_modules/itk-image-io/GDCMImageIOReadImage.js:9:10975) at _abort (file:///Users/timmevandermeer/Development/Test/itk-test/node_modules/itk-image-io/GDCMImageIOReadImage.js:9:85792) at wasm://wasm/00abea02:wasm-function[43]:0x25fc at wasm://wasm/00abea02:wasm-function[382]:0x2383d at wasm://wasm/00abea02:wasm-function[100]:0x666c at wasm://wasm/00abea02:wasm-function[1988]:0x12620b at wasm://wasm/00abea02:wasm-function[1995]:0x126670 at wasm://wasm/00abea02:wasm-function[54]:0x457c at wasm://wasm/00abea02:wasm-function[444]:0x3497a at wasm://wasm/00abea02:wasm-function[1273]:0xdb479 ```

The problematic DICOM files open fine in other software (e.g. Horos, VV, etc.). An example file with this issue is available here: https://drive.google.com/file/d/1KpFletcfreoR7w2ZQK8Hi1tamJsey7eN/view?usp=sharing

I think it has something to do with the compression, as reading the tags works fine. Furthermore, converting the file to a uncompressed DICOM results in a file that is able to load.

timmevandermeer commented 2 years ago

Got a bit more clear error with a local emscripten build on master. Seems like it runs out of memory (while other, larger jpeg lossless DICOM files don't). The issues seems to occur for about 20% of DICOM files I deal with

Cannot enlarge memory, asked to go up to 2248392704 bytes, but the limit is 2147483648 bytes!
Exception while running pipeline:
stdout: 
stderr: Cannot enlarge memory, asked to go up to 2248388608 bytes, but the limit is 2147483648 bytes!
Cannot enlarge memory, asked to go up to 2248392704 bytes, but the limit is 2147483648 bytes!

exception: /ITK/Modules/ThirdParty/GDCM/src/gdcm/Source/Common/gdcmException.h:74 ():
Impossible

file:///Users/timmevandermeer/Development/Test/itk-wasm/dist/image-io/GDCMImageIO-read-image.js:345
   throw ex;
   ^
10556128
Thrown at:
    at file:///Users/timmevandermeer/Development/Test/itk-wasm/dist/image-io/GDCMImageIO-read-image.js:345:4
    at emit (node:events:527:28)
    at node:internal/process/execution:167:25
    at loadESM (node:internal/process/esm_loader:94:31)

thewtex commented 2 years ago

@timmevandermeer thanks for the follow-up.

asked to go up to 2248392704 bytes, but the limit is 2147483648 bytes!

WASM is current 32-bit, and it looks like we are running into this limit here. 64-bit WASM is in the works, but is not readily available yet.

One item to look into whether we can reduce memory usage and release memory more aggressively in the module.

InsightSoftwareConsortium / ITK-Wasm

Some 3D DICOM files fail to load #608