Support DICOM files storing image as an array of pixel data frames

[raylinfordaruplab]

We are working with .dcm files of WSIs that slideio is not interpreting correctly. The pixel information is stored as an array of data frames that each represent a 256x256 tile of the overall image. Slideio currently only reads the first frame from the pixel data and treats it as if that is the entire slide. The file is still only one "scene" , but the scene is composed of an array of multiple "frames". The creator of the file is using the dicom metadata to indicate the structure like this:

(0028, 0002) Samples per Pixel                   US: 3
(0028, 0004) Photometric Interpretation          CS: 'YBR_FULL_422'
(0028, 0006) Planar Configuration                US: 0
(0028, 0008) Number of Frames                    IS: '126280'
(0028, 0010) Rows                                US: 256
(0028, 0011) Columns                             US: 256
(0028, 0100) Bits Allocated                      US: 8
(0028, 0101) Bits Stored                         US: 8
(0028, 0102) High Bit                            US: 7
(0028, 0103) Pixel Representation                US: 0
(0028, 0106) Smallest Image Pixel Value          US: 0
(0028, 0107) Largest Image Pixel Value           US: 250
(0028, 0301) Burned In Annotation                CS: 'NO'
(0028, 2110) Lossy Image Compression             CS: '01'
(0028, 2112) Lossy Image Compression Ratio       DS: '26.8038'
(0028, 2114) Lossy Image Compression Method      CS: 'ISO_10918_1'
(0028, 6010) Representative Frame Number         US: 1
(0040, 0512) Container Identifier                LO: ''
(0040, 0513)  Issuer of the Container Identifier Sequence  0 item(s) ---- 
(0040, 0518)  Container Type Code Sequence  0 item(s) ---- 
(0040, 051a) Container Description               LO: ''
(0040, 0555)  Acquisition Context Sequence  0 item(s) ---- 
(0040, 0560)  Specimen Description Sequence  0 item(s) ---- 
(0048, 0001) Imaged Volume Width                 FL: 18.236799240112305
(0048, 0002) Imaged Volume Height                FL: 24.27630043029785
(0048, 0003) Imaged Volume Depth                 FL: 1.0
(0048, 0006) Total Pixel Matrix Columns          UL: 78848
(0048, 0007) Total Pixel Matrix Rows             UL: 104960
(0048, 0008)  Total Pixel Matrix Origin Sequence  1 item(s) ---- 
   (0040, 072a) X Offset in Slide Coordinate System DS: '20.8873'
   (0040, 073a) Y Offset in Slide Coordinate System DS: '16.5625'
   ---------
(7fe0, 0010) Pixel Data                          OB: Array of 927354860 elements

We've been able to use highdicom to access the image data, but it is much slower than slideio, and we'd like to keep using slideio as our main WSI-handling library.

Looking into the slideio code, it seems like the issue might be that the number of frames is assumed to be 1 and for dicom files it isn't read from the metadata?

[mildewey]

We can provide a sample file, but it's too large to upload here.

[Booritas]

Thank you for your message! To be honest, the DICOM WSI format has not been implemented yet. The library currently handles the file as a regular DICOM file. However, implementing the WSI format should not be a major issue, and I will prioritize it. The implementation is expected to take up to a month, as most of the code has already been developed. I will provide a more precise estimation in the coming days.

Do you have a detailed description of the format? I found some information on the internet, but it does not cover all the details required for the implementation.

I also have a question for you. Is your slide represented by a single file or a directory? According to the format description I found, a WSI DICOM slide is typically represented by a directory with multiple files that correspond to different zoom levels.

I may need you to upload the file. Could you please let me know its size?

[mildewey]

We have a single zstack example that is just over 1 GB in a single file. I believe we'll be able to provide a multi-zstack example shortly. What would be the best way to get the files to you?

[Booritas]

Online service https://wetransfer.com/ allows to send files up to 2 GB for free. I have not tried it yet. Would you agree to try? Otherwise, I'll look for another option. You will need my email: stanislav.melnikov@gmail.com Thank you!

[mildewey]

I sent it. When I have the other set, I'll let you know.

[Booritas]

Thank you for the file! It looks like it is one file from the WSI slide. Normally, a WSI slide should have multiple zoom levels. Here is an example:

H01EBB50P-24777_label.dcm H01EBB50P-24777_level-0.dcm H01EBB50P-24777_level-1.dcm H01EBB50P-24777_level-2.dcm H01EBB50P-24777_level-3.dcm H01EBB50P-24777_level-4.dcm H01EBB50P-24777_level-5.dcm H01EBB50P-24777_level-6.dcm H01EBB50P-24777_level-7.dcm H01EBB50P-24777_level-8.dcm H01EBB50P-24777_localizer.dcm H01EBB50P-24777_macro.dcm

Each of the *_level-x.dcm files represents a zoom level of the slide. Multiple zoom levels allow for easy rendering of any magnification of the image. Do you have the rest of the files? This does not impede development, as I have found some complete examples of WSI files. If you have only a single level of the WSI slide, you will still be able to read arbitrary regions of the slide, but you may encounter performance issues if you want to render the entire slide or a portion of the slide with significant downsampling.

[raylinfordaruplab]

The dicom format does allow for the multiple zoom levels as you've described. The scanner that we are working with only outputs the level 0 file, however. Also in my own testing, with the files we have received, the "label" and "macro" files have been identical to the "level" files, as far as the image data format and image-specific metadata is concerned. That means I've been able to use the much smaller label file for my local testing, which has been faster for small tests.

[Booritas]

Well, each zoom level is a separate DICOM dataset. I have some Whole Slide Imaging (WSI) slides with multiple zoom levels in separate files. You can check here. Maybe it is just scanner settings that produce an image pyramid.

Anyway, the performance without an image pyramid should be OK as long if you read original scale tiles or downsample them by not more than 4x. Bigger downsampling results in worse performance.

From your message, I understand that your're intersted in reading of tiles of original scale. Is it correct?

Thanks for your help! I'm starting the implementation today.

[raylinfordaruplab]

That is mostly correct. The actual ML process that will be consuming the WSIs will be using the original scale. We do have a separate application to allow viewing and browsing the slides, and we are doing some tiling at zoomed out scales for that application. If the performance at greater downsampling is a major issue, then we can work with the scanner to try and provide those zoom levels' files as well.

[Booritas]

I have just completed the initial implementation for reading DICOM WSI images. Could I ask you to test it with your images? You can download the wheel files for Linux from this link, and for Windows and Mac (x64) from this link. Please let me know if you require the wheel files for Mac arm, as they are hosted on another CI server. Please let me know if performance is acceptable for you.

FYI: I'm working on tutorials for the library. Here is jupyter notebook for the DICOM images: https://github.com/Booritas/slideio-tutorial/blob/dicom/dicom.ipynb (still in progress, not yet merged to the master branch).

[raylinfordaruplab]

We are running on Mac arm processors.

[Booritas]

What version of pythton do you use?

[raylinfordaruplab]

3.9.6. and mildewey was able to test it in a docker container (how we will actually be using it, most of the time). And it seems promising.

[raylinfordaruplab]

Our testing with our tiling process seems to be going well. Tiling a large image at different sizes, at different scales. The scaling appears to be working, and the processing time appears to be scaling relatively linearly for scales at 1/8 and lower.

[Booritas]

Thank you for your feedback. Here is the wheel file for Mac Arm processors. Let me know if you have any problem. slideio-2.4.0-cp39-cp39-macosx_11_0_arm64.whl.zip

[Booritas]

I just uploaded a new version 2.4.1 that includes support for DICOM WSI files