search

NA-MIC / ProjectWeek

Website for NA-MIC Project Weeks
https://projectweek.na-mic.org
81 stars 286 forks source link

Project: Efficient Handling and Progressive Loading of Compressed Multiframe DICOM Images #681

Closed OzgeYurtsever closed 1 year ago

OzgeYurtsever commented 1 year ago

Category

Cloud / Web

Key Investigators

Project Description

Loading compressed multiframe DICOM images as a whole causes frequent browser crashes, particularly on Microsoft machines. This issue arises due to the large file size of the DICOM images, exceeding the browser's memory capacity.

The browser's rendering engine attempts to load the entire file into memory, due to the significant size of these images, the browser can quickly exhaust its allocated memory, leading to crashes or unresponsive behavior.

This issue affects both ePAD and OHIF with the latest WADO-loader version.

Objective

Initiate a discourse about the methodologies for saving, storing, and reading DICOM data, and explore strategies for optimizing the handling of compressed multiframe images to achieve enhanced efficiency and avoid browser crashing.

Approach and Plan

Instead of loading the entire DICOM file at once, the image loading process can be modified to load the image in smaller chunks or frames progressively. This approach may allow the browser to handle smaller portions of the image, reducing the memory burden and enhancing overall stability.

Progress and Next Steps

We attempted to adapt a solution approach inspired by the PR link below. The link's solution specifically addresses uncompressed images. In our case, we tried a similar method to handle compressed images within the dicom-parser library, unfortunately, the attempted solution did not yield the desired outcome.

PR link: https://github.com/cornerstonejs/cornerstoneWADOImageLoader/pull/454#issue-1287614710 Ticket link: https://github.com/cornerstonejs/dicomParser/issues/248

Illustrations

crash-image

Background and References

Unfortunately the ultrasound images are not deindentified, we can not provide sample data yet. We are working on getting a data set.

Related libraries: https://github.com/cornerstonejs/cornerstoneWADOImageLoader https://github.com/cornerstonejs/dicomParser

sjh26 commented 1 year ago

@OzgeYurtsever Could you add a title for the project?

fedorov commented 1 year ago

@OzgeYurtsever can you provide the characteristics of the dataset that would allow reproducing the issue? We may be able to locate a publicly available sample in Imaging Data Commons.

OzgeYurtsever commented 1 year ago

@fedorov Two key variables contribute to this problem:

fedorov commented 1 year ago

The query below will identify all MR US series from IDC along with the URL and total file size. 

SELECT
  SeriesInstanceUID,
  StudyInstanceUID,
  NumberOfFrames,
  SUM(instance_size)/POW(1024,3) AS series_size,
  CONCAT("https://viewer.imaging.datacommons.cancer.gov/viewer/",StudyInstanceUID,"?seriesInstanceUID=",SeriesInstanceUID) as viewer_url
FROM
  `bigquery-public-data.idc_current.dicom_all`
WHERE
  Modality = "US"
  AND SOPClassUID = "1.2.840.10008.5.1.4.1.1.3.1" #     Ultrasound Multi-frame Image IOD
GROUP BY
  StudyInstanceUID,
  SeriesInstanceUID,
  NumberOfFrames
ORDER BY
  series_size desc

Here's a sample series returned: https://viewer.imaging.datacommons.cancer.gov/viewer/1.3.6.1.4.1.14519.5.2.1.1600.1201.280261060505804787745195562783?seriesInstanceUID=1.3.6.1.4.1.14519.5.2.1.1600.1201.169862383836550038574427412141

None of the series I tried crashed the viewer.

If you are not sure what to do with the query above or how to work with IDC, you can get up to speed by following this tutorial: https://github.com/ImagingDataCommons/IDC-Tutorials/tree/master/notebooks/getting_started.

github-actions[bot] commented 1 year ago

Project Page Pull Request Creation

:white_check_mark: COMPLETED: See https://github.com/NA-MIC/ProjectWeek/pull/683

OzgeYurtsever commented 1 year ago

The query below will identify all MR US series from IDC along with the URL and total file size.

SELECT
  SeriesInstanceUID,
  StudyInstanceUID,
  NumberOfFrames,
  SUM(instance_size)/POW(1024,3) AS series_size,
  CONCAT("https://viewer.imaging.datacommons.cancer.gov/viewer/",StudyInstanceUID,"?seriesInstanceUID=",SeriesInstanceUID) as viewer_url
FROM
  `bigquery-public-data.idc_current.dicom_all`
WHERE
  Modality = "US"
  AND SOPClassUID = "1.2.840.10008.5.1.4.1.1.3.1" #   Ultrasound Multi-frame Image IOD
GROUP BY
  StudyInstanceUID,
  SeriesInstanceUID,
  NumberOfFrames
ORDER BY
  series_size desc

Here's a sample series returned: https://viewer.imaging.datacommons.cancer.gov/viewer/1.3.6.1.4.1.14519.5.2.1.1600.1201.280261060505804787745195562783?seriesInstanceUID=1.3.6.1.4.1.14519.5.2.1.1600.1201.169862383836550038574427412141

None of the series I tried crashed the viewer.

If you are not sure what to do with the query above or how to work with IDC, you can get up to speed by following this tutorial: https://github.com/ImagingDataCommons/IDC-Tutorials/tree/master/notebooks/getting_started.

@fedorov Thank you for your reply. I tried the OHIF viewer demo locally with a breast ultrasound, but it crashed almost immediately. Unfortunately, I am unable to share the images I uploaded due to the visible PID. I have attached a screenshot of the crash. Do you think using the OHIF demo may not provide the exact same experience as what you shared earlier? Does OHIF demo page utilize the same image retrieving and loading methods with the one you shared above?

OHIF_demo _rash

fedorov commented 1 year ago

All I am trying to say is that if you can reproduce the problem with a public dataset, I think this can help debug the problem. I gave you some pointers to public datasets you can try to use to reproduce the problem. I do not know what is the causing the problem.

OzgeYurtsever commented 1 year ago

Thank you, I will try to find a dataset from the public sources that causes the same issue.

emelalkim commented 1 year ago

Hi @fedorov, I tried looking at Data Commons. Is there any compressed data there? Or is there a way to specify transfer syntax? I couldn't find it. This issue seems to be related to the images being compressed. (We are still trying to get some data we can share)

fedorov commented 1 year ago

@emelalkim you can use any DICOM attribute - including TransferSyntaxUID - as part of SQL query. If this turns out too difficult to figure out this week, I am happy to help at the PW!

emelalkim commented 1 year ago

Thanks @fedorov, I tried a query like this and opened one of them but I'm not sure if it is compressed as response header for transfer syntax is /. Let me see if you see something obvious, otherwise let's check it out at PW. SELECT SeriesInstanceUID, StudyInstanceUID, NumberOfFrames, SUM(instance_size)/POW(1024,3) AS series_size, CONCAT("https://viewer.imaging.datacommons.cancer.gov/viewer/",StudyInstanceUID,"?seriesInstanceUID=",SeriesInstanceUID, "&TransferSyntaxUID=1.2.840.10008.1.2.4.91") as viewer_url FROM bigquery-public-data.idc_current.dicom_all WHERE Modality = "US" AND SOPClassUID = "1.2.840.10008.5.1.4.1.1.3.1" # Ultrasound Multi-frame Image IOD GROUP BY StudyInstanceUID, SeriesInstanceUID, NumberOfFrames ORDER BY series_size desc

fedorov commented 1 year ago

Emel, the URL is just a convenience shortcut to give you the URL to the viewer to open in the browser the series in each row. You should add the attributes to the WHERE statement. Here's the query that takes into account your new condition:

SELECT
  SeriesInstanceUID,
  StudyInstanceUID,
  NumberOfFrames,
  SUM(instance_size)/POW(1024,3) AS series_size,
  CONCAT("https://viewer.imaging.datacommons.cancer.gov/viewer/",StudyInstanceUID,"?seriesInstanceUID=",SeriesInstanceUID) as viewer_url
FROM
  `bigquery-public-data.idc_current.dicom_all`
WHERE
  Modality = "US"
  AND SOPClassUID = "1.2.840.10008.5.1.4.1.1.3.1" #     Ultrasound Multi-frame Image IOD
  AND TransferSyntaxUID = "1.2.840.10008.1.2.4.91" # select JPEG2000 compression
GROUP BY
  StudyInstanceUID,
  SeriesInstanceUID,
  NumberOfFrames
ORDER BY
  series_size desc

Unfortunately, there are no US (or any other modality, except Slide Microscopy) with JPEG2000 compression in IDC :-(

emelalkim commented 1 year ago

Yeah, I had already tried that, and a couple other compression types. I was hoping I could give the Transfer Syntax to OHIF. :(

emelalkim commented 1 year ago

Hi @fedorov, I was able to clean one multiframe DICOM. It is available at sample image And after some troubleshooting we think it is about the size of the file, rather than compression. The sample image is uncompressed and a little over 500MB. In windows machines, both with Chrome and Edge, it crashes OHIF viewer almost instantly