visit-dav / summer-projects

A place to manage activity on summer projects
1 stars 0 forks source link

Identify and upload additional datasets to get #9

Closed markcmiller86 closed 1 year ago

markcmiller86 commented 2 years ago
Magicat-0 commented 2 years ago

I have been looking through my links and online to find datasets to play with but it has been difficult to find only blood vessel imaging studies (not because they're not common but because databases rely on shared/donated data), and micro-CTs as we know are used in preclinical/clinical research for small animals or small human [pieces], e.g., biopsies. I did find some other datasets if to try out (animal micro cts and human cts/MRIs) and see how they behave in VisIt or we could wait until we get the leg micro ct. Let me know what you think.

This is brain CT scans. 458GB https://www.kaggle.com/competitions/rsna-intracranial-hemorrhage-detection/data

This is 4D fan bean and cone beam CT scans of lungs with non-small cell cancer. 183GB https://wiki.cancerimagingarchive.net/display/Public/4D-Lung

Breast MRIs. 76GB https://wiki.cancerimagingarchive.net/display/Public/ISPY1

Visible human project data - CT and MRI cryosections https://www.nlm.nih.gov/databases/download/vhp.html

Micro CT head blood vessels - crocodilians. 3.6GB https://datadryad.org/stash/dataset/doi:10.5061%2Fdryad.mt64k

Early postmortem brain MRI in covid patients https://datadryad.org/stash/dataset/doi:10.5061%2Fdryad.4qrfj6q7p

Micro-CT of whole heart architecture in a small animal model. 3.3GB https://datadryad.org/stash/dataset/doi:10.5061%2Fdryad.hdr7sqvg5

Micro-CT automated segmentation of biological tissues- stingray https://datadryad.org/stash/dataset/doi:10.5061%2Fdryad.f53s5

Micro-CT dynamic contrast enhanced images - mouse https://datadryad.org/stash/dataset/doi:10.5061%2Fdryad.s1j6b

Micro-CT noninvasive protocol for in vivo anatomical studies - reptiles and amphibians https://datadryad.org/stash/dataset/doi:10.5061%2Fdryad.pv6jf

Just a general and exhaustive database if you are ever interested. https://www.cancerimagingarchive.net/access-data/

markcmiller86 commented 2 years ago

I think all the sizes here are very easily handled...at least from a storage perspectve. We have plenty of storage space at LLNL to handle these sizes. What is less clear at this point is how easily they can be digested by VisIt which depends on file format and composition.

I think Prof. Mondy's data is 5-10 micron resolution. Is that right? Do any of these sources approach that resolution?

I like the head blood vessel and whole heart stuff because its on target with our primary subject matter. But, I wonder how big a difference there is between scans of actual tissue specimens and scans of polymerized specimens. Prof. Mondy's scans are from polymerized specimens. I think that should make vessel network 3D reconstruction easier but I am not sure either.

Just curious but can you find any datasets or researchers working from scans of polymerized specimens?

Magicat-0 commented 2 years ago

Corrosion casting is a very old and widely used technique so i can find a lot of papers about it. But finding their raw datasets will be more difficult but i am sure i can find some.

Image on page 16 of this article: "Brains were scanned at a resolution of 7.5 μm using equispaced angles of view around 360°, and 3D reconstructions were prepared with Bruker’s CTVox 3D visualization software". We have CTvox/CTan (software package for very popular micro-ct, skyscan) that they gave prof. Mondy. https://actaneurocomms.biomedcentral.com/articles/10.1186/s40478-018-0647-5

https://www.sciencedirect.com/science/article/abs/pii/S0142961220302799?via%3Dihub https://physoc.onlinelibrary.wiley.com/doi/epdf/10.1113/JP282292

The above articles use this software: https://www.mbfbioscience.com/vesselucida360 Maybe we can get ideas for helpful tools from it or this: https://www.materialise.com/en/cases/3d-arterial-modeling-replacing-casting-by-vivo-micro-ct

https://www.nature.com/articles/s41598-017-04379-0 https://link.springer.com/article/10.1007/s00429-020-02158-8 https://www.intechopen.com/chapters/38120

Nebuloni_et_al_2014.pdf Human_inner_ear_blood_supply_revisited_the_Uppsala.pdf

https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0150085 https://vascularcell.biomedcentral.com/articles/10.1186/2040-2384-2-7 https://www.thno.org/v08p2117.htm https://www.nature.com/articles/srep41842 https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0031179 https://link.springer.com/article/10.1007/s11548-016-1378-3 https://link.springer.com/article/10.1007/s11307-010-0335-8

Most of them describe in good detail the image analysis/processing protocols they used.

markcmiller86 commented 2 years ago

Corrosion casting is a very old and widely used technique so i can find a lot of papers about it. But finding their raw datasets will be more difficult but i am sure i can find some.

Right but what about its use as a pre-processing step to produce a digital, 3D reconsruction of a vessel network? When I search for any researchers doing that...Prof. Mondy's work is the first several hits I find. So, corrosion casting...old. But, corrosion casting + 3D digital reconstruction from microCT sounds pretty new. It has the effect of cleaning up the input images, a lot!

That said, from the few images I've looked at in Prof Mondy's datasets, the microCT system adds a lot of annotations around the edges of the image and that is one challenge we'll have working with them...removing that stuff. Just spatially clipping it out is the obvious and first thing to try but I worry that as text annotations might vary in size that we wind up clipping out variable amounts of the image resulting in different sized images and complicating the 3D reconstruction. Just one of the many little hurdles we'll have to get over here.

Magicat-0 commented 2 years ago

Right but what about its use as a pre-processing step to produce a digital, 3D reconsruction of a vessel network?

Yes, also as a pre-processing step for use in 3D reconstruction of vessels (see articles above). The oldest papers i have found on this are from like early 2000's, so definitely a newer method. I agree, working with a micro-ct image of a corrosion cast vs a micro ct of something with other tissue/material volume or differing densities gives images with less artifacts and that need less processing with different software prior to exporting them for modelling. Hopefully in the near future we will have micro-cts not just for small animals but for humans as well (but with low-null radiation ofc). Here are 3 more articles: Corrosion Cast and 3D Reconstruction of the Murine Biliary Tree After Biliary Obstruction: Quantitative Assessment and Comparison With 2D Histology: https://www.sciencedirect.com/science/article/pii/S0973688321005880 Maximizing modern distribution of complex anatomical spatial information: 3D reconstruction and rapid prototype production of anatomical corrosion casts of human specimens: https://anatomypubs.onlinelibrary.wiley.com/doi/10.1002/ase.1287 Quantification of the Whole Lymph Node Vasculature Based on Tomography of the Vessel Corrosion Casts: https://www.nature.com/articles/s41598-019-49055-7

image

We may be getting closer to a better imaging modality:

Synchrotron Phase Tomography: An Emerging Imaging Method for Microvessel Detection in Engineered Bone of Craniofacial Districts: https://www.frontiersin.org/articles/10.3389/fphys.2017.00769/full

PS. Sorry for the long comments and Alt Text (berserker barrage) of articles 😅

markcmiller86 commented 2 years ago

Some remarks below after visiting various of these data source sites. In some cases, the data reference here is the results data from some published algorithm, not the input data.

This is brain CT scans. 458GB https://www.kaggle.com/competitions/rsna-intracranial-hemorrhage-detection/data

  • Appears to be part of some AI competition
  • There appears to be training data and then test data (are they all of the same specimen...are they slices through one common specimen...that isn't clear to me).
  • Must register (presumably to be part of the competition) to see the data
  • Many (slices I think) .dcm files

This is 4D fan bean and cone beam CT scans of lungs with non-small cell cancer. 183GB https://wiki.cancerimagingarchive.net/display/Public/4D-Lung

  • Due to structure of scanning process (beam and cone), probably quite interestig data
  • Note says its DICOM data so probably .dcm
  • Requires special software just to download the data (Centos, Ubunto, macOS and Windows)

Breast MRIs. 76GB https://wiki.cancerimagingarchive.net/display/Public/ISPY1

  • Note says its DICOM data so probably .dcm
  • Requires special software just to download the data (Centos, Ubunto, macOS and Windows)

Visible human project data - CT and MRI cryosections https://www.nlm.nih.gov/databases/download/vhp.html

  • probably pretty low-res by today's standards
  • Multiple data sources of same specimen (CT/MRI)
  • Example data: MRI-MaleFeet.zip
  • PNG formatted images

Micro CT head blood vessels - crocodilians. 3.6GB https://datadryad.org/stash/dataset/doi:10.5061%2Fdryad.mt64k

  • All data in a 2.4G .zip file. Didn't wait to download it all to see what underlying format is.
  • Notes say it is DICOM data
  • I like this because it involves blood vessels, our key subject matter
  • 90 micrometer microCT scans...very similar though lower res than Prof. Mondy's data

Early postmortem brain MRI in covid patients https://datadryad.org/stash/dataset/doi:10.5061%2Fdryad.4qrfj6q7p

  • Didn't find much actual data here.
  • Download was a .zip file that expanded to a single PDF file with some tables in it.

Micro-CT of whole heart architecture in a small animal model. 3.3GB https://datadryad.org/stash/dataset/doi:10.5061%2Fdryad.hdr7sqvg5

  • Includes normal and abnormal datasets for comparative purposes (could be useful)
  • 3.5 Gig .zip file to download
  • SEM data (so its unique...not CT or MRI)

Micro-CT automated segmentation of biological tissues- stingray https://datadryad.org/stash/dataset/doi:10.5061%2Fdryad.f53s5

  • There is an associated dataset, DOI: 10.1111/joa.12508
  • Downloaded what was there...not much. Some .csv and .ma files.
  • Appears to be results data from a segmentation algorithm.
  • Not sure we can do much with this

Micro-CT dynamic contrast enhanced images - mouse https://datadryad.org/stash/dataset/doi:10.5061%2Fdryad.s1j6b

  • Appears to be results data
  • Downloaded it and just a small amount of data here.

Micro-CT noninvasive protocol for in vivo anatomical studies - reptiles and amphibians https://datadryad.org/stash/dataset/doi:10.5061%2Fdryad.pv6jf

  • Appears to be results data
  • Downloaded it and just a small amount of data here.

Just a general and exhaustive database if you are ever interested. https://www.cancerimagingarchive.net/access-data/

After reviewing these candidates (thanks for finding them @Magicat-0), I'd like to propose a refined list of criteria and maybe spend another few hours searching for matching datasets. Here are some criteria...

Thoughts?

Magicat-0 commented 2 years ago

For the first one, they are using the brain ct scans as training datasets but they are still usable dcms and yes, you need a kaggle account to download it.

The cancer imaging archive datasets require a little extension they provide you with to view/download any study in their database. It runs easily and there is a Mac version too https://apps.apple.com/us/app/downloader-app/id1399207860?mt=12

Some of the other datasets are small or low res (cryosliced one) or take extra steps to download, but i figured having more options is always good. Maybe we could find something interesting or helpful or just to play with it in VisIt.

The other articles in the later comments list the image acquisition and processing steps they used for the micro cts of corrosion casts or live animal models too.

I did just find this: https://www.facebase.org/chaise/record/#1/isa:dataset/RID=V92 It's still downloading but they are micro ct images of adult mice heads.

Magicat-0 commented 2 years ago

Nevermind on the adult mice head micro ct. It is too low res for a micro ct and only dense tissue is visible. Couldn't get any vessels 😢

markcmiller86 commented 1 year ago

We have identified a number of datasets to possibly play with here but have not uploaded any of these and that is fine for now.