angelolab / Nimbus

Other
12 stars 1 forks source link

Adaptation of input images/directory structure for OME.TIFF formatted files. #79

Closed raymond301 closed 2 weeks ago

raymond301 commented 9 months ago

At my institution we run multiple platforms and have developed a centralized infrastructure to support multiple investigators by employing an internal, institutional instance of OMERO. https://www.openmicroscopy.org/omero/

omeroex01

As such we have chosen to standardize out multiplex imaging into OME.TIFF format. I have a ETL processes that converts acquired images, from which ever image acquisition machine used into standardized OME.TIFF format. 1 channel per marker. Including structured metadata. Here is an example of a typical image's format.

<OME Creator="OME Bio-Formats 7.0.1" UUID="urn:uuid:24e883cc-d05f-4f32-be50-fe98ab2a95d6" xmlns="http://www.openmicroscopy.org/Schemas/OME/2016-06" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.openmicroscopy.org/Schemas/OME/2016-06 http://www.openmicroscopy.org/Schemas/OME/2016-06/ome.xsd">
<Image ID="Image:0" Name="SLIDE-1942_FullPanel_R13.ome.tiff #1">
<Pixels BigEndian="false" DimensionOrder="XYCZT" ID="Pixels:0" Interleaved="false" PhysicalSizeX="324.9989823263602" PhysicalSizeXUnit="nm" PhysicalSizeY="324.9989823263602" PhysicalSizeYUnit="nm" SignificantBits="16" SizeC="65" SizeT="1" SizeX="2040" SizeY="2040" SizeZ="1" Type="uint16">
<Channel Color="-1" ID="Channel:0:0" Name="DAPI_AF_R01" SamplesPerPixel="1">
<LightPath/>
<Channel Color="1358106367" ID="Channel:0:1" Name="aSMA" SamplesPerPixel="1">
<LightPath/>
<Channel Color="-74044417" ID="Channel:0:2" Name="CD21" SamplesPerPixel="1">
<LightPath/>
<Channel Color="-1188299777" ID="Channel:0:3" Name="CD4" SamplesPerPixel="1">
<LightPath/>
<Channel Color="737932799" ID="Channel:0:4" Name="PanCK" SamplesPerPixel="1">
<LightPath/>
<Channel Color="-81033473" ID="Channel:0:5" Name="CD8" SamplesPerPixel="1">
<LightPath/>
<Channel Color="-68015105" ID="Channel:0:6" Name="FoxP3" SamplesPerPixel="1">
<LightPath/>
<Channel Color="-928083457" ID="Channel:0:7" Name="PDL1" SamplesPerPixel="1">
<LightPath/>
<Channel Color="-1" ID="Channel:0:8" Name="CD163" SamplesPerPixel="1">
<LightPath/>
<Channel Color="1358106367" ID="Channel:0:9" Name="S6" SamplesPerPixel="1">
<LightPath/>
<Channel Color="-74044417" ID="Channel:0:10" Name="T-BET" SamplesPerPixel="1">
<LightPath/>
<Channel Color="-1188299777" ID="Channel:0:11" Name="CD20" SamplesPerPixel="1">
<LightPath/>
<Channel Color="737932799" ID="Channel:0:12" Name="CD3" SamplesPerPixel="1">
<LightPath/>
<Channel Color="-81033473" ID="Channel:0:13" Name="CD11c" SamplesPerPixel="1">
<LightPath/>
<Channel Color="-68015105" ID="Channel:0:14" Name="CD45" SamplesPerPixel="1">
<LightPath/>
<Channel Color="-928083457" ID="Channel:0:15" Name="CD86" SamplesPerPixel="1">
<LightPath/>
<Channel Color="-1" ID="Channel:0:16" Name="CD38" SamplesPerPixel="1">
<LightPath/>
<Channel Color="1358106367" ID="Channel:0:17" Name="SOX10" SamplesPerPixel="1">
<LightPath/>
<Channel Color="-74044417" ID="Channel:0:18" Name="PD1" SamplesPerPixel="1">
<LightPath/>
<Channel Color="-1188299777" ID="Channel:0:19" Name="MART1" SamplesPerPixel="1">
<LightPath/>
<Channel Color="737932799" ID="Channel:0:20" Name="CD15" SamplesPerPixel="1">
<LightPath/>
<Channel Color="-81033473" ID="Channel:0:21" Name="CD11b" SamplesPerPixel="1">
<LightPath/>
<Channel Color="-68015105" ID="Channel:0:22" Name="LAG3" SamplesPerPixel="1">
<LightPath/>
<Channel Color="-928083457" ID="Channel:0:23" Name="CD206" SamplesPerPixel="1">
<LightPath/>
<Channel Color="-1" ID="Channel:0:24" Name="GzB" SamplesPerPixel="1">
<LightPath/>
<Channel Color="1358106367" ID="Channel:0:25" Name="FAP" SamplesPerPixel="1">
<LightPath/>
<Channel Color="-74044417" ID="Channel:0:26" Name="NAK" SamplesPerPixel="1">
<LightPath/>
<Channel Color="-1188299777" ID="Channel:0:27" Name="HLAII" SamplesPerPixel="1">
<LightPath/>
<Channel Color="737932799" ID="Channel:0:28" Name="gp100" SamplesPerPixel="1">
<LightPath/>
<Channel Color="-81033473" ID="Channel:0:29" Name="CD14" SamplesPerPixel="1">
<LightPath/>
<Channel Color="-68015105" ID="Channel:0:30" Name="MPO" SamplesPerPixel="1">
<LightPath/>
<Channel Color="-928083457" ID="Channel:0:31" Name="HLAI" SamplesPerPixel="1">
<LightPath/>
<Channel Color="-1" ID="Channel:0:32" Name="TGFB" SamplesPerPixel="1">
<LightPath/>
<Channel Color="1358106367" ID="Channel:0:33" Name="CD28" SamplesPerPixel="1">
<LightPath/>
<Channel Color="-74044417" ID="Channel:0:34" Name="Ki67" SamplesPerPixel="1">
<LightPath/>
<Channel Color="-1188299777" ID="Channel:0:35" Name="TenC" SamplesPerPixel="1">
<LightPath/>
<Channel Color="737932799" ID="Channel:0:36" Name="IDO" SamplesPerPixel="1">
<LightPath/>
<Channel Color="-81033473" ID="Channel:0:37" Name="TCF7" SamplesPerPixel="1">
<LightPath/>
<Channel Color="-68015105" ID="Channel:0:38" Name="TRAIL" SamplesPerPixel="1">
<LightPath/>
<Channel Color="-928083457" ID="Channel:0:39" Name="cGAS" SamplesPerPixel="1">
<LightPath/>
<Channel Color="-1" ID="Channel:0:40" Name="TIM3" SamplesPerPixel="1">
<LightPath/>
<Channel Color="1358106367" ID="Channel:0:41" Name="PVR" SamplesPerPixel="1">
<LightPath/>
<Channel Color="-74044417" ID="Channel:0:42" Name="Kappa" SamplesPerPixel="1">
<LightPath/>
<Channel Color="-1188299777" ID="Channel:0:43" Name="CD31" SamplesPerPixel="1">
<LightPath/>
<Channel Color="737932799" ID="Channel:0:44" Name="STING" SamplesPerPixel="1">
<LightPath/>
<Channel Color="-81033473" ID="Channel:0:45" Name="CD68" SamplesPerPixel="1">
<LightPath/>
<Channel Color="-68015105" ID="Channel:0:46" Name="CD56" SamplesPerPixel="1">
<LightPath/>
<Channel Color="-928083457" ID="Channel:0:47" Name="B2M" SamplesPerPixel="1">
<LightPath/>
<Channel Color="-1" ID="Channel:0:48" Name="CD27" SamplesPerPixel="1">
<LightPath/>
<Channel Color="1358106367" ID="Channel:0:49" Name="Casp3" SamplesPerPixel="1">
<LightPath/>
<Channel Color="-74044417" ID="Channel:0:50" Name="tryptase" SamplesPerPixel="1">
<LightPath/>
<Channel Color="-1188299777" ID="Channel:0:51" Name="CD16" SamplesPerPixel="1">
<LightPath/>
<Channel Color="737932799" ID="Channel:0:52" Name="CD83" SamplesPerPixel="1">
<LightPath/>
<Channel Color="-81033473" ID="Channel:0:53" Name="TIGIT" SamplesPerPixel="1">
<LightPath/>
<Channel Color="-68015105" ID="Channel:0:54" Name="CD19" SamplesPerPixel="1">
<LightPath/>
<Channel Color="-928083457" ID="Channel:0:55" Name="AR" SamplesPerPixel="1">
<LightPath/>
<Channel Color="-1" ID="Channel:0:56" Name="Gal9" SamplesPerPixel="1">
<LightPath/>
<Channel Color="1358106367" ID="Channel:0:57" Name="VEGFR2" SamplesPerPixel="1">
<LightPath/>
<Channel Color="-74044417" ID="Channel:0:58" Name="EZH2" SamplesPerPixel="1">
<LightPath/>
<Channel Color="-1188299777" ID="Channel:0:59" Name="GATA3" SamplesPerPixel="1">
<LightPath/>
<Channel Color="737932799" ID="Channel:0:60" Name="NRP2" SamplesPerPixel="1">
<LightPath/>
<Channel Color="-81033473" ID="Channel:0:61" Name="BAD" SamplesPerPixel="1">
<LightPath/>
<Channel Color="-68015105" ID="Channel:0:62" Name="SURVIVIN" SamplesPerPixel="1">
<LightPath/>
<Channel Color="-928083457" ID="Channel:0:63" Name="DAPI_R03" SamplesPerPixel="1">
<LightPath/>
<Channel Color="-1" ID="Channel:0:64" Name="H3K27" SamplesPerPixel="1">
<LightPath/>

As such I have modify the workflow to generate DeepCell Mesmer segmentation masks on those files by giving the memser application the same file for nuclear and membrane input, and selecting my chosen index separately, like so:

## whole-cell segmentation
NCHANNEL=0
MCHANNEL=26   # Nakaptase - see metadata above
apptainer run --unsquash -H $bse -B $wkfldir,$inImgDir $bse/deepcell.sif mesmer \
 --nuclear-image $imgfov \
 --membrane-image $imgfov \
 --nuclear-channel $NCHANNEL \
 --membrane-channel $MCHANNEL \
 --output-directory $OUTDIR \
 --output-name ${prefix_name}__whole_cell.tiff \
 --compartment whole-cell

My directory structure looks like this:

|-- base_dir
|   |-- image_data
|   |   |-- fov_1.ome.tiff
|   |   |-- fov_2.ome.tiff
|   |-- segmentation
|   |   |-- fov_1_whole_cell.tiff
|   |   |-- fov_2_whole_cell.tiff
|   |-- nimbus_output

Can you help me to find away to leverage my image data as input to Nimbus, in it's current OME.TIFF format? I'd rather not write an ELT to 'unpack' a single file OME into 64 individual files for each and every FOV if I can avoid it.

ngreenwald commented 8 months ago

@JLrumberger, I talked with @raymond301 at the conference I presented at a couple months ago.

My sense is that right now, all of the I/O is baked around the directory structure, but the actual model inference doesn't depend on this at all. The simplest would be to just call the single-channel inference directly. However, the normalization computation also depends on the folder structure, right?

Moving forward, it's definitely the case that people with OME-TIFF or other TIFF-based formats will want to use it. What would be the easiest way to abstract the data storage format so this isn't an issue?

JLrumberger commented 8 months ago

Hi @raymond301 and @ngreenwald,

thanks for opening this issue Raymond. I agree that this enhancement would totally make sense, since many institutions run OMERO servers. Yes, I/O for inference assumes a given directory structure, but it should be possible without too much hassle to add functionality for ome.tiffs. I'd need to change a few bits in Nimbus/src/cell_classification/inference.py and Nimbus/src/cell_classification/viewer_widget.py to make it work. I'll write a design doc and will implement it in the coming weeks.

raymond301 commented 8 months ago

I will happy-ily test the new functionality when it becomes available!

JLrumberger commented 2 weeks ago

Inference code moved to this repo Nimbus-inference. We included a wrapper for .ome.tiff files.