Working with IF Images from NanoString's GeoMx

SalimSoria commented 2 months ago

I'm currently working with NanoString GeoMx ome.tiff whole-slide IF images containing the following four channels: FITC, Cy3, TexasRed, and Cy5. Ideally, I would want to use MCMICRO as the end-to-end pipeline for marker quantification, but my current goal is to simply create a segmentation mask for my tissues.

I've now gotten Exemplar-001 and Exemplar-002 to complete using the Unmicst segmentation module. I haven't been able to do this on my image.

Some of my concerns of the tissue are:

DNA staining was performed with SYTO-13 and not DAPI, could I still use unmicst? I get that the model is trained with DAPI, but I wish to compare its results with QuPath's built-in cell detection.
There's a chance my images may be too large. I've gotten nump.core_exceptions.MemoryError with cellpose and unmicst stating that it can't allocate an X amount of GiB. Is there any way to crop or reduce the area where unmicst is applied? My ome.tiff images contains a lot of blank space that separates the two sections of tissue. However, once I used the scalingFactor parameter in params.yml, the image no longer produced an error but instead was stuck here for multiple hours:

$ cgexec -g cpuset:24cores nextflow run labsyspharm/mcmicro --in ./SLE004 -profile singularity,GPU --start-at segmentation --stop-at quantification
N E X T F L O W  ~  version 23.10.1
Launching `https://github.com/labsyspharm/mcmicro` [fervent_turing] DSL2 - revision: 69ee2efe21 [master]
executor >  local (1)
[-        ] process > illumination                    -
[-        ] process > registration:ashlar             -
[-        ] process > background:backsub              -
[-        ] process > dearray:coreograph              -
[-        ] process > dearray:roadie:runTask          -
[-        ] process > segmentation:roadie:runTask     -
[f9/475255] process > segmentation:worker (unmicst-1) [  0%] 0 of 1
[-        ] process > segmentation:s3seg              -
[-        ] process > quantification:mcquant          -
[-        ] process > downstream:worker               -
[-        ] process > viz:autominerva                 -

Here is some of the metadata for one of my images: Screen Shot 2024-04-30 at 10 21 53 AM

Here was the output log when I tried using cellpose and the run failed:

N E X T F L O W  ~  version 23.10.1
Launching `https://github.com/labsyspharm/mcmicro` [thirsty_austin] DSL2 - revision: 69ee2efe21 [master]
executor >  local (1)
[-        ] process > illumination                     -
[-        ] process > registration:ashlar              -
[-        ] process > background:backsub               -
[-        ] process > dearray:coreograph               -
[-        ] process > dearray:roadie:runTask           -
[-        ] process > segmentation:roadie:runTask      -
[ea/437f16] process > segmentation:worker (cellpose-1) [  0%] 0 of 1
[-        ] process > segmentation:s3seg               -
[-        ] process > quantification:mcquant           -
[-        ] process > downstream:worker                -
[-        ] process > viz:autominerva                  -
ERROR ~ Error executing process > 'segmentation:worker (cellpose-1)'

Caused by:
  Process `segmentation:worker (cellpose-1)` terminated with an error exit status (1)

Command executed:

  cellpose --channel_axis 0 --save_tif --savedir . --verbose --image_path SLE004.ome.tiff

Command exit status:
  1

Command output:
  creating new log file
  2024-04-30 10:07:15,581 [INFO] WRITING LOG OUTPUT TO .cellpose/run.log
  2024-04-30 10:07:15,581 [INFO] >>>> using CPU
  2024-04-30 10:07:15,581 [INFO] >>>> running cellpose on 1 images using chan_to_seg GRAY and chan (opt) NONE
  2024-04-30 10:07:15,581 [INFO] >>>> using CPU
  2024-04-30 10:07:15,581 [INFO] >> cyto << model set to be used
  2024-04-30 10:07:15,582 [INFO] Downloading: "https://www.cellpose.org/models/cytotorch_0" to .cellpose/models/cytotorch_0

  2024-04-30 10:07:17,982 [INFO] Downloading: "https://www.cellpose.org/models/cytotorch_1" to .cellpose/models/cytotorch_1

executor >  local (1)
[-        ] process > illumination                     -
[-        ] process > registration:ashlar              -
[-        ] process > background:backsub               -
[-        ] process > dearray:coreograph               -
[-        ] process > dearray:roadie:runTask           -
[-        ] process > segmentation:roadie:runTask      -
[ea/437f16] process > segmentation:worker (cellpose-1) [100%] 1 of 1, failed: 1 ✘
[-        ] process > segmentation:s3seg               -
[-        ] process > quantification:mcquant           -
[-        ] process > downstream:worker                -
[-        ] process > viz:autominerva                  -
ERROR ~ Error executing process > 'segmentation:worker (cellpose-1)'

Caused by:
  Process `segmentation:worker (cellpose-1)` terminated with an error exit status (1)

Command executed:

  cellpose --channel_axis 0 --save_tif --savedir . --verbose --image_path SLE004.ome.tiff

Command exit status:
  1

Command output:
  creating new log file
  2024-04-30 10:07:15,581 [INFO] WRITING LOG OUTPUT TO .cellpose/run.log
  2024-04-30 10:07:15,581 [INFO] >>>> using CPU
  2024-04-30 10:07:15,581 [INFO] >>>> running cellpose on 1 images using chan_to_seg GRAY and chan (opt) NONE
  2024-04-30 10:07:15,581 [INFO] >>>> using CPU
  2024-04-30 10:07:15,581 [INFO] >> cyto << model set to be used
  2024-04-30 10:07:15,582 [INFO] Downloading: "https://www.cellpose.org/models/cytotorch_0" to .cellpose/models/cytotorch_0

  2024-04-30 10:07:17,982 [INFO] Downloading: "https://www.cellpose.org/models/cytotorch_1" to .cellpose/models/cytotorch_1

  2024-04-30 10:07:20,208 [INFO] Downloading: "https://www.cellpose.org/models/cytotorch_2" to .cellpose/models/cytotorch_2

  2024-04-30 10:07:22,452 [INFO] Downloading: "https://www.cellpose.org/models/cytotorch_3" to .cellpose/models/cytotorch_3

  2024-04-30 10:07:24,962 [INFO] >>>> model diam_mean =  30.000 (ROIs rescaled to this size during training)
  2024-04-30 10:07:24,962 [INFO] Downloading: "https://www.cellpose.org/models/size_cytotorch_0.npy" to .cellpose/models/size_cytotorch_0.npy

  2024-04-30 10:07:25,249 [INFO] >>>> using diameter 30.000 for all images
  2024-04-30 10:07:25,249 [INFO] 0%|          | 0/1 [00:00<?, ?it/s]
  2024-04-30 10:07:44,564 [INFO] ~~~ FINDING MASKS ~~~
  2024-04-30 10:12:12,769 [INFO] 0%|          | 0/1 [04:47<?, ?it/s]

Command error:
  100%|██████████| 25.3M/25.3M [00:01<00:00, 14.1MB/s]2024-04-30 10:07:22,452 [INFO] Downloading: "https://www.cellpose.org/models/cytotorch_3" to .cellpose/models/cytotorch_3

    0%|          | 0.00/25.3M [00:00<?, ?B/s]
    0%|          | 48.0k/25.3M [00:00<00:58, 453kB/s]
    1%|          | 208k/25.3M [00:00<00:30, 860kB/s] 
    2%|▏         | 512k/25.3M [00:00<00:15, 1.68MB/s]
    5%|▌         | 1.34M/25.3M [00:00<00:06, 4.17MB/s]
   13%|█▎        | 3.22M/25.3M [00:00<00:02, 8.07MB/s]
   20%|█▉        | 5.05M/25.3M [00:00<00:01, 11.1MB/s]
   28%|██▊       | 7.15M/25.3M [00:00<00:01, 14.3MB/s]
   36%|███▌      | 9.14M/25.3M [00:00<00:01, 16.2MB/s]
   42%|████▏     | 10.7M/25.3M [00:01<00:01, 14.9MB/s]
   49%|████▊     | 12.3M/25.3M [00:01<00:00, 15.3MB/s]
   56%|█████▌    | 14.2M/25.3M [00:01<00:00, 16.5MB/s]
   65%|██████▍   | 16.3M/25.3M [00:01<00:00, 18.1MB/s]
   71%|███████▏  | 18.1M/25.3M [00:01<00:00, 18.0MB/s]
   78%|███████▊  | 19.9M/25.3M [00:01<00:00, 16.6MB/s]
   86%|████████▌ | 21.7M/25.3M [00:01<00:00, 17.3MB/s]
   94%|█████████▎| 23.7M/25.3M [00:01<00:00, 18.4MB/s]
  100%|██████████| 25.3M/25.3M [00:01<00:00, 14.1MB/s]2024-04-30 10:07:24,962 [INFO] >>>> model diam_mean =  30.000 (ROIs rescaled to this size during training)
  2024-04-30 10:07:24,962 [INFO] Downloading: "https://www.cellpose.org/models/size_cytotorch_0.npy" to .cellpose/models/size_cytotorch_0.npy

    0%|          | 0.00/5.23k [00:00<?, ?B/s]
  100%|██████████| 5.23k/5.23k [00:00<00:00, 30.7MB/s]2024-04-30 10:07:25,249 [INFO] >>>> using diameter 30.000 for all images
  2024-04-30 10:07:25,249 [INFO] 0%|          | 0/1 [00:00<?, ?it/s]
  2024-04-30 10:07:44,564 [INFO] ~~~ FINDING MASKS ~~~
  2024-04-30 10:12:12,769 [INFO] 0%|          | 0/1 [04:47<?, ?it/s]

  Traceback (most recent call last):
    File "/usr/local/bin/cellpose", line 8, in <module>
      sys.exit(main())
    File "/usr/local/lib/python3.8/site-packages/cellpose/__main__.py", line 246, in main
      out = model.eval(image, channels=channels, diameter=diameter,
    File "/usr/local/lib/python3.8/site-packages/cellpose/models.py", line 239, in eval
      masks, flows, styles = self.cp.eval(x, 
    File "/usr/local/lib/python3.8/site-packages/cellpose/models.py", line 552, in eval
      masks, styles, dP, cellprob, p = self._run_cp(x, 
    File "/usr/local/lib/python3.8/site-packages/cellpose/models.py", line 616, in _run_cp
      yf, style = self._run_nets(img, net_avg=net_avg,
    File "/usr/local/lib/python3.8/site-packages/cellpose/core.py", line 367, in _run_nets
      y0, style = self._run_net(img, augment=augment, tile=tile, 
    File "/usr/local/lib/python3.8/site-packages/cellpose/core.py", line 441, in _run_net
      y, style = self._run_tiled(imgs, augment=augment, bsize=bsize, 
    File "/usr/local/lib/python3.8/site-packages/cellpose/core.py", line 539, in _run_tiled
      y = np.zeros((IMG.shape[0], nout, ly, lx))
  numpy.core._exceptions.MemoryError: Unable to allocate 86.7 GiB for an array with shape (77264, 3, 224, 224) and data type float64

Work dir:
  /localtmp/ssoria/work/ea/437f16b602365bb2fe997259ff5807

Tip: view the complete command output by changing to the process work dir and entering the command `cat .command.out`

 -- Check '.nextflow.log' file for details

clarenceyapp commented 2 months ago

Hi @SalimSoria UnMICST can possibly work with Syto13 depending on how similar it looks to Hoechst/DAPI. Just specify the channel number as an option in the params file. Both cellpose and UnMICST require a certain amount of RAM allocation for the output masks. This is independent on your GPU memory. How much RAM have you allocated and available to use?

When you downsampled the image, this allowed your image to fit into RAM however this may not yield optimal results since the pixel size between your image and the model's training data might be different. UnMICST is trained at 0.65 microns per pixel. Please confirm your image's pixel size. If it is still running for several hours, it is likely that we haven't fully solved your GPU driver issue and it is falling back on CPU (slower but will likely eventually finish).

SalimSoria commented 2 months ago

Hi @clarenceyapp

After about 5 hours, my tissue image passed the segmentation:worker (unmicst-1) step successfully then failed at the segmentation:s3seg. Any thoughts on what this could be?

Below is the .command.log:

Matplotlib is building the font cache; this may take a moment.
/app/S3segmenter.py:18: DeprecationWarning: Please use `uniform_filter` from the `scipy.ndimage` namespace, the `scipy.ndimage.filters` namespace is deprecated.
  from scipy.ndimage.filters import uniform_filter
Traceback (most recent call last):
  File "/app/S3segmenter.py", line 484, in <module>
    tissueCrop =tifffile.imread(imagePath,key=iChan)
  File "/usr/local/lib/python3.8/site-packages/tifffile/tifffile.py", line 800, in imread
    return tif.asarray(**kwargs)
  File "/usr/local/lib/python3.8/site-packages/tifffile/tifffile.py", line 3128, in asarray
    pages = self.pages._getlist(key)
  File "/usr/local/lib/python3.8/site-packages/tifffile/tifffile.py", line 5311, in _getlist
    return [getitem(key)]
  File "/usr/local/lib/python3.8/site-packages/tifffile/tifffile.py", line 5360, in _getitem
    self._seek(key)
  File "/usr/local/lib/python3.8/site-packages/tifffile/tifffile.py", line 5281, in _seek
    raise IndexError('index out of range')
IndexError: index out of range

clarenceyapp commented 2 months ago

@SalimSoria this appears similar to this thread. https://forum.image.sc/t/the-error-in-processing-s3seg/87313/55 There are two issues:

the original ome.tiff likely has channels concatenated in the Z -axis instead of the C-axis, which can be confirmed if you open it in ImageJ/Fiji. Go to Image/Properties.
s3segmenter is looking for an additional channel (default is channel 3) to get the general tissue shape from autofluorescence. You can set this to the same channel as your nuclear marker (channel 1) as a test but it would be good to use the right channel if you have one.

labsyspharm / mcmicro

Working with IF Images from NanoString's GeoMx #548