search

labsyspharm / mcmicro

Multiple-choice microscopy pipeline
https://mcmicro.org/
MIT License
103 stars 58 forks source link

Channels metadata info in sequential staining #377

Closed asmagen closed 2 years ago

asmagen commented 2 years ago

I'm trying to apply the method on sequential staining (MICSSS) and so far I encountered the following issues:

  1. When using a pre-registered tile (one PNG file for each marker) and following the advise in this earlier issue I run and get the following: 
    
N E X T F L O W  ~  version 20.10.0
Launching `../main.nf` [shrivelled_ramanujan] - revision: e375cef610
executor >  lsf (3)
[-        ] process > illumination                    -
executor >  lsf (3)
[-        ] process > illumination                    -
[-        ] process > registration:ashlar             -
[-        ] process > dearray:coreograph              -
[ae/d317a7] process > segmentation:worker (unmicst-1) [100%] 1 of 1 ✔
[88/0b2462] process > segmentation:s3seg (1)          [100%] 1 of 1 ✔
[c8/5fdd51] process > quantification:mcquant (1)      [100%] 1 of 1, failed: 1 ✘
[-        ] process > cellstates:worker               -
Error executing process > 'quantification:mcquant (1)'

Caused by: Process quantification:mcquant (1) terminated with an error exit status (1)

Command executed:

python /app/CommandSingleCellExtraction.py --image unmicst-1003_bx.ome.tif --masks cell*.tif --output . --channel_names markers.csv

Command exit status: 1

Command output: {'masks': ['cell.ome.tif'], 'image': 'unmicst-1003_bx.ome.tif', 'channel_names': 'markers.csv', 'output': '.', 'intensity_props': {'intensity_mean'}, 'mask_props': None} Extracting single-cell data for unmicst-1003_bx.ome.tif...

Command error: Traceback (most recent call last): File "/app/CommandSingleCellExtraction.py", line 11, in SingleCellDataExtraction.MultiExtractSingleCells(**args) File "/app/SingleCellDataExtraction.py", line 263, in MultiExtractSingleCells ExtractSingleCells(masks,image,channel_names,output, mask_props=mask_props, intensity_props=intensity_props) File "/app/SingleCellDataExtraction.py", line 215, in ExtractSingleCells raise Exception("The number of channels in %s doesn't match the image"%channel_names) Exception: The number of channels in markers.csv doesn't match the image

Work dir: /sc/arion/scratch/magena01/mcmicro/testing/work/c8/5fdd5130d8e080f4d635f9c3822395

Tip: view the complete command output by changing to the process work dir and entering the command cat .command.out


I tried having the markers file list each marker (corresponding to different files, not image channels) or alternatively just one row named 'all' to indicate that each image has one stain, but both fail. How do I indicate the image channels in this example of a single stain per png image?

2. Not sure if it would be possible to use the original SVS files as input (not registered)?

Thank you
ArtemSokolov commented 2 years ago

Hi @asmagen,

  1. Can you share what your markers.csv looks like? Also, how many channels does unmicst-1003_bx.ome.tif have?

  2. We haven't tested the pipeline with .svs files, but they appear to be bio-formats compatible, so it should work theoretically. You can tell MCMICRO what formats to look for explicitly in the raw/ folder.

asmagen commented 2 years ago

I tried a file containing: cycle,marker_name 1,CD3 2,CD8 3,FOXP3 4,CD20

as well as cycle,marker_name 1,all

and both failed with the error I mentioned in the thread.

Yu-AnChen commented 2 years ago

It seems like you are working with multiplexed IHC type of method and generating brightfield images. The SVS files are likely already stitched. Currently, I would suggest using palom to generate a "registered" image (ome-tiff), place the ome-tiff in the registration directory and start the pipeline from the segmentation step.

To solve the markers.csv validation issue, please inspect the ome-tiff with fiji or qupath to see how many channels are in the output file - the markers.csv must have the same number of rows (excluding the header).

asmagen commented 2 years ago

But I already have very well registered images, a process I would really like to avoid redoing.

Also that registration outputs each AEC channel after masking as a combined TIF file. The issue is that they do not contain any non-AEC channels and that their assignments are as “Stacks” instead of “Channels”.

Aren't the pre-registered png images I have or this combined file sufficient to run this package without realignment?

From: Yu-An Chen @.> Sent: Thursday, April 7, 2022 11:49 PM To: labsyspharm/mcmicro @.> Cc: Magen, Assaf @.>; Mention @.> Subject: Re: [labsyspharm/mcmicro] Channels metadata info in sequential staining (Issue #377)

USE CAUTION: External Message.

It seems like you are working with multiplexed IHC type of method and generating brightfield images. The SVS files are likely already stitched. Currently, I would suggest using palomhttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_Yu-2DAnChen_palom&d=DwMCaQ&c=shNJtf5dKgNcPZ6Yh64b-A&r=SInwANxfOqZV8sfAkxasc63IcQTR_urEPBUbRUaFT94&m=2h301ylsDvicJKOrgjMdSarM_t_M0FgVM9MYxwJRK_Y&s=4PLvssMeuK1yNHIcRClRshu5kUGYMJLLyEBLQpKzWGw&e= to generate a "registered" image (ome-tiff), place the ome-tiff in the registration directoryhttps://urldefense.proofpoint.com/v2/url?u=https-3A__mcmicro.org_instructions_nextflow_-23output&d=DwMCaQ&c=shNJtf5dKgNcPZ6Yh64b-A&r=SInwANxfOqZV8sfAkxasc63IcQTR_urEPBUbRUaFT94&m=2h301ylsDvicJKOrgjMdSarM_t_M0FgVM9MYxwJRK_Y&s=kGlk4F5CFsjDVJswnHMQVe8ELga42JT8hVbQfyVccJI&e= and start the pipeline from the segmentation step.

To solve the markers.csv validation issue, please inspect the ome-tiff with fiji or qupath to see how many channels are in the output file - the markers.csv must have the same number of rows (excluding the header).

— Reply to this email directly, view it on GitHubhttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_labsyspharm_mcmicro_issues_377-23issuecomment-2D1092413900&d=DwMCaQ&c=shNJtf5dKgNcPZ6Yh64b-A&r=SInwANxfOqZV8sfAkxasc63IcQTR_urEPBUbRUaFT94&m=2h301ylsDvicJKOrgjMdSarM_t_M0FgVM9MYxwJRK_Y&s=CRU1rFcFrkEn98TJxlym7mcnNQmy2Ii3HNXQLS7rpwE&e=, or unsubscribehttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_notifications_unsubscribe-2Dauth_ACWGV4AM3JPKZTV5B3UDIXDVD6UFLANCNFSM5S2DYARQ&d=DwMCaQ&c=shNJtf5dKgNcPZ6Yh64b-A&r=SInwANxfOqZV8sfAkxasc63IcQTR_urEPBUbRUaFT94&m=2h301ylsDvicJKOrgjMdSarM_t_M0FgVM9MYxwJRK_Y&s=Il_5WIGnCqoHRaDM52c3HjhX7coM6Wl1lSjKvKZn0jY&e=. You are receiving this because you were mentioned.Message ID: @.***>

Yu-AnChen commented 2 years ago

I see, sorry I misunderstood. Did you do stain separation to extract AEC stains into single channels? What I have done in the past is to run stain separation and generate a stack of separated colors where each channel corresponds to one stain. And I use the generated multichannel TIFF as the input for mcmicro, and the rows in markers.csv map to each of the channels.

ArtemSokolov commented 2 years ago

@asmagen, when you open unmicst-1003_bx.ome.tif in an image viewer (FIJI or Napari), how many channels do you see?

The following format you used is correct:

cycle,marker_name
1,CD3
2,CD8
3,FOXP3
4,CD20

but MCMICRO is not detecting 4 channels in your image, which is why you are seeing the error.

asmagen commented 2 years ago

Where can I find it?

From: Artem Sokolov @.> Sent: Friday, April 8, 2022 5:00 PM To: labsyspharm/mcmicro @.> Cc: Magen, Assaf @.>; Mention @.> Subject: Re: [labsyspharm/mcmicro] Channels metadata info in sequential staining (Issue #377)

USE CAUTION: External Message.

@asmagenhttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_asmagen&d=DwMCaQ&c=shNJtf5dKgNcPZ6Yh64b-A&r=SInwANxfOqZV8sfAkxasc63IcQTR_urEPBUbRUaFT94&m=xsvJt3ySp5JCCB9I-NPtc3G2N5jt4l7R4VPQIOORPy8&s=dg5-zl1l4_wEqdzkfILAYthlGneEM3sDDdVaeTyZYPs&e=, when you open unmicst-1003_bx.ome.tif in an image viewer (FIJI or Napari), how many channels do you see?

The following format you used is correct:

cycle,marker_name 1,CD3 2,CD8 3,FOXP3 4,CD20

but MCMICRO is not detecting 4 channels in your image, which is why you are seeing the error.

— Reply to this email directly, view it on GitHubhttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_labsyspharm_mcmicro_issues_377-23issuecomment-2D1093358877&d=DwMCaQ&c=shNJtf5dKgNcPZ6Yh64b-A&r=SInwANxfOqZV8sfAkxasc63IcQTR_urEPBUbRUaFT94&m=xsvJt3ySp5JCCB9I-NPtc3G2N5jt4l7R4VPQIOORPy8&s=5jX8W8-PBDf8DvC6qARisFjyQ71QfV474xXqQsDK9_0&e=, or unsubscribehttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_notifications_unsubscribe-2Dauth_ACWGV4GAIW5PTV6N7ERZJ5TVECNAJANCNFSM5S2DYARQ&d=DwMCaQ&c=shNJtf5dKgNcPZ6Yh64b-A&r=SInwANxfOqZV8sfAkxasc63IcQTR_urEPBUbRUaFT94&m=xsvJt3ySp5JCCB9I-NPtc3G2N5jt4l7R4VPQIOORPy8&s=g1l6dLWN0munuLgLQtf3-70lA7TGiyznP6YOQl74cms&e=. You are receiving this because you were mentioned.Message ID: @.***>

ArtemSokolov commented 2 years ago

FIJI and Napari are publicly available image viewers that can be downloaded from https://imagej.net/software/fiji/ and https://napari.org/, respectively. The documentation of each viewer can give you more details about loading images, but finding the number of channels in both viewers should be pretty straightforward.

For example, when I process exemplar-001 with MCMICRO and open registration/exemplar-001.ome.tif in FIJI, it presents me with the following menu:

image

From here, I can tell that the file contains an image pyramid with three levels, and each level contains 12 channels. Since my .ome.tif contains 12 channels, the matching markers.csv must contain 12 marker names.

ArtemSokolov commented 2 years ago

Closing due to inactivity. Please reopen if the issue persists.