Clarification of input data

[JiayuanDing100]

Hi,

When i was running example 1, it worked well. The example 1 data contains 3 cycles, 4 channels for each cycle. There are 24 images in ome.tiff file for each cycle, which equals to 4 channels * 6 FOVs. Please correct me if I am wrong.

I am wondering how the system identifies the number of FOVs. Because when I was running mcmicro on my own data, which is also 3 cycles, and 4 channels for each cycle, 6 FOVs. So I manually combine 24 images (6FOVs * 4 channels for each FOV) into single ome.tiff file. But I got the issue:

**```

Error executing process > 'quantification:mcquant (1)'

Caused by: Process quantification:mcquant (1) terminated with an error exit status (1)

Command executed:

shopt -s nullglob python /app/CommandSingleCellExtraction.py --image localdata--unmicst.ome.tif --masks cell*.tif --output . --channel_names markers.csv

Command exit status: 1

Command output: {'masks': ['cell.ome.tif'], 'image': 'localdata--unmicst.ome.tif', 'channel_names': 'markers.csv', 'output': '.', 'intensity_props': {'intensity_mean'}, 'mask_props': None} Extracting single-cell data for localdata--unmicst.ome.tif...

Command error: {'masks': ['cell.ome.tif'], 'image': 'localdata--unmicst.ome.tif', 'channel_names': 'markers.csv', 'output': '.', 'intensity_props': {'intensity_mean'}, 'mask_props': None} Extracting single-cell data for localdata--unmicst.ome.tif... Traceback (most recent call last): File "/app/CommandSingleCellExtraction.py", line 11, in SingleCellDataExtraction.MultiExtractSingleCells(**args) File "/app/SingleCellDataExtraction.py", line 263, in MultiExtractSingleCells ExtractSingleCells(masks,image,channel_names,output, mask_props=mask_props, intensity_props=intensity_props) File "/app/SingleCellDataExtraction.py", line 215, in ExtractSingleCells raise Exception("The number of channels in %s doesn't match the image"%channel_names) Exception: The number of channels in markers.csv doesn't match the image

[ArtemSokolov]

Hi @JiayuanDing100,

The FOVs get stitched together and aligned across cycles into a single multi-channel image. Quantification then works on that entire image. Here is how it verifies that the number of channels in the image matches the number of channels specified in markers.csv: https://github.com/labsyspharm/quantification/blob/master/SingleCellDataExtraction.py#L84-L99

In other words, it tries to guess the number of channels based on the image dimensionality. When exemplar-001 FOVs get stitched and aligned, we can check the resulting shape of the image with the tifffile package in Python:

$ python -c "import tifffile; print(tifffile.TiffFile('exemplar-001/registration/exemplar-001.ome.tif').series[0].shape)"
(12, 3139, 2511)

From here, quantification would determine that the image has 12 channels, which matches the number of markers specified in markers.csv.

Hopefully, this clarifies the process. If you are seeing that error message, then the dimensionality is off somewhere. If you run the above Python command (assuming your have tifffile installed), what does it report as the image dimensionality?

[JiayuanDing100]

Hi @ArtemSokolov ,

Very helpful!!!

I just took a look at the dimension of data-exemplar.ome.tif under the registration folder. It is (3, 1728, 1728). It is supposed to be (12, 1728, 1728) because there are 3 cycles and 4 channels per cycle, 6 FOVs.

I am wondering whether i made a mistake during combining 24 individual TIF (4 channels * 6 FOVs) into single ome.tiff file for each cycle. So basically how do you combine individual TIF into single ome.tiff file ? How does the system differentiate how many FOVs in the single ome.tiff file ?

Thanks,

[JiayuanDing100]

Hi @ArtemSokolov ,

One more experiment done from my side is that if i just input only one FOV same 3 cycles, 4 channels per cycle, it worked expectedly.

[ArtemSokolov]

Hi @JiayuanDing100,

MCMICRO uses ASHLAR (https://labsyspharm.github.io/ashlar/) to stitch and align individual FOVs. Depending on the image format, ASHLAR can read metadata to determine the microscope stage coordinates and the cycle number of individual image tiles. Do your .tif files come with any additional metadata files, or is the order of tiles encoded in the filenames?

If your files don't have associated metadata, you can still stitch them together using the (undocumented) fileseries feature of ASHLAR. There are a few examples in this thread: https://forum.image.sc/t/ashlar-how-to-pass-multiple-images-to-be-stitched/49864/4 The main idea is to tell ASHLAR what order your tiles were acquired by the instrument and how that order is encoded in the filename. Unfortunately, this feature is not propagated to MCMICRO, so you would need to run ASHLAR directly and then manually put the resulting .ome.tif into the registration/ folder. You can then run the rest of the pipeline by doing:

workflow:
  start-at: segmentation

in your params.yml (see https://mcmicro.org/parameters/workflow.html#start-at). MCMICRO will then find the .ome.tif and proceed from there.

[jmuhlich]

The exemplar data uses OME-XML metadata in the "header" of the .ome.tif files to describe important metadata like the number of FOVs and their initial stage positions, which ashlar can then read. For the exemplar we converted our microscope's native image format (.rcpnl) to .ome.tif using bioformats, choosing to extract only a small subset of the the hundreds of FOVs in the original files.

Here is a thread on image.sc with some ideas about how to convert your own data into .ome.tif which can then be passed directly to mcmicro and ashlar. https://forum.image.sc/t/microscope-image-stitching-in-python/75871/10

[JiayuanDing100]

Hi @ArtemSokolov,

Thanks for your help! I refer to your link(https://forum.image.sc/t/ashlar-how-to-pass-multiple-images-to-be-stitched/49864/4) to convert multiple FOVs to ome.tiff file. Since the fileseries is undocumented, I was struggling with those. Just repeat my steps below to verify with you where is wrong. Thank you so much!

Step 1: ashlar -o cycle1.ome.tif "fileseries|./cycle1_copy/|pattern=PIO51_cycle1_w{channel:1}_s{series:2}_t1.TIF|width=6|height=5|direction=vertical|overlap=0"

For example, one of filename is PIO51_cycle1_w1_s1.tif (w1 means channel 1, s1 means FOV 1). My FOV layout is below: S1 S6 S11 S2 S7 S12 S3 S8 S13 S4 S9 S14 S5 S10 S15 So I set the "direction" to be "vertical". After running the command line above, for each cycle, I can get one ome.tiff which contains four images. Each of four is fullview (multiple FOVs) corresponding to one protein marker channel.

Step 2: ashlar cycle1.ome.tif cycle2.ome.tif cycle3.ome.tif cycle4.ome.tif cycle5.ome.tif cycle6.ome.tif cycle7.ome.tif --align-channel 2 --maximum-shift 50 --filter-sigma 1 For this command, I am not pretty sure about the definition of "align-channel" what if the first channel in each cycle is DAPI for our case, what value should I set it?

I failed in the registration step, and it returned the error "Image series must all have the same dimensions".

It seems my metadata was not set correctly. But I am confused about where it was wrong.

[JiayuanDing100]

Hi,

The result I got from the command running "ashlar -o cycle1.ome.tif "fileseries|./cycle1_copy/|pattern=PIO51_cycle1_w{channel:1}_s{series:2}_t1.TIF|width=6|height=5|direction=vertical|overlap=0" is 4 overview images per cycle. Each overview image corresponds to a protein marker channel.

I am wondering whether there is a detailed tutorial about how to composite ome.tiff with individual FOV as a single image in ome.tiff from my own data, same with the demo example ome.tiff input data.

Thanks,

[JiayuanDing100]

Hi @ArtemSokolov @jmuhlich

I just checked our individual FOV tif image, and it actually includes metadata info. I am wondering how to composite them into ome.tiff same with demo example input data.

metadata for individual fov image for our data: https://www.dropbox.com/s/k72nlx6hoiwen1y/metadata_per_fov.txt?dl=0

[ArtemSokolov]

Hi @JiayuanDing100,

I think you should be able to provide all cycles at once, instead of running ASHLAR on each cycle separately. Can you try ashlar -o PIO51.ome.tif "fileseries|./path_to_all_files/|pattern=PIO51_cycle{cycle:1}_w{channel:1}_s{series:2}_t1.TIF|etc..." (where I am assuming your specify the cycle number with a single digit)? Replace path_to_all_files and etc... as appropriate.

@jmuhlich can provide more information. He is the main developer of ASHLAR.