Limit to Number of Image Params Saved?

[cmbean2]

Hi,

I am working on using your package in order to stitch High Resolution SEM microscopy images that are taken in a snake like grid pattern. I am having issue with the package only saving out the first 200 image coordinates while I have 240 images in a 80x3 grid array. However, it does provide the filenames for all 240 images in the json file.

I have been using the structured mosaic aspect of the package to do this:

from stitch2d import StructuredMosaic

StructuredMosaic.num_cores = 1
mosaic= StructuredMosaic(
    Step0_SEM_Images_Folder,
    dim = 80, # number of tiles in primary axis
    origin="upper left", # position of first tile
    direction="horizontal",  # primary axis (i.e., the direction to traverse first)
    pattern="snake"  # snake or raster
    )

mosaic.align() 
mosaic.save_params(SEM_Params_Filename_Path)
mosaic.save(SEM_Stitched_Image_Path) 

Similarly, the images I am using are high resolution each small image is 6144 by 4096 pixels and the code takes a very long time to run (over 300 min), but if I scale the images down by a factor of 3 then it take only 20 seconds.

If helpful I can provide the images or the output json.

I'd appreciate any help you can offer as well as want to thank you for doing this development as this fits my application very well.

Thanks, Chris

[adamancer]

Hi Chris! There isn't a hard limit on the number of tiles (or there isn't supposed to be anyway). It's hard to say what's going on here without seeing your tiles, but one possibility is that the image detection algorithm isn't able to find/match enough features on the missing tiles to place them. One way to test that would be to use the build_out() method, which estimates the placement of the missing tiles based on the tiles that have already been placed.

In regards to your comment about performance, you may want to use the downsample() method prior to aligning the mosaic. That method resizes the tiles for the feature matching step; the tiles can then be restored to full size using reset_tiles() after alignment is complete. As you've seen, feature detection and matching is very, very slow on large images, and it is common to resize them to make this step faster.

Here's your code with those suggestions included:

from stitch2d import StructuredMosaic

StructuredMosaic.num_cores = 1
mosaic= StructuredMosaic(
    Step0_SEM_Images_Folder,
    dim = 80, # number of tiles in primary axis
    origin="upper left", # position of first tile
    direction="horizontal",  # primary axis (i.e., the direction to traverse first)
    pattern="snake"  # snake or raster
    )

mosaic.downsample(0.25)  # downsample tiles for alignment
mosaic.align()
mosaic.build_out(from_placed=True)  # place tiles that could not be aligned
mosaic.reset_tiles()  # restore tiles to full size
mosaic.save_params(SEM_Params_Filename_Path)
mosaic.save(SEM_Stitched_Image_Path) 

Are you able to save the partial mosaic using your original code? Depending on the file format and number of channels, that mosaic may be too large for some file formats (for example, the maximum size for a TIFF is 4 GB, and I'd expect the mosaic described here to be much larger than that).

[cmbean2]

I will work on retesting the script with the code you provided here and will see if that addresses the problem.

It is fine for me if we don't save out the full large high resolution image, but I do need to coordinates of all the images to ideally be calculated compared to simply guessed from the others as there is a potential for error if not placed right.

Attached are the images I am using if you are willing to look and help. I am also attaching the saved json and images for both runs. '_raw' ran without downscaling. Without is run with downscaling manually the images, separate from this package. The images do save out at the moment, but the size will soon be an issue as you said, but the lack of coordinates is more the challenge.

StitchedImages_and_json.zip

The images could not be uploaded to github as there are too many but can be found here.

[cmbean2]

Your code results in a RuntimeError: Could not align tiles

[adamancer]

I can't open either of those zipfiles. The stitch images file is empty and the tiles file is corrupt. Are you able to open them?

You can try a higher value in downsample (like 0.5) to see if that yields a result.

[cmbean2]

I have updated the folder on the box and all of the files I mentioned should now be there. Link

[adamancer]

I reviewed your tiles. They appear to be in raster format, and one tile is missing (row 2, tile 75). Is it possible that the program you're using to collect the images is converting them to a raster automatically?

I was able to get the other 239 tiles to align. I did misremember how the downsample method works--I was thinking it was relative (e.g., resize by 25% or 50% in the code I asked you to try) but the argument is actually megapixels. Here is some updated code to try:

from stitch2d import StructuredMosaic

StructuredMosaic.num_cores = 1
mosaic= StructuredMosaic(
    Step0_SEM_Images_Folder,
    dim = 80, # number of tiles in primary axis
    origin="upper left", # position of first tile
    direction="horizontal",  # primary axis (i.e., the direction to traverse first)
    pattern="raster"  # snake or raster
    )

mosaic.downsample(4)
mosaic.align()
mosaic.reset_tiles()
mosaic.save_params(SEM_Params_Filename_Path)
mosaic.save(SEM_Stitched_Image_Path)

I'm going to think about how to make the build_out() method less opaque (for example, by providing an estimate of how accurately it places tiles relative to the feature-based aligning). Your tiles seem to be feature-rich but tilesets I've worked in on the past sometimes have gaps of one kind of another, so it would be good to get a handle on that anyway.

[cmbean2]

The images are labeled as if they were taken as a raster, but they are taken experimentally in a serpentine format, but saved out such that they are labeled with "step0{x}{y}.tif" for where they fall relative to the first image, which would be top left as step0_1_1.tif. So if raster is the correct command I will use that.

I missed 2_75 in the upload originally, but it is now included in the folder should it be needed.

So when using the Raster setting it saved all 240 (or 239) image coordinates out for the grid into the json? I am testing the code you gave now, how long did it take to run for you?

Thank you again for looking into this!

[adamancer]

Not a problem. Yeah, the script only cares about the filenames so raster should be the right command. That code took about fifteen minutes to run on an ancient MacBook. If you're curious about progress, you can add a logging command to the top of your script.

[cmbean2]

This seems to have solved my issue and I am now getting the coordinates out for all 240 images.

If it matters took 5 minutes to run with that downsample(4) .

The resultant image is too large to open, but I simply needed the coordinates to stitch a grid of images that are 1/3 the size on both axes and I now have those. Thank you again!