Open arundasan91 opened 10 months ago
Hi @arundasan91, Just wanted to let you know that I'm planning to get this sorted out this week. I'll let you know how it goes.
Best, -Chandler
P.S. was good to meet you at the CSBC/PSON meeting last week
Hi @cdgatenbee,
It was great meeting you too at the meeting! Thanks a lot for looking into this.
Best, Arun
Hi I am wondering if you managed to solve this? I am having issues saving my images out too. And I would like to solve this soon as it works so well but now I can't move on. Thank you.
@arundasan91 , @Mystique27M, sorry for the delay, but I'm have a bit of trouble replicating this error. It looks like it may have something to do with the channel names, but I'm able to run the following without issue:
in_image_path = '/data/image_quadrants/membrane_quadrant1.ome.tiff'
out_folder = '/data/valis_pipeline_out/'
reference_img_fname = 'DAPI_quadrant1.ome.tiff'
image_px = 2000
micro_px = 3000
rigid = True
save = True
verbose = True
in_folder = os.path.dirname(in_image_path) # "/Users/arundas/images/membrane.ome.tiff" --> /Users/arundas/images/
in_file = os.path.basename(in_image_path) # membrane.ome.tiff
img_list = [os.path.join(in_folder, reference_img_fname), in_image_path]
ch_name_f = valtils.get_name(in_image_path)
ch_name_list = ch_name_f.split(" ")
slide_f = os.path.splitext(in_file)[0]
channel_names = ch_name_list
c = 1
default_colors = slide_tools.get_matplotlib_channel_colors(c)
colormap = {channel_names[i]:tuple(default_colors[i]) for i in range(c)}
channels = [slide_io.create_channel(i, name=channel_names[i], color=colormap[channel_names[i]]) for i in range(c)]
Does that also work for you?
I am also able to save a multichannel image without providing channel names, so it would seem having channel_names=None
is not the issue either.
Another potential source of this issue might be in creating the colormap, which is being done automatically right now. Are you able to save your slides when you provide the colormap, like this?
c = slide_obj.reader.metadata.n_channels
channel_colors = slide_tools.get_matplotlib_channel_colors(c)
colormap = {ch_name_list[i]:tuple(channel_colors[i]) for i in range(c)}
slide_obj.warp_and_save_slide(
out_path,
tile_wh=1024,
level=0,
compression="lzw",
channel_names=ch_name_list,
colormap=colormap
)
Since I'm unable to recreate this error, would either of you be able to share a small example dataset so that I can recreate this issue? If so, you can send me a link over email (Chandler.Gatenbee@moffitt.org) if you would prefer to keep the images private.
Best, -Chandler
Hi @cdgatenbee, Apologies for the late reply. Thanks for looking into it. I will try this over the weekend and update you.
Hi @arundasan91, Thanks again for sharing some images, they really went a long way in helping me figure out this issue, which should be fixed the most recent update (1.0.3) that got pushed on Friday. Please try updating valis and let me know if it resolves this issue for you.
Best, -Chandler
Thanks, Chandler! I will try this and let you know EOD today.
Arun
Hi @cdgatenbee,
I built a new docker image based on the latest Dockerfile and tried to run a simple example from the tutorial with the images I had shared with you before. This is the code I am using to register the DAPI and MS images together, with DAPI as the reference:
from valis import registration
max_non_rigid_registration_dim_px=2000
align_to_reference=True
slide_src_dir = "/data/src_imgs/"
results_dst_dir = "/data/output/"
registered_slide_dst_dir = results_dst_dir + "/registered_slides"
reference_slide = "DAPI_1-2C.ome.tiff"
# Create a Valis object and use it to register the slides in slide_src_dir, aligning *towards* the reference slide.
registrar = registration.Valis(slide_src_dir, results_dst_dir, reference_img_f=reference_slide,)
rigid_registrar, non_rigid_registrar, error_df = registrar.register()
# Perform micro-registration on higher resolution images, aligning *directly to* the reference image
registrar.register_micro(max_non_rigid_registration_dim_px=max_non_rigid_registration_dim_px, align_to_reference=align_to_reference)
I get the following error:
root@85437b3d29b5:/code/valis_code# python runReg.py
==== Converting images
Converting images: 100%|███████████████████████████████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 10.38image/s]
==== Processing images
Processing images : 50%|█████████████████████████████████████████████ | 1/2 [00:00<00:00, 8.35image/s]
/usr/local/src/valis/valtils.py:24: UserWarning: list index out of range
warnings.warn(warning_msg, warning_type)
Traceback (most recent call last):
File "/usr/local/src/valis/slide_io.py", line 923, in get_channel_index
best_match = get_close_matches(channel.upper(), cnames)[0]
IndexError: list index out of range
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/src/valis/registration.py", line 4166, in register
self.process_imgs(processor_dict=slide_processors)
File "/usr/local/src/valis/registration.py", line 2586, in process_imgs
processed_img = processor.process_image(**processing_kwargs)
File "/usr/local/src/valis/preprocessing.py", line 126, in process_image
chnl_idx = self.reader.get_channel_index(channel)
File "/usr/local/src/valis/slide_io.py", line 933, in get_channel_index
f" Using channel number {matching_channel_idx}, which has name {self.metadata.channel_names[matching_channel_idx]}")
IndexError: list index out of range
JVM has been killed. If this was due to an error, then a new Python session will need to be started
Traceback (most recent call last):
File "/code/valis_code/runReg.py", line 16, in <module>
registrar.register_micro(max_non_rigid_registration_dim_px=max_non_rigid_registration_dim_px, align_to_reference=align_to_reference)
File "/usr/local/src/valis/valtils.py", line 35, in wrapper
return f(*args, **kwargs)
File "/usr/local/src/valis/registration.py", line 4308, in register_micro
self.prep_images_for_large_non_rigid_registration(max_img_dim=max_non_rigid_registration_dim_px,
File "/usr/local/src/valis/registration.py", line 3475, in prep_images_for_large_non_rigid_registration
full_out_shape = self.get_aligned_slide_shape(s)
File "/usr/local/src/valis/registration.py", line 4500, in get_aligned_slide_shape
aligned_out_shape_rc = np.ceil(np.array(ref_slide.reg_img_shape_rc)*s_rc).astype(int)
TypeError: unsupported operand type(s) for *: 'NoneType' and 'float'
root@85437b3d29b5:/code/valis_code#
Could you please help me debug this?
Thanks, Arun
There are two images in the /data/src_imgs
folder. I can see that the registrar has the reference image loaded already.
Hi @arundasan91, Sorry you're running into another issue. I tried re-running your script using the images you shared previously (DAPI_1-2C.ome.tiff and MS_1-2C.ome.tiff) using my local environment, as well as the Docker container (available here), but in both cases the code ran without error. Below is the code and the output when using the docker container:
from valis import registration
reference_img_fname = 'DAPI_1-2C.ome.tiff'
max_non_rigid_registration_dim_px = 2000
align_to_reference = True
registrar = registration.Valis(in_folder, out_folder, reference_img_f=reference_img_fname)
rigid_registrar, non_rigid_registrar, error_df = registrar.register()
registrar.register_micro(max_non_rigid_registration_dim_px=max_non_rigid_registration_dim_px, align_to_reference=align_to_reference)
registrar.warp_and_save_slides(
out_folder,
level=0,
non_rigid=False,
crop=True,
interp_method="bicubic",
tile_wh=1024,
compression="lzw",
)
==== Converting images
Converting images: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 34.49image/s]
==== Processing images
Processing images : 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 5.47image/s]
Normalizing images: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 10.00image/s]
Denoising images : 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 7.02image/s]
==== Rigid registration
/usr/local/src/valis/valtils.py:24: UserWarning: ('The reference was specified as DAPI_1-2C ', 'but `align_to_reference` is `False`, and so images will be aligned serially. ', 'If you would like all images to be directly aligned to DAPI_1-2C, then set `align_to_reference` to `True`')
warnings.warn(warning_msg, warning_type)
Detecting features : 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 2/2 [00:09<00:00, 4.91s/image]
QUEUEING TASKS | Matching images : 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 694.65image/s]
PROCESSING TASKS | Matching images : 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████| 2/2 [00:05<00:00, 2.69s/image]
COLLECTING RESULTS | Matching images : 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 41943.04image/s]
Finding transforms : 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 1376.54image/s]
Finalizing : 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 5733.84image/s]
======== Rigid registration complete in 16.641 seconds
==== Non-rigid registration
Preparing images for non-rigid registration: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 5.83image/s]
/usr/local/src/valis/valtils.py:24: UserWarning: ('The reference was specified as DAPI_1-2C ', 'but `align_to_reference` is `False`, and so images will be aligned serially. ', 'If you would like all images to be directly aligned to DAPI_1-2C, then set `align_to_reference` to `True`')
warnings.warn(warning_msg, warning_type)
Finding non-rigid transforms: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:01<00:00, 1.24s/image]
======== Non-rigid registration complete in 1.362 seconds
==== Measuring error
Measuring error: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 12.39image/s]
Preparing images for non-rigid registration: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████| 2/2 [00:01<00:00, 1.28image/s]
==== Performing microregistration
Finding non-rigid transforms: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 2/2 [00:11<00:00, 5.86s/image]
======== Non-rigid registration complete in 12.329 seconds
==== Measuring error
Measuring error: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 3.43image/s]
Saving images: 0%| | 0/2 [00:00<?, ?image/s]saving /Users/gatenbcd/Dropbox/Documents/image_processing/valis_project/debugging/github_issue_78/arun_reg_docker/DAPI_1-2C.ome.tiff (42202 x 43314 and 1 channels)
[====================================================================================================] 100.0% in 2.298 minutes
Complete
Saving images: 50%|██████████████████████████████████████████████████████████████████ | 1/2 [02:18<02:18, 138.01s/image]some non-RGB channel names were `None` or not provided. Renamed channels are: ['C1']
/usr/local/src/valis/valtils.py:24: UserWarning: Missing colors in colormap for the following channels: {'C1'}. Will not try to add channel colors
warnings.warn(warning_msg, warning_type)
some non-RGB channel names were `None` or not provided. Renamed channels are: ['C1']
saving /Users/gatenbcd/Dropbox/Documents/image_processing/valis_project/debugging/github_issue_78/arun_reg_docker/MS_1-2C.ome.tiff (42202 x 43314 and 1 channels)
[====================================================================================================] 100.0% in 2.259 minutes
Complete
Saving images: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 2/2 [04:33<00:00, 136.96s/image]
Could you maybe see if you are able to get the code to run with the docker container that's on Dockerhub?
Best, -Chandler
Hi @cdgatenbee ,
Thanks for looking into this. I was able to get this up and running as well; thanks a lot.
However, there are a few concerns left:
align_to_reference
is set to True
. However, the logs say otherwise. Could you please check? I see that both DAPI and MS are saved in the out_folder
. My idea is to register MS to DAPI and save the MS image. Is there a way I can ensure that no transformations are applied to the reference image?: ==== Rigid registration
/usr/local/src/valis/valtils.py:24: UserWarning: ('The reference was specified as DAPI_1-2C ', 'but `align_to_reference` is `False`, and so images will be aligned serially. ', 'If you would like all images to be directly aligned to DAPI_1-2C, then set `align_to_reference` to `True`')
warnings.warn(warning_msg, warning_type)
out_folder
has all channel names as None
and all channel colors as #ff7800
. Is there a way I can perhaps pass a dictionary with channel names and colors? I understand that they are captured from the images themselves, please advise./usr/local/src/valis/valtils.py:22: UserWarning: Number of colors in colormap not same as the number of channels. Using default colormap
warnings.warn(warning_msg, warning_type)
saving /data/output/registered_slides/MS_1-2C.ome.tiff (42202 x 43314 and 3 channels)
[====================================================================================================] 100.0% in 3.436 minutes
Complete
No series provided. Selecting series with largest image, which is series 0
.Best, Arun
Hi @arundasan91, Glad you were able to get it working! I'll start looking into these other issues.
Best, -Chandler
Hi @arundasan91, I just pushed an update (1.0.4) that I think should address these issues:
check_for_no_transforms_in_ref
, to test that the reference image isn't being warped. It checks that there aren't any rigid transformations (rotation, scaling, shearing), and that the displacement fields for non-rigid registration are all zero. The transformation matrices do have translations to pad and crop each image (including the reference), but there are tests to make sure that the reference is cropped to the origin, and so should be the same as the original. This same logic is also used to test the point warping method. I did notice that this padding and cropping could result in the image having somewhat different values (at least for these uint16 images), which I believe is due to interpolation. To avoid this, valis now returns the original reference image when warping the reference image with crop="reference"
(which is set to true when align_to_reference=True
and/or reference_img_f
is not None
), and so the the function below also compares the values in the original and "warped" images as a final check. I've added most of these tests in the source code as well, so there are built in checks to make sure that the reference image isn't getting warped. If the logs still seem to indicate that the reference image is still being warped, please let me know.channel_name_dict
when using Valis.warp_and_merge_slides
, but right now there isn't any way to pass in channel names into Valis.warp_and_save_slides
(I should probably add this though). Luckily, it's pretty easy to get around this current limitation, as you can update each slide's metadata before writing each image (see below). You can, however, provide a colormap dictionary (key=filename, value=list of uint8 RGB colors) that can be used to set each channel's color (see below).I hope this update fixes these issues for you, but if not, please let me know.
Best, -Chandler
from valis import registration, valtils, slide_tools, warp_tools, slide_io
import numpy as np
def check_for_no_transforms_in_ref(ref_slide, reference_img_fname):
"""
Do checks to make sure that the reference image isn't be warped
"""
from skimage import transform
assert ref_slide.name == valtils.get_name(reference_img_fname), "Reference image is not the same as specified image"
og_crop = ref_slide.crop
ref_slide.crop = registration.CROP_REF
try:
tformer = transform.AffineTransform(ref_slide.M)
# M may have translations for cropping to overlap, but they are ignored when cropping to reference (verified below)
assert np.all(tformer.scale == [1.0, 1.0]), "Reference image has unexpected scaling"
assert tformer.rotation == 0, "Reference image has unexpected rotation"
assert tformer.shear == 0, "Reference image has unexpected shearing"
# Check that translations only crop padded image to the reference image's origin
out_shape_rc = ref_slide.slide_dimensions_wh[0][::-1]
sxy = (out_shape_rc/ref_slide.processed_img_shape_rc)[::-1]
scaled_txy = sxy*ref_slide.M[:2, 2]
crop_bbox, _ = ref_slide.get_crop_xywh(ref_slide.crop, out_shape_rc=out_shape_rc)
cropping_to_origin = np.all(np.abs(crop_bbox[0:2] + scaled_txy) < 1)
assert cropping_to_origin, "translations don't move reference to the origin"
# Check for non-rigid transforms
assert np.dstack(ref_slide.bk_dxdy).min() == 0 and np.dstack(ref_slide.bk_dxdy).max() == 0, "Found unexpected non-rigid transforms"
# Check that points are warped correctly too
xy = np.array([[0, 0]])
warped_origin = np.round(ref_slide.warp_xy(xy))
assert np.all(warped_origin == 0), "reference image not warping points correctly"
# Compare raw values
unwarped_ref_img = ref_slide.slide2vips(level=0)
warped_ref_img = ref_slide.warp_slide(level=0)
eq_img = (unwarped_ref_img == warped_ref_img)
min_eq = eq_img.min()
assert min_eq == 255, "warped and original images do not have the same values"
except AssertionError:
print("test failed")
ref_slide.crop = og_crop
reference_img_fname = 'DAPI_1-2C.ome.tiff'
max_non_rigid_registration_dim_px = 2000
align_to_reference = True
registrar = registration.Valis(in_folder, out_folder, reference_img_f=reference_img_fname, align_to_reference=align_to_reference, series=0)
rigid_registrar, non_rigid_registrar, error_df = registrar.register()
registrar.register_micro(max_non_rigid_registration_dim_px=max_non_rigid_registration_dim_px, align_to_reference=align_to_reference)
# Verify that reference has no transformations
ref_slide = registrar.get_ref_slide()
check_for_no_transforms_in_ref(ref_slide, reference_img_fname)
n_channels = sum([slide_obj.reader.metadata.n_channels for slide_obj in registrar.slide_dict.values()])
cmap = slide_tools.perceptually_uniform_channel_colors(n_channels)
#Other built in colormaps:
# cmap = slide_tools.get_matplotlib_channel_colors(n_channels)
# cmap = slide_tools.turbo_channel_colors(n_channels)
ordered_img_list = [registrar.original_img_list[slide_obj.stack_idx] for slide_obj in registrar.slide_dict.values()]
colormap_dict = {registrar.get_slide(ordered_img_list[i]).src_f :[cmap[i]] for i in range(registrar.size)}
# Change channel names by accessing each slide's reader's metadata
for slide_obj in registrar.slide_dict.values():
slide_obj.reader.metadata.channel_names = [slide_obj.name]
print(slide_obj.reader.metadata.channel_names)
# Warp and save the slides with the new channel names and colormaps
registrar.warp_and_save_slides(
dst_dir=out_folder,
level=0,
non_rigid=False,
crop=True,
interp_method="bicubic",
tile_wh=1024,
compression="lzw",
colormap=colormap_dict,
pyramid=True,
Q=100
)
Hi @arundasan91,
I forgot to mention this, but if you'd like to avoid saving DAPI since you won't need it, and just want save the aligned version of MS_1-2C, you can do so using the Slide
object associated with MS_1-2C. In this case, you can set the channel names and colormap using Slide.warp_and_save_slide
:
ref_slide = registrar.get_ref_slide() # Should be DAPI
other_slide = [so for so in registrar.slide_dict.values() if so != ref_slide][0] # Should be MS_1-2C since there are only 2 images
dst_f = os.path.join(out_folder, other_slide.name + "_aligned.ome.tiff")
other_slide.warp_and_save_slide(dst_f=dst_f, channel_names=[other_slide.name], colormap=cmap[0])
Best, -Chandler
Hello,
I am using
Valis '1.0.2'
from the Docker Image to register two single-channelome.tiff
files using the following code. However, I get aKeyError: 'None'
error when I attempt to warp and save the image using the warp_and_save_slide method. Could you please help? I'm happy to share/answer any environment or code-related questions.This is the error: