search

MathOnco / valis

Virtual Alignment of pathoLogy Image Series
https://valis.readthedocs.io/en/latest/
MIT License
108 stars 26 forks source link

Problems with tiled non-rigid registration #118

Open batukav opened 3 months ago

batukav commented 3 months ago

Hello @cdgatenbee ,

Once again, I'd like to start by thanking you for this nice repository.

I am having some issues with the tiled registration. My images are normally large (> 20K x 20K px) and I generally use the tiled non-rigid registration. Recently I suspect that the tiled registration is actually not working as intended: for the same pair of images, when I use tiled registration I get 0 px displacements, whereas without the tiled registration I get non-zero px displacements.

I wanted to create a minimal example demonstrating the issue by registering the test images from this repository. However, my code

from valis import registration
import numpy as np
slide_src_dir = "/home/bkav/valis_tile_reg_test/input_images"
results_dst_dir = "/home/bkav/valis_tile_reg_test/tile"
registered_slide_dst_dir = "/home/bkav/valis_tile_reg_test/no_tile"
reference_slide = "/home/bkav/valis_tile_reg_test/input_images/CD4_CD68_CD3.ome.tiff"

# Create a Valis object and use it to register the slides in slide_src_dir, aligning *towards* the reference slide.
rigid_transform_dict = {'/home/bkav/valis_tile_reg_test/input_images/CD8_FOXP3_CD20.ome.tiff':{'M': np.eye(3)}}
registrar = registration.Valis(slide_src_dir, results_dst_dir, reference_img_f=reference_slide, max_non_rigid_registration_dim_px = 2500, do_rigid = rigid_transform_dict, align_to_reference=True, img_list = ["/home/bkav/valis_tile_reg_test/input_images/CD4_CD68_CD3.ome.tiff","/home/bkav/valis_tile_reg_test/input_images/CD8_FOXP3_CD20.ome.tiff"])
rigid_registrar, non_rigid_registrar, error_df = registrar.register()

returns the below error

======== Registering tiles

 50%|█████████████████████████████████████████████████████████████████████████████████████                                                                                     | 1/2 [00:00<00:00, 60.79it/s]
/home/bkav/anaconda3/envs/valis/lib/python3.10/site-packages/valis/valtils.py:21: UserWarning: expected str, bytes or os.PathLike object, not NoneType                               | 0/154 [00:00<?, ?it/s]
  warnings.warn(warning_msg, warning_type)
joblib.externals.loky.process_executor._RemoteTraceback: 
"""
Traceback (most recent call last):
  File "/home/bkav/anaconda3/envs/valis/lib/python3.10/site-packages/valis/non_rigid_registrars.py", line 1273, in process_tile
    processed_img = processor.process_image(**self.processing_kwargs)
  File "/home/bkav/anaconda3/envs/valis/lib/python3.10/site-packages/valis/preprocessing.py", line 108, in process_image
    reader_cls = slide_io.get_slide_reader(self.src_f, series=self.series)
  File "/home/bkav/anaconda3/envs/valis/lib/python3.10/site-packages/valis/slide_io.py", line 2301, in get_slide_reader
    f_extension = slide_tools.get_slide_extension(src_f)
  File "/home/bkav/anaconda3/envs/valis/lib/python3.10/site-packages/valis/slide_tools.py", line 127, in get_slide_extension
    f = os.path.split(src_f)[1]
  File "/home/bkav/anaconda3/envs/valis/lib/python3.10/posixpath.py", line 103, in split
    p = os.fspath(p)
TypeError: expected str, bytes or os.PathLike object, not NoneType

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/bkav/anaconda3/envs/valis/lib/python3.10/site-packages/joblib/_parallel_backends.py", line 273, in _wrap_func_call
    return func()
  File "/home/bkav/anaconda3/envs/valis/lib/python3.10/site-packages/joblib/parallel.py", line 589, in __call__
    return [func(*args, **kwargs)
  File "/home/bkav/anaconda3/envs/valis/lib/python3.10/site-packages/joblib/parallel.py", line 589, in <listcomp>
    return [func(*args, **kwargs)
  File "/home/bkav/anaconda3/envs/valis/lib/python3.10/site-packages/valis/non_rigid_registrars.py", line 1324, in reg_tile
    fixed_processed = self.process_tile(np_fixed)
  File "/home/bkav/anaconda3/envs/valis/lib/python3.10/site-packages/valis/non_rigid_registrars.py", line 1276, in process_tile
    processed_img = processor.process_image()
  File "/home/bkav/anaconda3/envs/valis/lib/python3.10/site-packages/valis/preprocessing.py", line 108, in process_image
    reader_cls = slide_io.get_slide_reader(self.src_f, series=self.series)
  File "/home/bkav/anaconda3/envs/valis/lib/python3.10/site-packages/valis/slide_io.py", line 2301, in get_slide_reader
    f_extension = slide_tools.get_slide_extension(src_f)
  File "/home/bkav/anaconda3/envs/valis/lib/python3.10/site-packages/valis/slide_tools.py", line 127, in get_slide_extension
    f = os.path.split(src_f)[1]
  File "/home/bkav/anaconda3/envs/valis/lib/python3.10/posixpath.py", line 103, in split
    p = os.fspath(p)
TypeError: expected str, bytes or os.PathLike object, not NoneType
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/bkav/anaconda3/envs/valis/lib/python3.10/site-packages/valis/registration.py", line 4397, in register
    non_rigid_registrar = self.non_rigid_register(
  File "/home/bkav/anaconda3/envs/valis/lib/python3.10/site-packages/valis/registration.py", line 3867, in non_rigid_register
    non_rigid_registrar = serial_non_rigid.register_images(
  File "/home/bkav/anaconda3/envs/valis/lib/python3.10/site-packages/valis/serial_non_rigid.py", line 1226, in register_images
    nr_reg.register(non_rigid_reg_class, non_rigid_reg_params, img_params=img_params)
  File "/home/bkav/anaconda3/envs/valis/lib/python3.10/site-packages/valis/serial_non_rigid.py", line 1028, in register
    self.register_to_ref(
  File "/home/bkav/anaconda3/envs/valis/lib/python3.10/site-packages/valis/serial_non_rigid.py", line 955, in register_to_ref
    moving_obj.calc_deformation(
  File "/home/bkav/anaconda3/envs/valis/lib/python3.10/site-packages/valis/serial_non_rigid.py", line 477, in calc_deformation
    warped_moving, moving_grid_img, moving_bk_dxdy = non_rigid_reg.register(
  File "/home/bkav/anaconda3/envs/valis/lib/python3.10/site-packages/valis/non_rigid_registrars.py", line 1468, in register
    bk_dxdy, fwd_dxdy = self.calc()
  File "/home/bkav/anaconda3/envs/valis/lib/python3.10/site-packages/valis/non_rigid_registrars.py", line 1366, in calc
    Parallel()(delayed(self.reg_tile)(i, lock) for i in range(self.n_tiles))
  File "/home/bkav/anaconda3/envs/valis/lib/python3.10/site-packages/joblib/parallel.py", line 1952, in __call__
    return output if self.return_generator else list(output)
  File "/home/bkav/anaconda3/envs/valis/lib/python3.10/site-packages/joblib/parallel.py", line 1595, in _get_outputs
    yield from self._retrieve()
  File "/home/bkav/anaconda3/envs/valis/lib/python3.10/site-packages/joblib/parallel.py", line 1699, in _retrieve
    self._raise_error_fast()
  File "/home/bkav/anaconda3/envs/valis/lib/python3.10/site-packages/joblib/parallel.py", line 1734, in _raise_error_fast
    error_job.get_result(self.timeout)
  File "/home/bkav/anaconda3/envs/valis/lib/python3.10/site-packages/joblib/parallel.py", line 736, in get_result
    return self._return_or_raise()
  File "/home/bkav/anaconda3/envs/valis/lib/python3.10/site-packages/joblib/parallel.py", line 754, in _return_or_raise
    raise self._result
TypeError: expected str, bytes or os.PathLike object, not NoneType

The same error does not happen if I don't use the tiled registration. I am using the Valis version 1.0.0rc15. Could you please guide me solving this issue?

cdgatenbee commented 3 months ago

Hi @batukav, Thanks for bringing this to my attention. It's a little difficult to tell what exactly is going on based on the error message (thanks for sharing), so I'll see if I can recreate it using the example cycif data. But, does it work for you when you don't provide rigid_transform_dict?

Best, -Chandler

batukav commented 3 months ago

Hi @cdgatenbee , thank you for the quick response.

I removed the rigid_transform_dict and the code failed with the same error message.

cdgatenbee commented 1 month ago

Hi @batukav, I've made quite a few updates to the latest version of valis (1.1.0), which hopefully have addressed this issue. Could you please try this version and see if it resolves this issue?

Best, -Chandler