alihaydaroglu / suite3d

Fast, accurate, volumetric cell detection. Developed for Light Beads Microscopy, usable for other volumetric 2P. In development
6 stars 0 forks source link

Standard 2p pipeline - initial IO refactor #72

Closed landoskape closed 2 months ago

landoskape commented 4 months ago

In reference to #62, this PR contains:

  1. Initial progress on refactoring the IO of suite3d
    • One central s3dio class that handles all data loading operations.
    • Moving around functions from tiff_utils and file_utils to simplify imports and keep related functions in one place.
    • Integration of new s3dio API into the init_pass and iter_step modules.
    • Note: I tested this on the demo files with the AH007_ROI3p33um_NOSCREEN_-400um_00001_00001 header.

Usage (example from init_pass.py module):

jobio = s3dio(job) # created a lightweight IO object that stores the job's parameters and knows how to load data

# since s3dio objects stores a copy of the job, they have access to job.params and know how to load data by default
init_mov = jobio.load_data(init_tifs) 

# note: if one wants to use different params then the one in ``job`` defined at construction,
# the ``s3dio`` method has a mechanism to update parameters on demand - it would look like this:
init_mov = jobio.load_data(init_tifs, **any_updates_to_overwrite_job_params)

# the s3dio object also knows how to load roi_start_pix directly (I think I also made this a bit more efficient)
xs = jobio.load_roi_start_pix()[1]
  1. Some "developer tools" I started using for marking code as deprecated or needing attention.
    • Two decorators, @deprecated and @deprecated_inputs that allow us to mark methods that are deprecated or that have deprecated inputs.
    • A todo() method that prints todo messages that need attention, and to highlight places in the code where I think multiple eyes would be useful.
    • Note: all three of these tools will only print anything if os.environ["SUITE3D_DEVELOPER", "").lower() == "true", so that non-developers don't need to see messages related to code management. I added ``os.environ["SUITE3D_DEVELOPER"]="True" to the main Demo notebook with a comment explaining it's purpose.

Usage (another example from init_pass.py module which is deprecated due to the use of the new s3dio module):

@deprecated("Using s3dio.load_data instead")
def load_init_tifs(init_tifs, planes, filter_params, n_ch_tif = 30, convert_plane_ids_to_channel_ids=True, fix_fastZ=False, log_cb=default_log, lbm=True, num_colors=None, functional_color_channel=None):
    """ ... code here ... """
  1. Other things:
    • Addition of non-lbm related parameters to default_params.py
    • Smart setting of n_proc_... to make sure it doesn't go above the cores in the machine.
    • Clean up of some unused imports (habit, sorry)
    • Refactor of default_log so it's just defined once
landoskape commented 4 months ago

There's still work to do with handling registration (since we're still hacking our way around LBM related data for the registration part of the pipeline), so this is just a first-pass PR.