search

czbiohub-sf / shrimPy

shrimPy: Smart High-throughput Robust Imaging & Measurement in Python
BSD 3-Clause "New" or "Revised" License
8 stars 1 forks source link

Change default for deskew to YX and create_empty_zarr dtype #97

Closed edyoshikun closed 1 year ago

edyoshikun commented 1 year ago

This PR modifies the default dtype of utils.create_empty_zarr from the input dataset dtype to np.float32.

It also changes the default output chunk size of utils.create_empty_zarr to XYZ stacks limited to 500 MBs, unless explicitly provided.

ieivanov commented 1 year ago

Let's use the image size and the data type to calculate the chunk size in MB. Then we can limit chunks to 500 MB, splitting as necessary. We should apply this to all pipelines, not just deskew.

ieivanov commented 1 year ago

Repost: Let’s use logic like this for deciding the chunk size:

chunk_size = [1, 1, sizeZ, sizeY, sizeX]
while np.prod(chunk_size) * bytes_per_pixel > 500e6:
    chunk_size[-3] = chunk_size[-3] // 2
edyoshikun commented 1 year ago

This is now ready for review @ieivanov

ieivanov commented 1 year ago

@talonchandler could you also take a look at the changes in this PR?

The CLI tests fail because the iohub Position object does not have a scale attribute. Is this related to upgrading to iohub dev5? I see that the example_plate fixture does not set a scale explicitly.

ieivanov commented 1 year ago

@edyoshikun can you also take another look?