Fusion crashes for very large images

m-albert / MVRegFus

Python module for registering and fusing multi-view microscopy data

BSD 3-Clause "New" or "Revised" License

23 stars 4 forks source link

Fusion crashes for very large images #3

Open max-brambach opened 4 years ago

max-brambach commented 4 years ago

Hi, When I try to fuse very large images (4 volumes with 4 views each consisting of ~600x2000x2000px; ca. 40GB) I get a Memory Allocation Error. When I use smaller files, it works like a charm. Is there a way to also fuse files that are too large for my memory? Best, Max

m-albert commented 4 years ago

Hey @max-brambach,

since the fusion of the views is performed in a block wise manner, I suspect your memory problem occurs while transforming each view into the target space. So far the processing pipeline assumes that an individual target view fits into memory, but as you show this doesn't have to be the case!

I'm looking into how to perform the transformation step in a more memory efficient way and will come back to it here.

m-albert commented 3 years ago

Experimenting with a simplified workflow in the branch dask_flow. Currently the problem is that too many interdependent input chunks are loaded simultaneously..