itk-dreg.reduce_dfield currently implements a method to reduce multiple in-memory bounded transforms to an in-memory output displacement field. Depending on the input transform types and output grid subdivision, the output displacement field could grow prohibitively large.
Feature Request
Explore methods to reduce memory requirements for transform reduction.
To reduce memory requirements for the output displacement field, consider streaming image subregions to cloud storage with ITKIOOMEZarrNGFF. If all input transforms are available in memory on a single worker, the worker could sample successive subregions of the output displacement field image and then offload those subregions to the cloud before continuing with the next subregion. Likely requires advance knowledge of the output displacement field size for cloud allocation with OME-Zarr / tensorstore. Could be implemented by extending ReduceResultsMethod for use in existing register_images scheduling infrastructure.
To reduce memory requirements for the input transforms, we could similarly map/reduce reduction tasks such that a worker is only responsible for sampling a subregion involving the transforms from a fixed image block and its overlapping neighbors, rather than all possible transform inputs. Each worker may then be responsible for streaming a displacement field subimage to the cloud as in (1) above. May require extending the register_images scheduling infrastructure with another scheduling method to map transform input processing among workers.
Background
itk-dreg.reduce_dfieldcurrently implements a method to reduce multiple in-memory bounded transforms to an in-memory output displacement field. Depending on the input transform types and output grid subdivision, the output displacement field could grow prohibitively large.Feature Request
Explore methods to reduce memory requirements for transform reduction.
ITKIOOMEZarrNGFF. If all input transforms are available in memory on a single worker, the worker could sample successive subregions of the output displacement field image and then offload those subregions to the cloud before continuing with the next subregion. Likely requires advance knowledge of the output displacement field size for cloud allocation with OME-Zarr / tensorstore. Could be implemented by extendingReduceResultsMethodfor use in existingregister_imagesscheduling infrastructure.register_imagesscheduling infrastructure with another scheduling method to map transform input processing among workers.