halo should only be allowed for output axes with a size reference

[FynnBe]

we define halo on output axes, denoting the border region that may suffer from artifacts. When tiling an input image and cropping the halo from the output it is essential to know how the output halo maps to the input tile stride (overlap). This is only possible if the output axis with halo has a SizeReference size to an input axis.

cc @k-dominik @buglakova @josh-talks

[k-dominik]

Not sure I get what the overlap would do here... All tiles should have the same halo to crop (or mask out for that matter). If you want to do any smoothing/blending (hence the lower than tile size stride) you have to do it yourself?

Halo: possibly invalid/erroneous region, should be removed (masked out, cropped). Overlap: all valid, how to deal with stride smaller than tile, up to creativity of the user.

[FynnBe]

I really mean "stride" here. (I added "(overlap)" as an output tensor's halo that is cropped can only be stitched without gaps if input tiles overlap. (but I'm not considering overlapping inner output blocks here))

[Buglakova]

Hello. I'm still not sure what's the question but from my experience with different libraries I'd expect the following without checking the documentation:

• tile shape: the size of the image that will be run through the network. Doesn't matter if it's all data or padded or filled in with some other values
• overlap: value between 0 and 1 (or a vector if different overlap for different axes) meaning the percentage of the tile shape that will be predicted twice
• stride: value in pixels meaning distance between centers of consecutive tiles
• halo: value in pixels meaning part of the prediction within tile that will be zeroed out to remove edge artifacts

So overlap and stride are somewhat interchangeable while tile shape/stride/halo can be independent

Now I looked up and this is how Constantine explained what happens in this library:

ttttttttttttttttttttttttttttttttttttttttttttt hhhh-------------------------------------hhhh

tttttttttttttt ----------hhhh ssssssssssss hhhh----------

So the overlap corresponds to 2 * halo, and the part of the halo that is closer to the tile is left.

So currently there's no averaging of predictions. Is that what you want to change? Here is the sliding window prediction in Monai that I mentioned: https://docs.monai.io/en/stable/inferers.html#sliding-window-inference-function It's a bit of a mess but it is an example of a much more flexible implementation

[FynnBe]

Thanks @Buglakova for this overview! Adding tile averaging (and overlapping tiles) is left for the future (new issue in core: https://github.com/bioimage-io/core-bioimage-io-python/issues/373)