Add backends and image visualization for transform `TileOnGrid`

[Nic-Ma]

Is your feature request related to a problem? Please describe. Thanks for @myron 's contribution for the new transform TileOnGrid in PR: https://github.com/Project-MONAI/MONAI/pull/3237. I will try to add the visualization image and backend related logic when the PR merged.

Thanks.

[Nic-Ma]

Hi @rijobro @wyli ,

I have a quick question that: currently, all the transforms under monai.apps.deepgrow and monai.apps.pathology don't have backend, inverse and example images, I am not sure should we try to enhance them or just leave it for now? I feel at least we can add NUMPY backend related logic to them to make sure they can work with other MONAI transforms. What do you guys think?

Thanks.

[wyli]

I feel this is not a requirement for v0.8, perhaps we can postpone it.

[Nic-Ma]

Yes, sounds good to me. And @drbeh and @SachidanandAlle could you please help on Pathology and Deepgrow parts as you are experts on that? I can try to add the logic to Pathology transforms as a reference after v0.8 release.

Thanks in advance.

[drbeh]

I think having numpy backend and inverse is not necessary for pathology transforms. I don't see any added value. On the other hand, one of pathology transforms does not have torch backend which might be good to be added only if we want to go for a uniform torch Tensor backend.

[SachidanandAlle]

@diazandr3s @finalelement
can you check if we can support invert on deepgrow/deepedit transforms..

[diazandr3s]

Thanks, @SachidanandAlle for pinging me.

To be honest, I don't see the added value of having the inverse of the DeepGrow/DeepEdit transforms. Most of them only add info to the dictionary (i.e. slices IDs (sids), guidance), they don't modify the label or the image. There is only one that adds channels to the input image: AddGuidanceSignald But again, not sure if this has an application.

Please let me know whether I'm missing or misunderstanding something :)

[Nic-Ma]

Thanks for the discussion, I see your points. @drbeh , do you think necessary to add NUMPY backend to the Patholopy transforms? Then we can combine these transforms with other MONAI transforms even other transforms are handling Tensor data. If yes, I can try to add the NUMPY backend ASAP following the same pattern of other transforms.

Thanks in advance.

[drbeh]

Thanks @Nic-Ma! Right now, there is only one pathology transform that does not support numpy (SplitOnGrid). I don't think that it is necessary to add it but if you feel strongly about it, I can add the numpy backend. However, it cannot be directly translated to numpy since unfold does not exist in numpy.

[victorvicpal]

Hello, sorry for jumping into this closed issue.

First of all, thank you very much for all the work done and more specifically in pathology. It is helping me a lot in my work.

I'm implementing multiple instance learning and I am following the tutorial . Everything has worked perfectly, but I would like to perform heat maps on images. Is there any way to revert TileOnGrid to reorder the patches and visualize the whole image again?

Something similar to the related paper

[Nic-Ma]

Hi @drbeh @myron ,

Could you please help take a look at this ticket? Should we add invert logic for the transform?

Thanks in advance.

[drbeh]

Hi @victorvicpal, thanks for expressing your interest in MONAI Pathology. That definitely is a useful feature and we have had related request before, like https://github.com/Project-MONAI/MONAI/issues/4090. We are having it on our development plan, and will update you once we have this feature.

[drbeh]

Hi @drbeh @myron ,

Could you please help take a look at this ticket? Should we add invert logic for the transform?

Thanks in advance.

Hi @Nic-Ma, that's right and invert seems reasonable but let's see if we need a transform that can do this regardless of how patches has been generated.