MinkowskiNormalization

[tkdtks123]

Thanks for the great code.

I have questions on the Minkowski sparse tensor normalization.

I see that the MinkowskiBatchrnom directly uses pytorch batchnorm 1D, while MinkowskiInstancenorm does not and is implemented from scratch. What is the reason for this? Is there any reason that we cannot use pytorch instancenorm 1D here?

Pluse, for the layernorm, can we directly use pytorch layernorm (or should we re-implement it)?

[lizuoyue]

We cannot use PyTorch InstanceNorm / GroupNorm / LayerNorm directly because the feature map of a batched sparse tensor cannot represent each instance in the batch, as it is just a concatenation of the feature map of each instance.

In my opinion, a possible way would be to unbatch the sparse tensor first and then perform PyTorch normalization, followed by rebatching them, though it is not that elegant.

The reason why BatchNorm can use the PyTorch utility is that the operation is on the entire batch.