1d version of evo_norms

[pinouchon]

I've looked at the code for evonorm layers, it seems that you expect a 4d (n,c,h,w) shape. I need evonorm with 1d convolutions, for this it seems the only way is to rewrite the evo_norm function to handle the different shape. Do you plan to provide evonorm1d version for tensors of 3d shape? Thanks

[PistonY]

I’ll check it out later.

[PistonY]

Hi @pinouchon ,I read the paper again, and it didn't say that evo_norm could work with 1d conv layer, so I may not provid 1d version of this. But I could tell you how to change it to suit for this. You only need to re-write instance_std and group_std like this.

def instance_std_1d(x, eps=1e-5):
    var = torch.var(x, dim=(2, ), keepdim=True)
    std = torch.sqrt(var + eps)
    return std

def group_std_1d(x: torch.Tensor, groups=32, eps=1e-5):
    n, c, l = x.size()
    x = torch.reshape(x, (n, groups, c // groups, l))
    var = torch.var(x, dim=(2, 3), keepdim=True)
    std = torch.sqrt(var + eps)
    return torch.reshape(std, (n, c, h, w))

And the training var should be

var = torch.var(x, dim=(0, 2), keepdim=True)

So if you test evo_norms could work with 1d conv, you could make a pr to me. Good luck.