Pre-trained weights incompatible with backbone

[LucFrachon]

The weights for ConvNeXt-V2-Base, pretrained on INet1k, provided here, have several incompatibilities with the encoder architecture:

• The MinkowskiLayerNorm layers have aself.ln element that means that the checkpoint keys should look like downsample_layers.1.0.ln.bias, whereas the weights provided have downsample_layers.1.0.bias.
• Similarly, all the layers named pwconv<i> have self.linear, which the checkpoints doesn't have.
• The MinkowskiConvolution layers have a parameter kernel (e.g., downsample_layers.1.1.kernel), whereas the provided checkpoint has keys "downsample_layers.1.1.weight.
• The MinkowskiConvolution biases have shape (1, c) (e.g., (1, 256)) in the model but shape (c,) in the checkpoint, and weights have shapes (p**2, in, out) in the model but (out, in, p, p) in the checkpoint
• Same problem with the DW convolution weights and biases in stages: biases need to be (1, c) but are (c,), and weights need to be (p ** 2, c) but are (c, -1, p, p)

These can all be fixed with some code, but it would make life easier for everyone if you could upload the correct weights. Thanks a lot!

[blackpearl1022]

@LucFrachon Same idea, I have. Do you have any updates on your side ? Thanks !

[LucFrachon]

I wrote code to update the state dict and it worked. I haven't checked if the provided checkpoints have been updated. I've moved on to other things now...