ViCCo-Group / thingsvision

Python package for extracting representations from state-of-the-art computer vision models
https://vicco-group.github.io/thingsvision/
MIT License
157 stars 21 forks source link

Could not find a custom model #121

Closed alireza-kr closed 1 year ago

alireza-kr commented 1 year ago

Hello

I created a pull request to include AlexNet_SalObjSub. The file (alexnet_salobjsub.py) is inside the 'custom_models' folder. I am using the following code in Colab to load the custom model:

## load custom model
model_name = 'AlexNet_SalObjSub'

# specify model source 
source = 'custom'
# initialize the extractor
extractor = get_extractor( 
            model_name=model_name,
            pretrained=pretrained,
            model_path=model_path,
            device=device,
            source=source)

However I get the following error when I run it:

ValueError: 
Could not find AlexNet_SalObjSub among custom models.
Choose a different model.

Should I do something else that I haven't done yet?

LukasMut commented 1 year ago

This should work now!

LukasMut commented 1 year ago

@alireza-kr Does everything work for you now? It should AFAIK.

alireza-kr commented 1 year ago

@LukasMut I could download the file successfully from OSF but there is an error:

Missing key(s) in state_dict: "features.0.weight", "features.0.bias", "features.3.weight", "features.3.bias", "features.6.weight", "features.6.bias", "features.8.weight", "features.8.bias", "features.10.weight", "features.10.bias", "classifier.1.weight", "classifier.1.bias", "classifier.4.weight", "classifier.4.bias", "classifier.6.weight", "classifier.6.bias". 
    Unexpected key(s) in state_dict: "fc8_sbt.weight", "conv2.bias", "fc8_sbt.bias", "conv5.bias", "conv2.weight", "conv3.weight", "conv1.weight", "fc6.bias", "fc7.weight", "fc7.bias", "conv3.bias", "conv4.bias", "conv4.weight", "conv1.bias", "conv5.weight", "fc6.weight".

It is because the name that has been used for the layers differs from yours. I will change layer names soon and upload a new file.

alireza-kr commented 1 year ago

I changed the key names and uploaded a new file. Now, I get the size mismatch error:

Error(s) in loading state_dict for AlexNet:
    size mismatch for features.0.weight: copying a param with shape torch.Size([96, 3, 11, 11]) from checkpoint, the shape in current model is torch.Size([64, 3, 11, 11]).
    size mismatch for features.0.bias: copying a param with shape torch.Size([1, 1, 1, 96]) from checkpoint, the shape in current model is torch.Size([64]).
    size mismatch for features.3.weight: copying a param with shape torch.Size([256, 48, 5, 5]) from checkpoint, the shape in current model is torch.Size([192, 64, 5, 5]).
    size mismatch for features.3.bias: copying a param with shape torch.Size([1, 1, 1, 256]) from checkpoint, the shape in current model is torch.Size([192]).
    size mismatch for features.6.weight: copying a param with shape torch.Size([384, 256, 3, 3]) from checkpoint, the shape in current model is torch.Size([384, 192, 3, 3]).
    size mismatch for features.6.bias: copying a param with shape torch.Size([1, 1, 1, 384]) from checkpoint, the shape in current model is torch.Size([384]).
    size mismatch for features.8.weight: copying a param with shape torch.Size([384, 192, 3, 3]) from checkpoint, the shape in current model is torch.Size([256, 384, 3, 3]).
    size mismatch for features.8.bias: copying a param with shape torch.Size([1, 1, 1, 384]) from checkpoint, the shape in current model is torch.Size([256]).
    size mismatch for features.10.weight: copying a param with shape torch.Size([256, 192, 3, 3]) from checkpoint, the shape in current model is torch.Size([256, 256, 3, 3]).
    size mismatch for features.10.bias: copying a param with shape torch.Size([1, 1, 1, 256]) from checkpoint, the shape in current model is torch.Size([256]).
    size mismatch for classifier.1.weight: copying a param with shape torch.Size([1, 1, 4096, 9216]) from checkpoint, the shape in current model is torch.Size([4096, 9216]).
    size mismatch for classifier.1.bias: copying a param with shape torch.Size([1, 1, 1, 4096]) from checkpoint, the shape in current model is torch.Size([4096]).
    size mismatch for classifier.4.weight: copying a param with shape torch.Size([1, 1, 4096, 4096]) from checkpoint, the shape in current model is torch.Size([4096, 4096]).
    size mismatch for classifier.4.bias: copying a param with shape torch.Size([1, 1, 1, 4096]) from checkpoint, the shape in current model is torch.Size([4096]).
    size mismatch for classifier.6.weight: copying a param with shape torch.Size([1, 1, 5, 4096]) from checkpoint, the shape in current model is torch.Size([565, 4096]).
    size mismatch for classifier.6.bias: copying a param with shape torch.Size([1, 1, 1, 5]) from checkpoint, the shape in current model is torch.Size([565]).

Do you have any suggestions to solve this issue?

LukasMut commented 1 year ago

I have to look into this. Could you post this in a new issue? Maybe @andropar has an idea how to solve this. I assume that your version of AlexNet differs from the torchvision version.