Using Resnet backbone

[afvca]

Hi!

I was trying to run with a Resnet50 backbone with this prompt:

!python train.py --dataset visa --train_data_path /content/visa-dataset/ \
--save_path ./exps/mvtec/RN50x16_384 --config_path ./open_clip/model_configs/RN50x16.json --model RN50x16 \
--features_list 1 2 3 4 --pretrained openai --image_size 384  --batch_size 8 --aug_rate -1 --print_freq 1 \
--epoch 3 --save_freq 1

But it does not work correctly:

Traceback (most recent call last):
  File "/code/VAND-APRIL-GAN/train.py", line 170, in <module>
    train(args)
  File "/code/VAND-APRIL-GAN/train.py", line 108, in train
    image_features, patch_tokens = model.encode_image(image, features_list)
  File "/code/VAND-APRIL-GAN/open_clip/model.py", line 213, in encode_image
    features = self.visual(image, out_layers)
  File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1130, in _call_impl
    return forward_call(*input, **kwargs)
TypeError: ModifiedResNet.forward() takes 2 positional arguments but 3 were given

Could you share the changes that should be made in modified_resnet.py to allow this? Thank you.

[ByChelsea]

Hello, I'll update the code as soon as possible to support ResNet50. I will reply to you immediately once the update is completed. :)

[ByChelsea]

Hi, the code has been updated. I highly encourage you to use the new version instead of just replacing modified_resnet.py. You can run with a Resnet50 backbone with this command:

python train.py --dataset mvtec --train_data_path ./data/mvtec \ 
--save_path ./exps/visa/RN50x16 --config_path ./open_clip/model_configs/RN50x16.json --model RN50x16 \ 
--features_list 1 2 3 --pretrained openai --image_size 384  --batch_size 8 --aug_rate 0.2 --print_freq 1 \ 
--epoch 5 --save_freq 1 --learning_rate 0.0001

Note that the parameter features_list here represents the output features of the block to use, whereas in ViT, this parameter refers to the output features of which layer to use.

[afvca]

Hi! Thank you for your fast response!

Training is now working, but when I run the test, I got the following error:

Traceback (most recent call last):
  File "/code/VAND-APRIL-GAN/test.py", line 308, in <module>
    test(args)
  File "/code/VAND-APRIL-GAN/test.py", line 183, in test
    cos = pairwise.cosine_similarity(normal_features_ls[cls_name[0]][layer].cpu(),
  File "/usr/local/lib/python3.10/dist-packages/sklearn/metrics/pairwise.py", line 1251, in cosine_similarity
    X, Y = check_pairwise_arrays(X, Y)
  File "/usr/local/lib/python3.10/dist-packages/sklearn/metrics/pairwise.py", line 156, in check_pairwise_arrays
    X = check_array(
  File "/usr/local/lib/python3.10/dist-packages/sklearn/utils/validation.py", line 794, in check_array
    raise ValueError(
ValueError: Found array with dim 3. check_pairwise_arrays expected <= 2.

[ByChelsea]

The bug has been fixed. I just forgot to modify the few-shot part. Thank you for pointing it out!

[afvca]

Thank you so much for your (fast) help! Everything is working as expected now :)