Visual Prompting model not correctly built

[theamaya]

Running the pretraining on shapenet55 results in the following error

Traceback (most recent call last):
  File "/home/amaya/repos/CLIP-goes-3D/main.py", line 88, in <module>
    main()
  File "/home/amaya/repos/CLIP-goes-3D/main.py", line 79, in main
    pretrain_CG3D_VPT(args, config, train_writer, val_writer)
  File "/home/amaya/repos/CLIP-goes-3D/tools/runner_CG3D_VPT.py", line 170, in run_net
    optimizer_clip = builder.build_VPT_optimizer(params, config)
  File "/home/amaya/repos/CLIP-goes-3D/tools/builder.py", line 231, in build_VPT_optimizer
    optimizer = optim.SGD(
  File "/home/amaya/miniconda3/envs/cg3d/lib/python3.9/site-packages/torch/optim/sgd.py", line 101, in __init__
    super(SGD, self).__init__(params, defaults)
  File "/home/amaya/miniconda3/envs/cg3d/lib/python3.9/site-packages/torch/optim/optimizer.py", line 49, in __init__
    raise ValueError("optimizer got an empty parameter list")
ValueError: optimizer got an empty parameter list

In the modified timm.models.vision_transformer.py,

SLIP_VITB16

model is built with

'vit_base_patch16_224'

from modified timm with visual_prompting = True, but I see that function

_create_vision_transformer

in modified timm.models.vision_transformer.py has no functionality of adding visual prompts. I downloaded the modified timm files linked in the repository. What am I missing?

[deeptibhegde]

I have corrected the error in the model file, please re-download and let me know if you face any other issue.

[theamaya]

Hi, the file linked at https://drive.google.com/file/d/10ETHew_P3vGUA2HbvSZ9wxf3EmtoFmQ3/view appears to be unavailable.

[deeptibhegde]

Link is now fixed.