search

mlfoundations / task_vectors

Editing Models with Task Arithmetic
433 stars 36 forks source link

Can't get attribute 'VisualTransformer' on <module 'open_clip.model' from '~/open_clip/model.py'> #7

Closed zhuoyan-xu closed 1 year ago

zhuoyan-xu commented 1 year ago

Hi @gabrielilharco , Thank you for your exciting work! I tried to replicate the result using code from README.md. It showed an error when running 

# Create the task vector
task_vector = TaskVector(pretrained_checkpoint, finetuned_checkpoint)

The error is: AttributeError: Can't get attribute 'VisualTransformer' on <module 'open_clip.model' from '/srv/home/<user_name>/anaconda3/envs/task-vectors/lib/python3.10/site-packages/open_clip/model.py'>

The error is from loading the model with trained weights pretrained_state_dict = torch.load(pretrained_checkpoint).state_dict().

Could you help me with this? Thanks!

gabrielilharco commented 1 year ago

hey @OliverXUZY, this might be because of an inconsistency between the versions of openclip used to create the checkpoint and the one you are trying to load it with. Openclip had a refactor at some point that change some naming conventions (see https://github.com/mlfoundations/open_clip/commit/2c3d86eedd3e4a3caa418a759db2bf9110fad070#diff-6ab3a5f5da1d07042589cbaeae3ff6e10b05897bf1684f2f9b2b688ae7ecee16L172)

zhuoyan-xu commented 1 year ago

Hi @gabrielilharco, Thank you for the information and quick response! Do you have any suggestions for solving this issue? Is there a certain open_clip version I can download?

Thank you for your time!

gabrielilharco commented 1 year ago

Try pip install open-clip-torch==2.0.2

zhuoyan-xu commented 1 year ago

Thank you for your help!