search

ChenDelong1999 / RemoteCLIP

🛰️ Official repository of paper "RemoteCLIP: A Vision Language Foundation Model for Remote Sensing" (IEEE TGRS)
https://arxiv.org/abs/2306.11029
Apache License 2.0
228 stars 13 forks source link

Pretrained model loading #10

Closed BinSpa closed 8 months ago

BinSpa commented 8 months ago

When I tried to load the pre-trained model provided by the author using the ITRA method, I found that it can only be loaded through the '--resume' parameter for existing checkpoints. Moreover, the author's model could not be loaded successfully due to inconsistencies in some layers. I believe that in the 'openclip' loading method within model.py, code to load the model should be added.

ChenDelong1999 commented 8 months ago

Hi, thanks for your interest in running RemoteCLIP on the ITRA codebase. Our model was trained with ITRA, but in order to make our pretrained model accessed more conveniently, we have converted the checkpoints into the OpenCLIP compatible format.

It is possible to load RemoteCLIP through ITRA by modifying the code in model.py a bit. Please specify args.text_model_builder=='openclip' and args.image_model_builder=='openclip' and add codes to load RemoteCLIP checkpoints below this line and this line.