search

ml-jku / MIM-Refiner

A Contrastive Learning Boost from Intermediate Pre-Trained Representations
MIT License
36 stars 3 forks source link

model initializer code #7

Closed haribaskarsony closed 3 months ago

haribaskarsony commented 3 months ago 
          Hi @BenediktAlkin thank you for the reply.

I encountered a different issue while loading my model for refining. It seems that i need to modify the model backbone structure to load my model. Could you pls point me to the files where i can do this?

Reg_token is an additional layer just like cls_token. I need to explicitly mention the dimension of reg_token layer to initialize and load my model weights.

Originally posted by @haribaskarsony in https://github.com/ml-jku/MIM-Refiner/issues/6#issuecomment-2259735146

BenediktAlkin commented 3 months ago

answered in original issue