search

princetonvisualai / multimodal_dataset_distillation

40 stars 2 forks source link

Low-Rank Adaptation Matching #3

Open yfpeng1234 opened 4 months ago

yfpeng1234 commented 4 months ago

Dear Author, I noted that the new version of your paper in arxiv has "Low-Rank Adaptation Matching". I'm interested in how to implement it but I can't find the corresponding code in distill.py or buffer.py. Could you please share the code of Low-Rank Adaptation Matching.

yfpeng1234 commented 4 months ago

I also have another question. I noted that figure2 in your paper also has a projection head for vision encoder. But in distill.py, it seems that only the vision encoder was considered when matching training trajectory. What if I want to keep the vision encoder frozon and matching the training trajectory of vision projection head.

Could you please give me some guidance on how to deal with it when you are available? Thanks so much!

yfpeng1234 commented 4 months ago

Actually I try to use vit-base-patch16-224 as the visual encoder, but I encountered the oom error using a single A6000 GPU. So I hope to freeze the encoder and just train the projection head.