Open idankinderman opened 5 months ago
Hi,
Thanks for your interest in our work!
There are three tasks you need to do to add support for a new architecture:
In general, I suggest you to step through the experiment we provide in debug mode, to get a good idea of how our fusion pipeline works in detail before starting to add support for a new architecture.
– Moritz
Hi,
Let's say I want to apply your fusion method to a PyTorch-based ViT implemented in a package that isn't Hugging Face. Specifically, I want to merge ViTs from the open_clip package (https://github.com/mlfoundations/open_clip). What changes do I need to make in your code to make it work?
Thank you for your great work!
Edan