issues
search
Luodian
/
Otter
🦦 Otter, a multi-modal model based on OpenFlamingo (open-sourced version of DeepMind's Flamingo), trained on MIMIC-IT and showcasing improved instruction-following and in-context learning ability.
https://otter-ntu.github.io/
MIT License
3.55k
stars
242
forks
source link
[Feat/Model/Train] updates for Flamingo's lightweight pre-training.
#222
Closed
Luodian
closed
1 year ago
Luodian
commented
1 year ago
DeepSpeed ZeRo2 Integration + DDP Training
5-6x speed improvement in pretraining/instruction-tuning compared to our previous FSDP pipeline.
Support Flamingo pretraining on LIAON400M/CC3M.
Now you can train your lightweight flamingo model on smaller datasets.
Add LoRA support for tuning LLM decoder.
Yes, we have LoRA too. You can customize your model better in specific scenarios.
Support cross usage for Otter and Flamingo models.
Yes, you could load Flamingo weights directly to Otter models, and vice versa.