[Feat/Model] Simply add LoRA module to MPT1B

Luodian / Otter

🦦 Otter, a multi-modal model based on OpenFlamingo (open-sourced version of DeepMind's Flamingo), trained on MIMIC-IT and showcasing improved instruction-following and in-context learning ability.

https://otter-ntu.github.io/

MIT License

3.55k stars 242 forks source link

[Feat/Model] Simply add LoRA module to MPT1B #212

Closed Luodian closed 1 year ago

Luodian commented 1 year ago

Need to run otter/converting_otter_to_lora.py to convert a normal version OTTER to OTTER-LoRA first. This is for loading previous weights correctly into an OTTER-LoRA, and then start finetune the model.