search

Luodian / Otter

🦦 Otter, a multi-modal model based on OpenFlamingo (open-sourced version of DeepMind's Flamingo), trained on MIMIC-IT and showcasing improved instruction-following and in-context learning ability.
https://otter-ntu.github.io/
MIT License
3.56k stars 241 forks source link

[Feat/Model] add lora training to LLM part. #208

Closed ZhangYuanhan-AI closed 1 year ago

ZhangYuanhan-AI commented 1 year ago
  1. Change the data transform in the mimicti_dataset.py
  2. Change the dataset/dataloader from single-dataloader2single-dateset to single-dataloader2multiple-datesets. One dataloader process multiple datasets within the same task type:image-level in-context, general qa, etc.
Luodian commented 1 year ago

Please pull the recent commit and see if all things correct. I manually merged some conflicts.

@ZhangYuanhan-AI Please take a look at if they are all correct.