🦦 Otter, a multi-modal model based on OpenFlamingo (open-sourced version of DeepMind's Flamingo), trained on MIMIC-IT and showcasing improved instruction-following and in-context learning ability.
Change the data transform in the mimicti_dataset.py
Change the dataset/dataloader from single-dataloader2single-dateset to single-dataloader2multiple-datesets. One dataloader process multiple datasets within the same task type:image-level in-context, general qa, etc.