search

Luodian / Otter

🦦 Otter, a multi-modal model based on OpenFlamingo (open-sourced version of DeepMind's Flamingo), trained on MIMIC-IT and showcasing improved instruction-following and in-context learning ability.
https://otter-ntu.github.io/
MIT License
3.52k stars 241 forks source link

Where can I find the checkpoint of OtterHD? #331

Closed Labmem009 closed 5 months ago

Labmem009 commented 6 months ago

Thank you for your ongoing updates and improvements to this project. In your Hugging Face repository, I can find the three most recently updated models: OTTER-Image-MPT7B OTTER-Image-LLaMA7B-LA-InContext Flamingo-Llama2-Chat7B-CC3M Which one is OtterHD? Actually, I would like to know which model with multilingual capabilities performs the best in your repository? Because I believe this is a unique advantage of Otter. BTW, OtterHD-8B-x360 is still under preparation? Thanks a lot!

Luodian commented 6 months ago

Yes, as suggested, the OtterHD-8B-x360 is ongoing since we changed the resize and padding strategy for OtterHD, also the data loading pipeline enabling training with more and more data.

As for multilingual ability, I am not sure since we didnt benchmarked them with other languages yet. My observation is that OTTER-Image-MPT would understand Chinese questions but answer with English.

Labmem009 commented 5 months ago

Thank you very much for the information. But I learnt from paper that MIMIC-IT is a multi-language dataset, so I think model trained with MIMIC-IT will abtain multilingual ability, like otter? Or only a few versions of otter are trained with multi-language dataset MIMIC-IT? Could you please tell me the versions so I can have a try.