search

Luodian / Otter

🦦 Otter, a multi-modal model based on OpenFlamingo (open-sourced version of DeepMind's Flamingo), trained on MIMIC-IT and showcasing improved instruction-following and in-context learning ability.
https://otter-ntu.github.io/
MIT License
3.54k stars 242 forks source link

IdeficsForVisionText2Text does not exist #247

Closed zuwenqiang closed 1 year ago

zuwenqiang commented 1 year ago

Hi to the team! When training with SD_Instruction.json and SD.json, I encountered the following prompt: "IdeficsForVisionText2Text does not exist." I would like to kindly inquire about the potential impact of the absence of IdeficsForVisionText2Text on the training results. Additionally, I am eager to know how I can obtain IdeficsForVisionText2Text. Please note that my current environment aligns with the recommended versions: transformers=4.29.0; torch=2.0.1; accelerate=0.19.0. Your assistance and guidance would be greatly appreciated.

Luodian commented 1 year ago

Hi you can just comment out and ignore relevant lines of code about IDefics.

zuwenqiang commented 1 year ago

Yes, this solution indeed enables the code to run smoothly. However, I have noticed that IdeficsForVisionText2Text or transformer.model.idefics is required in many places, including other branches. It seems that these packages are essential.

Luodian commented 1 year ago

Sorry for make it misleading, it's not essential. It's a internal developed model code and we will remove this dependency in our next PR.

zuwenqiang commented 1 year ago

that's ok, thank you kindly for your response.

Luodian commented 1 year ago

never mind!