Models

[rbroc]

Looking both at foundation and instruction tuning models. For this project, the latter is probably going to be the only target, as it would probably work better.

Available

• Flan-T5: https://huggingface.co/google/flan-t5-xxl (on HuggingFace)
• Falcon: https://huggingface.co/tiiuae/falcon-40b (both base model and instruction tuned). Note that there is also a 7b instruction-tuned and a 7b version (on HuggingFace)
• Llama2 (probably just for comparison) and StableBeluga (https://huggingface.co/stabilityai/StableBeluga2)
• Alpaca (library, for both foundation model and instruction-tuned model)

Maybe for later Not open-source

• GPT-4 (pricing 0.03$ / 1k tokens for prompts; 0.06 $ / 1k tokens completions) - (on hold, because instruction tuning version is not available)
• PaLM - (on hold)
• BARD Open-source
• Cerebras GPT: https://huggingface.co/cerebras/Cerebras-GPT-6.7B
• Blender for dialogue

[rbroc]

see #51: at the end of the whole process, we might want to:

• update this with the models we are actually using
• reconsider our current choices (e.g., is using LLaMaChat & Mistral Instruct fair? do we want to include more models?) on the basis of an updated picture of the LLM landscape.

This should be done at the end of the project though, not before - too many new models all the time!