Add Llama 3.2 to Chat models

[XavierElon]

Validations

• [X] I believe this is a way to improve. I'll try to join the Continue Discord for questions
• [X] I'm not able to find an open issue that requests the same enhancement

Problem

Meta just released the 3.2 with Lightweight 1B & 3B and Multimodal 11B & 90B. It would be nice to have the option to use this as one of the models.] https://www.llama.com/

Solution

No response

[RomneyDa]

@Patrick-Erichsen @sestinj

• should 3.2 replace 3.1 as default for onboarding, docs, etc.?
• should/can support be added for multimodal?
• does llm-info package need to be maintained? packages/llm-info

[sestinj]

• I'm going to go with no for the moment because they are different size models (3.2 has 1,3,11,90B, and current rec is 8B). Some evaluation could convince me otherwise but can be done not now
• Yeah, look at autodetect.ts where we maintain a list of matchers for models that support multi-modal
• Yes, llm-info package will hopefully become our source of truth for all LLM metadata (replacing autodetect.ts for example)

[RomneyDa]

@sestinj adding your other comment on replacing llama 3.1 with 3.2 here:

Llama3.2 was more focused on multi-modal and edge (small) models, so they didn’t turn out to be as good for code. The one of comparable size to llama3.1 8b that we would recommend (llama 3.2 11b) does much worse on coding benchmarks: https://ai.meta.com/blog/meta-llama-3-1/ https://ai.meta.com/blog/llama-3-2-connect-2024-vision-edge-mobile-devices/ Meta AIMeta AI Llama 3.2: Revolutionizing edge AI and vision with open, customizable models Today, we’re releasing Llama 3.2, which includes small and medium-sized vision LLMs, and lightweight, text-only models that fit onto edge and mobile devices.

[RomneyDa]

TODO

• update Prompt Templates autodetectTemplateType function to return correct template type per model
• update ModelName in index.d.ts
• update core/llm/llms/ (model name translation), config_schema.json, config_rc_schema.json for providers:
  • Groq
  • Cerebras
  • Ollama
  • Replicate
  • SambaNova
  • Together
• update gui/AddNewModel with ModelPackage and add to providers
• update autodetect.ts - matchers for models that support multi-modal
• add note to docs/customize/tutorials llama 3.1 page about 3.2 also being supported but not recommended
• update docs/customize/model-providers/more cerebras.md with note about 3.2
• gui/src/pages/addNewModal/configs/providers sambanova cloud copy about llama 3.1?
• docs/customize deepdives - note 3.2 also includes repo map
• update core/llm/llms/freetrial? prob not
• update llm-info package including Bedrock provider models
• test