Chore: Model.yml new parameters

[hahuyhoang411]

Problems

Current model.yml only supports text

New features

• [ ] Add support parameters for VLM

  • [ ] Parameters
  • [ ] Cortex support: https://github.com/janhq/cortex.cpp/issues/1493

• [ ] Add support parameters for tool use

  • [ ] Parameters
  • [ ] Cortex support: https://github.com/janhq/cortex.cpp/issues/295

[hahuyhoang411]

1. For VLM/ SLM:

   • new parameters should be added in
   • VLM: img_end: ... path to mmproj
   • convert mmproj to gguf: https://github.com/ggerganov/llama.cpp/tree/master/examples/llava#llava-16-gguf-conversion
   • latest VLM doesn't have mmproj, they include the projector inside the model: https://github.com/ggerganov/llama.cpp/blob/master/examples/llava/README-minicpmv2.6.md
   • SLM: speech_end: ... path to Whisper
   • (Future) any2any model has both encoder and decoder for modality:
   • we might need to introduce speech_end, img_end and img_dec, speech_dec
   • Example Emu3: https://huggingface.co/BAAI/Emu3-Gen

2. For tool use:

   • new parameters should be added in tool: None as default value.tool accepts list (e.g tool: []) Ref: https://platform.openai.com/docs/guides/function-calling