janhq / cortex

Drop-in, local AI alternative to the OpenAI stack. Multi-engine (llama.cpp, TensorRT-LLM, ONNX). Powers 👋 Jan
https://cortex.so
Apache License 2.0
1.81k stars 98 forks source link

epic: Cortex supports preset configuration list #617

Open Van-QA opened 2 months ago

Van-QA commented 2 months ago

Problem: Users are unable to use imported GGUFs effectively as they all use default templates and parameters, requiring individual editing of each model's model.json file. https://discord.com/channels/1107178041848909847/1244886808979046452

Success Criteria:

louis-jan commented 1 month ago

For number 2, there is no need to put the YAML file under the model folder, since symlink support is available; all YAML files are located at /models.

louis-jan commented 3 weeks ago

Do you want to take the 4 @Van-QA?

Van-QA commented 3 weeks ago

Do you want to take the 4 @Van-QA?

I will work with @hahuyhoang411 on this, thank @louis-jan