Closed francis2tm closed 5 months ago
The model parameter in ai endpoint requests is now considered again. There are four valid cases:
model-file
, such that the <config model dir> + "/" + <model-file>
exists, e.g.:
"model": "my-model.gguf" => ~/.local/share/edgen/models/chat/completions/my-model.gguf
In this case, the huggingface API is bypassed. If the file does not exist, the endpoint returns "No Such Model".owner/repo/model
, e.g.:
"TheBloke/TinyLlama-1.1B-Chat-v1.0-GGUF/tinyllama-1.1b-chat-v1.0.Q2_K.gguf" =>
~/.local/share/edgen/models/chat/completions/models--TheBloke--TinyLlama-1.1B-Chat-v1.0-GGUF
In this case, the huggingface API is used to identify the model and, if necessary, to download the model.There is an ambiguity with the approach: if the model is in a subdirectory, e.g.: ~/.local/share/edgen/models/chat/completions/my-models/top-models/my-top-model.gguf It could be a file or a model identifier with "my-models" as owner, "top-models" as repo and "my-top-model.gguf" as model. If the file exists, the huggingface API will be by-passed. Otherwise, edgen will try to download it from huggingface.
This already happens with the default model, i.e. if there's no model present in an endpoint, edgen auto downloads the default model. The same behaviour should happen with the requested model (i.e. the value in the
model
attribute of the request)The format of
model
should be:<hf_repo_owner>/<hf_repo>/<model_name>
Example:
If the request model is not valid, return an error