Closed bufferoverflow closed 3 months ago
Sounds useful to me, would be great to PR it.
You can make it an EngineArg
, since its closely coupled with the model. You might name it model-display-name
, which defaults to None
. Hoping someone PRs #13 so that might be better compatible then.
Feature request
Add a
--served-model-name
option to control the model name.Motivation
I ran:
Query the models endpoint:
Query the embeddeings endpoint:
Via embeddings endpoint the model is
BAAIbge-m3
for the model endpoint it isBAAI/bge-m3
, Somehow would be cool to control the name.vLLM is doing this e.g. with the following options:
--served-model-name
: The model name used in the API. If not specified, the model name will be the same as the huggingface name.--model
: name or path of the huggingface model to useYour contribution
I can create a PR for this