Is your feature request related to a problem? Please describe.
we want dynamically show available models from the llm server instead of always have to hard-coded a static list.
And if there are some model specific configs in the backend that changes system prompts or other things then we need to update it so that the model coming in the request can be used instead of the default one.
Describe alternatives you've considered
alternative will be removing the static model setting in configuration and also all checks. but this requires more discussions as people have different opinion on it
Additional context
Add any other context or screenshots about the feature request here.
Is your feature request related to a problem? Please describe. we want dynamically show available models from the llm server instead of always have to hard-coded a static list.
but current it is a hard-requirement that the query model requested has to be in the list of models set in road-core configuration. and upon every chat query it validate if the requested model is defined in the list https://github.com/road-core/service/blob/3ea924f4945c0677e3ba8576161b37fb14b15075/ols/src/llms/llm_loader.py#L46-L50
see detailed description and discussion: https://redhat-internal.slack.com/archives/C07MC7G9T8A/p1732570149978479
Describe the solution you'd like
optionally disable the model check.
And if there are some model specific configs in the backend that changes system prompts or other things then we need to update it so that the model coming in the request can be used instead of the default one.
Describe alternatives you've considered
alternative will be removing the static model setting in configuration and also all checks. but this requires more discussions as people have different opinion on it
Additional context Add any other context or screenshots about the feature request here.