search

containers / ai-lab-recipes

Examples for building and running LLM services and applications locally with Podman
Apache License 2.0
103 stars 106 forks source link

fix(model_server): missing prefix env #712

Closed axel7083 closed 1 month ago

axel7083 commented 1 month ago

In AI Lab we requires to have the MODEL_ prefix for the chat format and the hf_pretrained_model_name_or_path

The prefix has been removed by mistake in https://github.com/containers/ai-lab-recipes/pull/531

In the documentation we are still showing the prefix. This PR restore it, to make AI Lab able to set the chat format and hf_pretrained_model_name_or_path

https://github.com/containers/ai-lab-recipes/blob/7409fdca6b01f2184e818cd15969b4eb3fac792a/model_servers/llamacpp_python/README.md?plain=1#L130

rhatdan commented 1 month ago

LGTM