When the backend is tgi, the build_url(...) function is simply returning the supplied URL parameter. When a user passes in the base URL of their TGI server, the result is that the a request is made against the root path and it is routed to /compat_generate. Most users would not expect to pass in ${TGI_BASE_URL}/generate. In addition, llm-ls doesn't appear to be compatible with the /generate_stream endpoint so there is no value in allowing the user to choose between the two.
Suggestion: detect and append /generate to the URL (or build out some more robust logic).
When the backend is
tgi, thebuild_url(...)function is simply returning the supplied URL parameter. When a user passes in the base URL of their TGI server, the result is that the a request is made against the root path and it is routed to/compat_generate. Most users would not expect to pass in${TGI_BASE_URL}/generate. In addition, llm-ls doesn't appear to be compatible with the/generate_streamendpoint so there is no value in allowing the user to choose between the two.Suggestion: detect and append
/generateto the URL (or build out some more robust logic).