Closed karthik19967829 closed 3 months ago
There is hugging face model support in the code. But I am not sure, if they tested with LLaMa or other HF models, as the paper do not share those results in any appendix. Looking forward, if authors can share some insights
one way to support this is to use vllm to set up an inference endpoint, and then serve the OSS model like LLAMA with it, and use that new OpenAI-compatible endpoint.
Hi, do we support huggingface models? If any internal internal tests were done with llama and the insights can be shared that will be great