search

xorbitsai / inference

Replace OpenAI GPT with another LLM in your app by changing a single line of code. Xinference gives you the freedom to use any LLM you need. With Xinference, you're empowered to run inference with any open-source language models, speech recognition models, and multimodal models, whether in the cloud, on-premises, or even on your laptop.
https://inference.readthedocs.io
Apache License 2.0
3.63k stars 303 forks source link

是否考虑支持Sqlcoder #1693

Open leeyis opened 2 weeks ago

leeyis commented 2 weeks ago

sqlcoder

目前sqlcoder在text2sql领域表现不错，在很多数据应用开发中被广泛使用，希望官方可以提供推理支持

qinxuye commented 2 weeks ago

这个模型之前看过，但用法好像和别的 LLM 不太一样，需要专门适配。你有啥看法吗？

mrkingsun commented 1 week ago

看到dbgpt适配了llama3-sqlcoder-8b，按照sqlcoder官方的prompt模板应该就可以，希望xinference可以适配

suiquantong commented 1 week ago

+1

nikelius commented 6 days ago

sqlcoder已经有三代版本了，三者的基座模型都不一样：

nikelius commented 6 days ago

ollama都适配了，差别会很大吗？