-
### 🚀 The feature, motivation and pitch
Using Qwen2.5 model : ValueError: This model does not support the 'embedding' task. Supported tasks: {'generate'}
reproduction :
`python -m vllm.entryp…
-
我在使用ChromaDB_VectorStore.__init__的时候没法设置使用Ollama的嵌入模型，类似这种embeddings = OllamaEmbeddings(base_url="http://127.0.0.1:8000",model="znbang/bge:large-zh-v1.5-f32")
现在的Ollama可以支持LLM和嵌入模型
-
Hello
I finetuned a voice using the finetune_gradio.py succesfully, now how am im using it as the model with the infer_gradio normal gui and not only in test tab of the finetune gui?
is it a way…
-
使用的是anaconda的虚拟环境，pythonpath环境变量也添加了modelscopeagent的路径
-
- [LMStudio](https://lmstudio.ai/)
- [Continue](https://continue.dev/docs/intro)
-
### ⚠️ 搜索是否存在类似issue
- [X] 我已经搜索过issues和disscussions，没有发现相似issue
### 总结
不支持ollama等本地大模型做后端吗?
### 举例
_No response_
### 动机
_No response_
-
```
Using chat settings: { model: 'groq/llama-3.2-90b-vision-preview', api: 'groq' }
Using embedding settings: { api: 'groq' }
Unknown embedding: groq . Using word overlap.
Logging in...
{}
Age…
-
I have been running the scripts from [https://docs.vllm.ai/en/latest/models/spec_decode.html](https://docs.vllm.ai/en/latest/models/spec_decode.html ) on how to do speculative decoding with vLLM.
H…
-
**机器配置**
Windows 11 专业版
16.0 GB
NVIDIA GeForce RTX 3050 Laptop GPU
**运行命令**
`bash ./run.sh -c local -i 0 -b hf -m Qwen-7B-QAnything -t qwen-7b-qanything`
选择1.8B
![image](https://github.com/…
-
尝试使用gradio来运行这个模型，但似乎运行完成后就自动停止了，用Jupyter Notebook运行的时候。
这是我的运行代码：
```model_id = r"/kaggle/input/baichuan2-chat-model-4bit/Baichuan2-7B-Chat-4bits"
model = AutoModelForCausalLM.from_pretrained(
…