-
### 🚀 The feature, motivation and pitch
FlashInfer v0.1.2 was just released with sliding window support:
https://github.com/flashinfer-ai/flashinfer/releases/tag/v0.1.2
This should allow vLLM…
-
**Description:**
One of the reasons Ollama is so widely adopted as a tool to run local models is its ease of use and seamless integration with other tools. Users can simply install an app that star…
-
As a [user],
I want a drop down menu in the Model accordion to display different model options.
So that I can choose the model that best fits my needs.
Acceptance Criteria:
1. The AI Model accordi…
-
### Your current environment
- vLLM CPU : v0.6.0
- Hardware: Intel(R) Xeon(R) Platinum 8480+ CPU
- Model: google/gemma-2-2b
### 🐛 Describe the bug
vLLM v0.6.0 (cpu) is throwing below erro…
-
### What is the issue?
I have updated Ollama to v0.2.1 and pulled gemma2:9b-instruct-fp16. When I run `ollama run gemma2:9b-instruct-fp16` it fails to run.
Hardware
CPU: Intel Core i9 14900K
RA…
-
When I use the "qwen2.5" LLM model in the file `using_ollama_as_llm_and_embedding.py`, I am unable to extract any entities.
After running the command:
```bash
$ python ./examples/using_ollama_a…
-
I found that the current repository configuration is not compatible with Gemma2. The reason might be that transformers and vllm are not fully compatible with Gemma2. Could you share the package config…
-
> Confirming exact same error ; mergekit can not find the "base_model" ; including if the path is local (absolute) on windows.
>
> Funny thing is some mergekits work fine - no issue, where as others…
-
比如bge-reranker-v2-minicpm-layerwise, bge-reranker-v2.5-gemma2-lightweight等模型
示例中
使用hunggingface的库加载，加入了prompt
使用FlagEmbedding的库加载，没有使用prompt，应该是自动加入了
但bge-reranker-v2.5-gemma2-lightweight，好像提出了4…
-
Hi All,
Thank you for your amazing work!
Where can we find a list of models that support Structured JSON Generation? Do all the models support that?
We were able to find a list of models in the [HF…