-
## 端测rag向量数据库
Gemma2 2B 才刚掀起了端侧模型热,端侧 RAG 就来了!🔥🤯
C 语言实现,基于 SQLite 插件的「最快⚡」端侧向量数据库 sqlite-vec 开源,短时间暴涨 1.8K Star⭐
- 处理50万个 960 维向量仅 41 毫秒
- 支持 JS/Rust,支持 Llama.cpp 离线Embedding 和在线Embedding
…
-
### Feature request
i believe `labels` in the training of causal LMs means the value to predict at time `n`, i.e., the next token. in other words, i'd assume, if `labels` is given, it should be alrea…
-
### 🐛 Describe the bug
Data for vllm:num_requests_waiting is missing.
vllm:num_requests_waiting is not being published at /metrics endpoint
docker image for vllm : vllm-openai:v0.5.3.post1
…
-
Hi,
I'm experiencing issues while running GEMMA2 on Ollama. Specifically, I'm encountering the following problems:
Error on Message Generation:
Whenever a new chat is created, every message…
-
### Your current environment
```text
The output of `python collect_env.py`
```
### 🐛 Describe the bug
python3 -m vllm.entrypoints.openai.api_server --model /model/models/gemma-2-27b-it/ --dtyp…
-
1) Load the Gemma2 2B model with Unsloth - OK
2) Perform fine tuning - OK
3) Test the resulting model - OK, responses indicate fine tuning is successful
4) Save 16 bit `model.save_pretrained_merged…
-
### Your current environment
Collecting environment information...
PyTorch version: 2.3.0+cu121
Is debug build: False
CUDA used to build PyTorch: 12.1
ROCM used to build PyTorch: N/A
OS: Cen…
-
Hey,
I had used unsloth for faster finetuning of gemma2 9B, with default configuration as suggested by unsloth.
Here’s the public colab of same https://colab.research.google.com/drive/1vIrqH5…
-
I've seen online that ollama works when using Gemma2 but nothing mething any ollama version of llama3. I want to create a chatbot that is based off of my local version of llama3 but every time I uploa…
-
### What happened?
I want to convert gemma2 model to GGUF, I am using this code convert-legacy-llama.py which is present inside examples folder. But got this error ImportError: cannot import name '…