-
# Tokenizer Import Error When Using Ollama Models
## Description
When attempting to use Ollama models (llama3, llama3.1, mistral), the application fails due to a tokenizer import error. The error …
-
### What is the issue?
The streamed chat-completion response from ollama's openai-compatible API repeats `"role": "assistant"` in all returned chunks. This is different to OpenAI's API which just has…
-
I'm using :
- MacOS Ventura 13.2.1
- MacBook Air M1
When I execute the command :
```python setup_env.py --hf-repo HF1BitLLM/Llama3-8B-1.58-100B-tokens -q i2_s```
I got the message:
```
INFO:root:C…
-
Hello technically its not issue but can you add llama3 also. I can help if needed
-
I am trying to run ``examples/ToT/blocksworld/test_tot_v1_dfs.sh'' on step 2, step 4, step 6. However, I noticed that Llama 8B returns empty strings as actions.
example: ['unstack the yellow bloc…
-
### Your current environment
```text
vllm-0.6.4.post1
```
### How would you like to use vllm
I am using the latest vllm version, i need to apply rope scaling to llama3.1-8b and gemma2-9b…
-
### Checklist
- [X] 1. I have searched related issues but cannot get the expected help.
- [X] 2. The bug has not been fixed in the latest version.
- [X] 3. Please note that if the bug-related issue y…
-
```python
from edsl import Model
import time
models_list = [['Austism/chronos-hermes-13b-v2', 'deep_infra', 0], ['BAAI/bge-base-en-v1.5', 'together', 1], ['BAAI/bge-large-en-v1.5', 'together', …
-
below evaluation is vllm settings for llama3.2 evaluation
```
lm_eval --model vllm \
--model_args pretrained=/home/jovyan/data-vol-1/models/meta-llama__Llama3.2-1B-Instruct,dtype=auto,gpu_m…
-
I'm trying to fine tune a model (llama 3.1 8B Instruct) on a custom dataset.
The dataset is made of 3 fields: input, metadata and output. I could use the Alpaca style prompt, but I don't think it fit…