-
# Description
Hey guys,
I'm trying out the repo with the same structure as the DeepSeek example with local llama 3.1 8B as both the best and cheap model. The problem is, I get empty dictionaries…
-
Fine tuning [example](https://github.com/huggingface/optimum-tpu/blob/main/examples/language-modeling/llama_tuning.md) **works** with `Llama-3.1 (8b)` with `Transformers version 4.43.3` and modificati…
Bihan updated
3 weeks ago
-
加载Meta-Llama-3.1-8B 做pretaining时报错
raise ValueError(
ValueError: Asking to pad but the tokenizer does not have a padding token. Please select a token to use as `pad_token` `(tokenizer.pad_token = to…
-
### System Info
text-generation-launcher --env
2024-07-26T03:39:42.960734Z INFO text_generation_launcher: Runtime environment:
Target: x86_64-unknown-linux-gnu
Cargo version: 1.79.0
Commit sha…
-
### System Info
(myenv) ubuntu@i~$ python --version
Python 3.10.14
(myenv) ubuntu@i~$ pip --version
pip 23.0.1 from /home/ubuntu/myenv/lib/python3.10/site-packages/pip (python 3.10)
### Informa…
-
LLaMa 3.1 can use what is called built-in tools. These built-in tools are signaled to the executor by the special token ``, also they are not user-defined JSON tools. On top of that there is a new `i…
-
We find that LLaMA-3.1-70B-Instruct outputs duplicated brackets at a high frequency. For example,
> Since you didn't provide the full error message, I couldn `[ [ [ [ [ [ [ [ [ [ [ [ [ [ []]]]]]]]]]]…
-
### Your current environment
- vLLM version: v0.5.3.post1 (Public Docker Image )
- Model: Llama 3 70 B
- Dtype: FP16
- GPU: Nvidia H100
### 🐛 Describe the bug
The vLLM metrics endpoint is showin…
-
### System Info
AWS p5 (4 x 80GB H100 GPUs)
TensorRT-LLM v0.11.0
### Who can help?
@byshiue @Tracin
### Information
- [X] The official example scripts
- [ ] My own modified scripts
…
-
The recent [Llama 3.1](https://github.com/meta-llama/llama-models/tree/main/models/llama3_1) release needs to be integrated with our optimized model definitions.
Import points that need to be updated…