-
i try to stf llama2-7b and oom, can it support fsdp or tensor parallel
-
-
The prompts for Llama 2 have not been provided in `prompts/adv_prompts`, so running `load_adv_prompt` doesn't work when using Llama 2. Could these be added, please? Thanks!
-
This issue tracks the open issues the model team must solve in order to hit Llama2 perf targets.
## Decode 128
We have a new perf target of 20 tok/s at seqlen = 128. This issue lists the problems …
-
I am not getting access to download Meta Llama2 and Llama3, I submitted request in early days when Llama2 was released and on the first day of Llama3 release, but still didn't got approval.
I alre…
-
### System Info
tensorrt-llm version 0.11.0.dev2024062500
Architecture: x86_64
AMD EPYC 9354 32-Core Processor
``` txt
+----------------------------------------------------------…
-
### System Info
- GPU Name: Tesla V100-SXM2-32GB
- TensorRT-LLM: 0.10.0
- CUDA: 12.4
- Nvidia Driver: 550.54.14
- OS: Ubuntu 18.04
### Who can help?
_No response_
### Information
- …
-
Hi, this is a very interesting work! One thing I don't understand is whether the self-distillation is rewriting using Llama2-chat and further fine-tuning Llama2-chat as well, or is it just fine-tuning…
-
感谢作者的工作,提供了一个解决 cl 灾难性遗忘的思路。
我采用 codebase 提供的 llama2 的脚本,跑出来的结果直接坏掉了,这是什么原因呢,跑实验的过程中,有什么要点需要注意么,或者参数设置上需要做些什么调整呢?是 olora 的 lamda 参数设置太小导致过多的遗忘么?下面是我在 tune order2 时的逐 task 结果
***** predict metrics **…
-
Hello!
Commit `2badd76` appears to break `examples.models.llama2.export_llama`, specifically with Llama 3.
### Expected Behavior
```
[INFO 2024-06-14 16:04:23,366 export_llama_lib.py:390] Ap…