-
I’ve tried to prune Llama2-7B on a MacBook Pro M1 but the system end it by killing the process because of OOM (I’ve 32GB)
Is there something I can do? Did somebody prone this model and publish it?
…
-
Dear community,
I tried to fine-tuning llama2-7B using dp-transformers based on examples, but received the below error:
`ValueError: Per sample gradient is not initialized. Not updated in backw…
-
In 05_dataloader.ipynb in examples folder https://github.com/NVIDIA/GenerativeAIExamples/blob/v0.4.0/notebooks/05_dataloader.ipynb
is there a way to use Llama2 7b model instead of default Llama2 13b…
-
1. Download weights from [meta-llama/Llama-2-7b-chat-hf](https://huggingface.co/meta-llama/Llama-2-7b-chat-hf).
2. Combine weights into a single safetensors file.
3. Convert safetensors to llama2-7b…
-
报错如下,请问对910B2C是否适配验证过呢?使用cann包为RC1,RC2,报错都如下所示。
(base) [root@A03-R40-I18-71-0001195 dengyingxu1]# cat /root/atb/log/atb_17738_20240815033442.log
[2024-08-15 03:34:42.482189] [info] [17738] [con…
-
Hi, I failed to reproduce the Llama2-7b-4k (w/o SFT) in the paper.
Here is our result:
|Methods | Tokens | Coursera | GSM | QuALITY | TOEFL | CodeU | SFiction | Avg |
…
-
i use the quantized model by smooth quant , Why hasn't the inference speed increased?
`lmdeploy lite smooth_quant /model/llama2-7b-hf/ --work-dir /model/lmdeploy/llama2-7b-w8/`
使用 smooth 量化后,模型文…
-
### Checklist
- [x] 1. I have searched related issues but cannot get the expected help.
- [x] 2. The bug has not been fixed in the latest version.
- [x] 3. Please note that if the bug-related iss…
-
@JLTastet @timinar Excuse me,how should i distill the Llama-2-7B model to obtain a 3.5B Llama-2 model by BabyLlama?At the same time, I want to use the local Llama-2-7B model whose path is ``` /home/Ll…
-
### System Info
GPU: H20 server
CUDA Version: 12.5
Driver: 555.42.02
TRTLLM Commit: 2d234357c6e69fa514f6e9b4d4a5ad3bc431c4a6
built from source on linux
### Who can help?
_No response_
…