-
### System Info
peft=0.3.0
accelerate=0.20.3
transformers=4.30.2
platform=debian.amd64
python=3.7.12
### Who can help?
_No response_
### Information
- [x] The official example scrip…
-
### System Info
- docker image: `pytorch/pytorch:2.1.1-cuda12.1-cudnn8-devel`
- pip list
```
Package Version
----------------------- ------------
accelerate 0.25.0…
-
@younesbelkada
(Thanks again for developing these great libraries and responding on Github!)
Related issue: https://github.com/huggingface/accelerate/issues/1412
With the bleeding edge `transf…
-
**功能描述 / Feature Description**
Bloom系列模型，包括 Bloomz_560m, bloomz_1b1, bloomz_3b, bloomz_7b1_mt
平均评测表现均超过MOSS模型。后两个模型接近ChatGLM.
感谢团队的大爱~~
参考文章：https://www.geekpark.net/news/318946
-
### Feature request
I would like to inquire if prompt tuning feature is available with 4-bit quantized LLAMA-2 model.
### Motivation
4-bit quantization makes the models easy to use and hardwa…
-
I try to finetune the bloomz-1b7 model for translation and using peft lora.
And the fine-tuned model without lora is twice as fast as the one with lora.
I use the TextGenerationPipeline to generate …
-
### System Info
```Shell
- `Accelerate` version: 0.22.0
- Platform: Linux-4.15.0-142-generic-x86_64-with-glibc2.27
- Python version: 3.10.12
- Numpy version: 1.25.2
- PyTorch version (GPU?): 2.0.…
-
Currently, BLOOMZ behaves well only for the first output in the few-shot mode, then outputs `` and forgets everything. This is visible in the English-to-Spanish translation example.
We need to use …
-
Bloom 7b model example:
**# int8 weight only + int8 kv cache**
`python3 hf_bloom_convert.py -i ./bloomz-7b -o ./bloom_7b_kv --calibrate-kv-cache -t float16`
**# Build model with both INT8 weigh…
-
### Describe the bug
在基于bloomz-560m模型训练rm模型，观察到训练过程中仍然是1块gpu在训练；
### To Reproduce
训练脚本如下：
…