-
Thank you again for your excellent work. I have trained a model mT0 using my own dataset, and it performs well. Now, I am attempting to train bloomz model, but I'm encountering an issue where the trai…
-
The article at https://huggingface.co/blog/generative-ai-models-on-intel-cpu mentions that smoothquant was applied on bloomz 7b1 model also. But in https://huggingface.co/mit-han-lab/smoothquant-scale…
-
Operaring system: Ubuntu 22.04.2, Python 3.10.6, CTranslate2 3.16.
When exporting the bigscience/bloomz model using:
_ct2-transformers-converter --force --model bigscience/bloomz --output_dir …
-
RUNNING
```
from awq import AutoAWQForCausalLM
from transformers import AutoTokenizer
model_path = '/data/bloomz_7b1'
quant_path = '/data/bloomz_7b1_4bit'
quant_config = { "zero_point": True, …
-
### System Info
GPU A100
TRT-LLM 0.8.0.dev2024013000
### Who can help?
@Tracin
### Information
- [X] The official example scripts
- [ ] My own modified scripts
### Tasks
- [X] An officially su…
-
Hi @PanQiWei
I'd be most grateful if you could give me a bit of help.
I have been trying to quantize BLOOMZ 175B but can't currently get it done. BLOOMZ has 70 layers, and is a total of 360GB.…
-
**Describe the bug**
I was looking for the bloomz models on jumpstart and I noticed the task name for them is oddly `textgeneration1`. Is that on purpose ?
**To reproduce**
Code snippet:
```
…
-
Hi thank you very much for this great library! I am really excited about it!
I tried to fine tune bloomz 7B with the 4 bit lora on alpaca by running: python qlora.py --model_name_path my_bloomz_path.…
-
Hi!
I'm trying to quantize a 3B Bloom Model (https://huggingface.co/bigscience/bloom-3b). But it seems like it's missing the alibi tensor when performing the forward pass of the model.
Could y…
-
I follow the readme :
## Build model with both INT8 weight-only and INT8 KV cache enabled
python convert_checkpoint.py --model_dir ./bloom/560m/ \
--dtype float16 \
…