-
### Evaluation
- [x] Code to clean up Dataset/Map for [HF release](https://github.com/bigscience-workshop/ShadesofBias/blob/master/map_dataset.py)
- [x] Add code to constrain generation to just a f…
-
Can MeZo be used on NLG tasks? I integrated the _inner_training_loop part of the code and the methods it relies on into the NLG task training code, and performed fine-tuning training on bloom (bloomz-…
-
Just a curious question I suppose!
GPTQ 4bit - https://github.com/qwopqwop200/GPTQ-for-LLaMa
Suppose someone eventually finetunes 175B OPT model, with loras or regular finetunng. or perhaps the BLOO…
-
I'm trying to run https://github.com/bigscience-workshop/petals/blob/main/examples/prompt-tuning-personachat.ipynb and it is failing on default settings with these exceptions:
```
Feb 08 10:24:01.…
-
**Describe the bug**
DeepSpeed-inference 2GPU performance is lower than 1GPU on Bloomz 7.1B
**To Reproduce**
run the flowing code using two V100 or A100 GPUs.
with this command:
deepspeed --nu…
-
**Describe the bug**
There is a problem with asynchronous communication in zero stage2 by using `overlap_comm`.
**To Reproduce**
Steps to reproduce the behavior:
Use deepspeed zero-2 on the hugg…
-
After finetuning the bigscience/bloomz-7b1, I encountered this issue while doing evaluation.
│ envs/lmflow/lib/python3.9/site-packages/peft/utils/save_and_load.py:74 │
│ in set_peft_model_state_…
-
**LocalAI version:**
commit 3829aba869f8925dde7a1c9f280a4718dda3a18c/ docker 6102e12c4df1
**Environment, CPU architecture, OS, and Version:**
MacBook Air M2, Ventura 13.4
**Describe the …
-
您好，在使用finetune脚本使用指令微调数据集微调bloom-7b模型时前几个step出现：
tried to get lr value before scheduler/optimizer started stepping, returning lr=0
这个warning是什么原因呢？
bloom config为:
{
"model_type": "bloom…
-
Thanks for your project. I have a few wishes: the most important thing is that the models cannot translate more than one sentence (after the dot it does not translate in most cases), the answers are c…