-
Not sure if we should consider this out of scope, but `bloomz.cpp` is a fork of `llama.cpp` that's capable of inference with the BLOOM family of models. The changes don't look very large, so there's r…
-
如题，在使用代码lora微调模型后，checkpoint中存的应该是lora的weight，但是在生成时，--model_name_or_path设置的是原始模型路径，--lora_weights设置的是存在checkpoint的lora权重，但是在Peft加载的时候报错Can't find config.json at trained_models/llama/checkpoint-*** 请…
-
### Feature request
Current [text-generation](https://github.com/huggingface/optimum-habana/tree/main/examples/text-generation) only support bloom & bloomz with deepspeed, but not support other gener…
-
-
训练参数如下
deepspeed --num_gpus 1 main.py \
--sft_only_data_path law_total_conv.js…
-
### Feature request
Currently models obtained by the [torch.compile()](https://pytorch.org/docs/stable/generated/torch.compile.html) feature introduced in Pytorch 2.0 are not supported in inference p…
-
**Describe the bug**
Deepspeed did not split the model into shards amoung gpus, instead it launches two identical models on two gpus, without saving both gpu memory and cpu memory.
**To Reproduce*…
-
**Enviroment**
peft == 0.4.0.dev0
transformers == 4.30.0.dev0
**Statement**
The error occurred when I load lora model which trained with peft==0.1.0, and loaded with a higher version peft(I trie…
-
### Describe the bug
prompt是采用multimodal/models中的minigpt4和instructblip中的样例进行prompt吗？还是有设计prompt，我是用样例的简单prompt复现mplug时远不及论文中的验证集的49%
'''python
img_prompt = '###Human: '
if '…
-
### Buggy output
```
WARNING:torch.distributed.run:
*****************************************
Setting OMP_NUM_THREADS environment variable for each process to be 1 in default, to avoid your system…