-
Loading these models and being able to use them for inference or in a repl is very cool.
To make this broadly useful, we need a way to verify that models loaded by llm are acting the same way as they…
-
It would be nice, for us non-English people, to have access to the some good multilanguage/multilingual LLMs.
Bloom comes to mind, but other would be very useful.
https://huggingface.co/blog/bloom
-
-
### Branch/Tag/Commit
main
### Docker Image Version
nvcr.io/nvidia/tritonserver:22.12-py3
### GPU name
A100
### CUDA Driver
470.103.01
### Reproduced Steps
```shell
I am us…
-
To save GPU memory, I want to load the multilingual model in 4bit mode, the code is as follows.
```python
import torch
from transformers import AutoTokenizer
from mplug_owl.modeling_mplug_owl impo…
-
**LocalAI version:**
at * f227e91 (origin/master, origin/HEAD) feat(llama.cpp): Bump llama.cpp, adapt grpc server (#1211)
**Environment, CPU architecture, OS, and Version:**
Mac Studio M2 Ult…
-
### Feature request
ggml is gaining traction (e.g. llama.cpp has 10k stars), and it would be great to extend optimum.exporters and enable the community to export PyTorch/Tensorflow transformers wei…
-
作者大佬您好，感谢您的贡献和输出，因为我对RLHF的这部分比较陌生，所以想咨询您几个问题，希望得到您的指点：
1. 如果我底座模型是其他的模型，比如：Baichuan2，或者ChatGLM2，然后SFT的时候是自定义的训练数据，这种模式是可以使用你们的发布的RLHF的代码么
2. 如果1可以的话，那么意味着我需要重新训练RM，然后PPO，我想了解这种场景，你们当前的代码是否可以支持
3. 如…
-
hi ,thanks for your sharing.
about custom own sentence, can you also show the sentence examples and usually how do you create a large amount of sentence?
-
**Describe the bug**
When inference bloom model with `replace_with_kernel_inject = True` , and `dtype = torch.int8`
For the reason that this model is trained by torch , I load the weight with tor…