-
MosaicML released its MPT 30B version today with 8k context, with Apache 2.0 license.
![image](https://github.com/ggerganov/llama.cpp/assets/27340033/fdb8a0aa-e73b-4e7f-9f3b-58a389f7b472)
## Why…
-
**Describe the bug**
A clear and concise description of what the bug is.
I encounter the following error when I am converting GPTNeoX models to Huggingface using the `tools/convert_module_to_hf.py…
-
**LocalAI version:**
V1.21
root@63429046747f:/build# ./local-ai --version LocalAI version 4548473 (4548473acf4f57ff149492272cc1fdba3521f83a) llmai-api-1 | 3:04AM DBG Loading model '
**Environment, C…
-
I tried to run the _gpt-neox_ demo CLI @0.4.60 on an **inf2.24xlarge** instance.
I first saved the model with float16 precision:
```
$ gptneox_demo --amp f16 save gpt-neox-20b
```
When tryi…
-
The GPT-Neox-20B model is too big to run on an **inf2.8xlarge** instance, so I tried to convert a model with the same architecture but less parameters: EleutherAI/pythia-1.4B.
I first saved the mo…
-
I get a tensor size mismatch for the regex guided generation here. Although this might be model specific
```python
model = models.transformers("togethercomputer/RedPajama-INCITE-Instruct-3B-v1")
…
-
**Describe the bug**
An error occurred while fine-tuning the model.
https://github.com/togethercomputer/OpenChatKit#fine-tuning-the-model
**Screenshots**
```
data_utils: get train_data_loader
…
-
I am running the latest code.
I was expecting a way to load the models from gptneox directly by running some kind of binary "main-gptneox" or something similar, instead, I see a ".so" library. Is t…
-
I accelerated GptNeoX with FT and deployed with Triton. But I found that GptNeoX can't stop inference when encountering end_id.
-
### Branch/Tag/Commit
main
### Docker Image Version
nvcr.io/nvidia/pytorch:22.09-py3
### GPU name
A6000
### CUDA Driver
525.60.11
### Reproduced Steps
```shell
0. download …