-
Currently https://github.com/0cc4m/GPTQ-for-LLaMa/tree/gptneox has to be manually installed and a few lines have to be modified in webui to load the pre-quantized model. Would be good to automate this…
-
Today I am trying to run AutoGPTQ on a GPTNeoX-based model. Specifically the model is: https://huggingface.co/h2oai/h2ogpt-oasst1-512-20b
Here is the error I receive when I execute `model = AutoGPT…
-
Hi there!
I'm one of the maintainers of https://github.com/rustformers/llama-rs, and we've recently been expanding our model support to cover GPT-NeoX as well as other models. As part of this, we'v…
-
Hi All, I am deploying a model on SageMaker and getting the following error. Any advice on how to resolve it?
```
UnexpectedStatusException: Error hosting endpoint gptneox-demo-v1-0-0-endpoint: Fa…
-
GPTNeoX get wrong outputs when input text is long.
The output is similar with https://github.com/microsoft/DeepSpeed/issues/2293 , but only occured when input text is long.
**Platform**
Nvidia …
-
I am not sure if this is how it is supposed to work, but no context is carried over to subsequent messages. For example, when running `./scripts/chat-pythia-12B.sh`.
You can do the context yourself…
-
Flash attention has already been integrated into gpt-neox models here: https://github.com/HazyResearch/flash-attention/blob/main/flash_attn/models/gpt.py#L215
Can add the swapped model definition …
-
Since GPT-NeoX support (Pythia based Open Assistant models) has been confirmed by #24 , we can port other chat-models.
Instructions:
- Port to GGML Format: `python3 cformers/cpp/converters/convert…
-
I was trying to train a language adapter for a `gpt-neox` I trained using the following script:
```
OUTDIR="."
python run_clm.py \
--language de \
--model_name_or_path gpt-neox/h…
-
**Describe the bug**
I am using 0.9.4 version after reflecting the two PRs below, but cpu inference is not working
CUDA optional deepspeed ops https://github.com/microsoft/DeepSpeed/pull/2507
Enabl…