-
Hi all - I'm working on an issue where users on M1 apple silicon get `ggml_new_tensor_impl: not enough space in the context's memory pool ` when they try to use starcoder or gptneox models from turbop…
-
huggingface model config:
{
"activation_function": "gelu",
"architectures": [
"GPTNeoXForCausalLM"
],
"bos_token_id": 0,
"eos_token_id": 2,
"hidden_act": "gelu",
"hidden_siz…
-
Hi,
GPT/GPT-J/GPT-Neox have similar nn architecures. In my view, the implementations of them in `src/fastertransformer/models` (`multi_gpu_gpt`, `gptj`,`gptneox`) are also very similar. I am wonde…
-
Ubuntu Ubuntu 22.04.2 LTS
After downloading the model and now trying to convert:
```
(OpenChatKit) georgi@georgi-hackintosh:~/Documents/GitHub/OpenChatKit$ python3.10 tools/convert_to_hf_gptneox.…
-
I play with `examples/cpp/gpt/gpt_example.cc` and found that generation of tokens does't finish when the first EOD is reached.
This is my gpt_config.ini.
I use gpt2 model which was converted int…
-
Hey guys
Today I was doing quants of a [new GPTNeoX model called Literature-7B-16384](https://huggingface.co/hakurei/Literature-7B-16384)
I tried making GGMLs through the usual process:
```
py…
-
1. [AOTAutograd](https://github.com/pytorch/functorch) is a novel engine provided by functorch that can fuse all parts of a neural network. I added it to [OSLO](https://github.com/tunib-ai/oslo/tree/m…
-
Error info:
File "/opt/conda/lib/python3.8/site-packages/deepspeed/runtime/hybrid_engine.py", line 99, in new_inference_container
File "/opt/conda/lib/python3.8/site-packages/deepspeed/module_…
-
**LocalAI version:**
quay.io/go-skynet/local-ai:v1.18.0-ffmpeg
rebuild with GO_TAGS=stablediffusion
**Environment, CPU architecture, OS, and Version:**
rtx4060/ryzen5700/32G
**Describ…
-
When inspecting the config of the hybrid model https://huggingface.co/state-spaces/mamba2attn-2.7b/blob/main/config.json, I came up with two questions:
- Why is the number of heads 30? Wouldn't we us…