-
Hello,
I'm trying to use paper-qa with a "mixtral-8x7b-instruct-v0.1.Q4_K_M" on a local network. The LLM executable llamafile is launched with "-cb -np 4 -a my-llm-model --embedding" options as des…
-
**Describe the bug**
When I am using the most recent Megatrone-LM fork I get the following error
```
make: Entering directory '/workspace/megatron-lm/megatron/core/datasets'
g++ -O3 -Wall -sha…
-
### Feature request
I would like to implement the Mixtral model in Flax
### Motivation
I am in the process of learning Flax and I have almost finished the model conversion to FLAX.
### Your contri…
-
Original Repository: https://github.com/ml-explore/mlx-examples/
Listing out examples from there which would be nice to have. We don't expect the models to work out the moment they are translated to …
-
-
Thanks for the great work here!
I'm following along on the guide here to fine-tune the mixtral MoE version of Sphinx. https://github.com/Alpha-VLLM/LLaMA2-Accessory/tree/main/SPHINX#finetune-sphin…
-
### ⚠️ Please check that this feature request hasn't been suggested before.
- [X] I searched previous [Ideas in Discussions](https://github.com/OpenAccess-AI-Collective/axolotl/discussions/categori…
-
After I modified the code, there was a problem with the gate size of lora weight. After loading, I found that lora_a was the same as base_layer, and a size_mismatch problem occurred. Thanks!
-
Running into issues when serving Mixtral 8x7B on 4 x H100 (TP=4) with deepspeed-mii v0.2.3 with all other arguments default in the base image from nvidia `nvidia/cuda:12.3.1-devel-ubuntu22.04`
The …
ghost updated
5 months ago
-
While loading mixtral I get
"AssertionError: Insufficient space in device allocation".
Command I used
"python ericLLM.py --model ./models/mistralai_Mixtral-8x7B-Instruct-v0.1 --gpu_split 24,24,24,24,…