-
I notice Mixtral has been added - but it's not included in the index. Is this an oversight, or is it still not supported?
-
I am getting "float division by zero" error whenever I try to quantize mixtral related models with autogptq,
and here is my code.
```
from transformers import AutoTokenizer, TextGenerationPipeli…
-
Hi there,
Thanks for the scripts and posts! I am interested in fine-tuning Mixtral 8x7b on sagemaker. The task I have requires around 8k token length.
I have tried running training following th…
-
## v0.3.0 openai.api_server fails for Mixtral-8x7B: FileNotFoundError
### Description
* vLLM v0.3.0 openai.api_server fails for Mixtral-8x7B: FileNotFoundError
* vLLM v0.2.7 openai.api_server w…
-
---------------------------------------------------------------------------
NameError Traceback (most recent call last)
Cell In[20], [line 2](vscode-notebook-cell:?ex…
-
### Please check that this issue hasn't been reported before.
- [X] I searched previous [Bug Reports](https://github.com/OpenAccess-AI-Collective/axolotl/labels/bug) didn't find any similar reports…
-
### System Info
ubuntu 20.04
tensorrt 10.0.1
tensorrt-cu12 10.0.1
tensorrt-cu12-bindings 10.0.1
tensorrt-cu12-libs 10.0.1
tensorrt-llm 0.10.…
-
I want to use the mixtral 8X7B model for inference, but currently it only supports autoTP. How to add more support to enable it to use more parallelism (e.g. EP, DP)
-
An increasing number of LLM tools are backend-agnostic, allowing users to process text using OpenAI, other hosted APIs, or self-hosted models based on the LLaMa and Mixtral series.
We should at lea…
-
https://github.com/nomic-ai/llama.cpp
GPT4All runs Mistral and Mixtral q4 models over 10x faster on my 6600M GPU