-
### Question
Does anyone have carried out the pretraining with Mixtral 8×7B? When I run the petraining script, one problem occured like the figure shown below. I just add a llava_mixtral.py to the ll…
-
```dockerfile
#Base Image
FROM nvcr.io/nvidia/tritonserver:24.04-trtllm-python-py3
USER root
RUN apt update && apt install --no-install-recommends rapidjson-dev python-is-python3 git-lfs curl uuid…
-
`/sbin` is not in the system path for certain GCP VMs. This is causing the vLLM + Mixtral example need to update the system path: https://github.com/skypilot-org/skypilot/blob/f4541059718a446917707930…
-
Hi folks, we are enabling slack notifications for regression tests. Here is a list I have of each t3000 test owner for each of the models. Please let me know if something is incorrect or should be rea…
-
Hi, thank you for the wonderful ollama project and the amazing community!
I am testing the Mixtral 3Bit Quantized model under a RTX400 with 20GB of VRAM. The model is 20GB of size and as you ca…
-
I found the mixtral example in this repo, and try to run it on A100 80GB, but the default Mixtral-8x7B-v0.1 runs out of memory.
I was curious what GPU can run it in one card?
-
### Model ID
NorwAI/NorwAI-Mixtral-8x7B-instruct
### Model type
Decoder model (e.g., GPT)
### Model languages
- [X] Danish
- [X] Swedish
- [X] Norwegian (Bokmål or Nynorsk)
- [ ] Icelandic
- [ ] …
-
Below is an example where we parsed the Updated File out of the LLM results, yet the contents of the file look incomplete, see how this stops early.
Using
```
provider = "IBMOpenSource"
args = …
-
### Your current environment
```
Collecting environment information...
PyTorch version: 2.1.2+cu121
Is debug build: False
CUDA used to build PyTorch: 12.1
ROCM used to build PyTorch: N/A
OS: …
-
Hi,
I am using AWS Sagemaker instance ml.g5.48xlarge which has 8 A10 Nvidia GPUs. I have 4 scripts each accessing 2 GPUs each. I am using vLLM to load mixtral LLM onto the respective GPUs such as f…