-
### Describe the bug
Crash with abort when trying to use AMD graphics card in editor
Model is mistral-7b-instruct-v0.2.Q4_K_M.gguf
ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon RX…
-
It seems that bumblebee is not capable of loading Mixtral-8x7B models (base or instruct). I've checked the files and it should be able to load the model (in theory) since it is capable of loading Mist…
-
Hi,
I'm trying to process large context sizes with lmql serve-model. Unfortunately, I'm running into "CUDA out of memory" issues. Using 1, 2, or 3 A100 did not make a difference. Reading the docume…
-
### Error Description
I am encountering the error, `Native API returns: -30 (PI_ERROR_INVALID_VALUE)`, when trying to run llama.cpp with the latest IPEX-LLM, following the official quickstart guide o…
-
Hi there,
I noticed another critical bug (at least from mine point of view): after LoRa training and even with `do_sample` is False, consecutive inference results in different results:
Loading t…
-
I am working with `onnxruntime-genai-directml 0.2.0rc3` and Mistral-7B for LLM inference. The code in [`model-qa.py`](https://github.com/microsoft/onnxruntime-genai/blob/514493be37eebbfcdf387c3d83ac71…
-
This is a very interesting work, an important attacking baseline for our future work on building robust defense.
For the models:
/ceph/ssd/staff/schwinn/models/Mistral-7B-Instruct-RR, and also the …
-
Hi,
We're running a project in the CivicLab in Grenoble, France. The project is based on LLM and is building as opensource project.
We're trying to build, test, prototype and evaluate an assistant…
-
# URL
- https://arxiv.org/abs/2404.05405
# Affiliations
- Zeyuan Allen-Zhu, N/A
- Yuanzhi Li, N/A
# Abstract
- Scaling laws describe the relationship between the size of language modelsand their…
-
Hey, I am a game developer and just wanted to ask if I can help on this game? I am currently making my own VR game, and I think we could sort of use the best of both worlds to create one magnificent g…