-
With this
```python
from exl2forall.model import Exl2ForCausalLM
# https://huggingface.co/turboderp/Mixtral-8x7B-instruct-exl2
model_name = 'turboderp/Mixtral-8x7B-instruct-exl2'
revision = '3.…
-
Hi,
I installed privateGPT following these steps:
```
git clone https://github.com/imartinez/privateGPT.git
cd privateGPT
conda create -n privategpt python=3.11
conda activate private…
-
- Allow user to specify multiple models to download when loading server
- Allow user to switch between models
- Allow user to load multiple models on the cluster (nice to have)
-
I am working with `onnxruntime-genai-directml 0.2.0rc3` and Mistral-7B for LLM inference. The code in [`model-qa.py`](https://github.com/microsoft/onnxruntime-genai/blob/514493be37eebbfcdf387c3d83ac71…
-
## 🐛 Bug
I'm facing issue in serving Mistral-7B-Instruct-v0.3 model via mlc_llm serve. I get the below error when performing model serve with "python -m mlc_llm compile ..." command
INFO engine_…
-
Any ideas on why this code doesn't work? I was trying to use sliding windows, but it seems like Mistral doesn't support that. So then I tried truncating, and setting a higher token limit - but that do…
-
**Description**
I encountered a `UnicodeEncodeError` when trying to write non-ASCII characters to a file. Below is a snippet of the code causing the issue:
``` bash
handle.write(
UnicodeEncodeEr…
-
I created the image using dockerfile.local running docker-compose.file. But, when I run the image, it cannot run, so I run it in interactive mode to view the problem. It cannot be initialized.
It …
-
Love it! Really!
But when trying to make it work it is strange because the error messages are too generic and doesn't explain much to help me solve it.
For example one is that I get the error me…
-
Hello,
before I start asking questions here, I would like to say thanks first. Among all AI projects related to this topic that I try, h2oGPT is always the fastest in terms of supporting new techn…