-
When trying to run the talk-llama example code with OpenCL enabled using a NVIDIA GeForce GT 755M, I get the following crash:
```
% LC_ALL=C ./obj-x86_64-linux-gnu/bin/talk-llama -mw ../nb-large-g…
-
Hi,
Some of the mistakes in the `TEMPLATE` definitions for the models you can download from https://ollama.ai are hurting the models to varying degrees. I only found this by accident when experimen…
-
## 🐛 Bug
With newest version of Docker image (tested on 2024-04-22 ) training with thunder.jit with additional inductor executor gives OOM error.
### To Reproduce
Before each testing each com…
-
### What behavior of the library made you think about the improvement?
Generation cannot be interleaved with function calls and return values.
### How would you like it to behave?
We should a…
lapp0 updated
6 months ago
-
Hi, thank you for the amazing model! Super excited to test it out!
I am trying to load it into my GeForce RTX 3090 (24 GB vRAM), which I believe to be more than enough for inference with 8 or 4 bit…
-
i use this guide for develop a RAG system: https://github.com/marklysze/LlamaIndex-RAG-WSL-CUDA/blob/master/LlamaIndex_Mixtral_8x7B-RAG.ipynb
i use the following code for use my local llm:
```
…
-
Hi Dustin,
thanks for your great work.
I was trying to run [mistral-7b](https://github.com/mistralai/mistral-src) on Jetson ORIN with Jetpack (# R35 (release), REVISION: 4.1, GCID: 33958178, B…
cj401 updated
7 months ago
-
I have been using unsloth to train a chatbot model using the mistral 7b instruct model. I have successfully fine-tuned it and got inference results accordingly.
But the only issue I am having is t…
-
## What
Local models (e.g. LLAMA based models available via HuggingFace in the 7B or 13B size classes) offer multiple benefits (e.g., can be finetuned/adapted, run locally etc).
While LIDA has been…
-
It seems that bumblebee is not capable of loading Mixtral-8x7B models (base or instruct). I've checked the files and it should be able to load the model (in theory) since it is capable of loading Mist…