-
Hi there,
I just installed ollama 0.1.27 and tried to run gemma:2b but it suggest CUDA out of memory error. Could you please investigate and figure out root cause?
I'm using CPU `i7-4700HQ` wi…
-
### System Info
```bash
Command: text-generation-launcher --model-id Phind/Phind-CodeLlama-34B-v2 --max-input-length 3072 --max-total-tokens 6144 --port 8080
Target: x86_64-unknown-linux-gnu (Ubu…
-
### What happened + What you expected to happen
if i run fine tuning without ray, it works.
but if i use ray, i will get
2024-02-20 21:43:02,050 ERROR tune_controller.py:1374 -- Trial task failed f…
-
`sky launch --disk-tier none -c test-mixtral --cloud azure llm/mixtral/serve.yaml`
```
I 12-27 11:40:38 log_lib.py:431] Start streaming logs for job 1.
INFO: Tip: use Ctrl-C to exit log streamin…
-
My code is working to extract data from a PDF with ChatGPT4 turbo, but when I try to change it to mistralai/Mixtral-8x7B-Instruct-v0.1 model from AnyScale It always includes text apologizing on the re…
alew3 updated
11 months ago
-
The code below throws an assertion error:
```
from transformers import AutoModelForCausalLM, AutoTokenizer, pipeline
#model_name_or_path = "TheBloke/Mixtral-8x7B-Instruct-v0.1-GPTQ"
model_name…
-
As a person who loves the llama index Python library, LlamaIndex.ts is a piece of shit if someone doesn't want to use OpenAI LLMs.
It will force you to use OpenAI LLMS (I broke my MacBook Screen a…
-
### System Info
ghcr.io/predibase/lorax:8ff0bf5
### Information
- [ ] Docker
- [ ] The CLI directly
### Tasks
- [ ] An officially supported command
- [ ] My own modifications
### Reproduction
r…
-
Hello,
I get this error when I try to load the mistralai/Mixtral-8X7B-Instruct-v0.1 model into the latest container with 2 A100s... Is it related to the hugginface key? I've run out of ideas!
/…
-
### Description
I'm using AI SDK to use together AI API using OpenAI SDK which supports this, I tried this in Node.js and it works, However, when I try to do that in Next.js using Vercel AI SDK it's …