-
### System Info
Tech stack: tgi 2.0.1, A100 GPU 80GB running on Kubernetes.
Model: Mixtral 8x7B-instruct-v0.1
### Information
- [x] Docker
- [ ] The CLI directly
### Tasks
- [x] An officially su…
-
### Question Validation
- [X] I have searched both the documentation and discord for an answer.
### Question
I need some help with an issue I'm facing. I was using a set of packages on my local mac…
-
### Checked other resources
- [X] I added a very descriptive title to this issue.
- [X] I searched the LangChain documentation with the integrated search.
- [X] I used the GitHub search to find a…
-
### Description
The server-action example on AI SDK docs gives me an error Warning: Only plain objects can be passed to Client Components from Server Components
### Code example
```
"use s…
-
Certain models throw an error when a provider is specified directly, despite functioning correctly when the same model is executed randomly via the `RetryProvider`.
To reproduce the issue, consider…
-
The tutorial for running mixtral on VLLM doesn't work since the model cannot be downloaded without a huggingface access token. This is because mixtral is now a gated model: https://huggingface.co/mist…
-
Hi I am running a fresh install of H2O-GPT:
Running script is: python generate.py --inference_server="vllm:0.0.0.0:5001" --guest_name="" --enable_tts=False --enable_stt=False --base_model=mistral…
-
Found a small vulnerability where there are currently no rate limits applied to AI api endpoints.
Proposed change in file: `apps/dashboard/src/actions/ai/chat/index.tsx`
```
"use server";
im…
-
### Checked other resources
- [X] I added a very descriptive title to this issue.
- [X] I searched the LangChain.js documentation with the integrated search.
- [X] I used the GitHub search to fin…
-
When I used vllm to serve my local model, the terminal displayed the following message:
ImportError: libcudart.so.11.0: cannot open shared object file: No such file or directory
The traceback pointe…