-
- [x] This is actually a bug report.
- [ ] I am not getting good LLM Results
- [ ] I have tried asking for help in the community on discord or discussions and have not received a response.
- [ ] I …
-
### Python -VV
```shell
Python 3.12.4 | packaged by Anaconda, Inc. | (main, Jun 18 2024, 15:12:24) [GCC 11.2.0]
```
### Pip Freeze
```shell
annotated-types==0.7.0
anyio==4.4.0
attrs==24.2.0
cer…
-
### Your current environment
docker pull vllm/vllm-openai:v0.6.2
### Model Input Dumps
docker run --runtime nvidia --gpus all \
-v ~/.cache/huggingface:/root/.cache/huggingface \
--env "H…
-
-
The current `HuggingFaceInferenceSUT` uses the `chat_completion` API. WS3 pointed out that not all models are accessible via this API (e.g. mistralai/Mistral-Nemo-Instruct-2407).
-
Hi,
I am trying to prune Mistral 7B (https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.2) and while I was able to successfully run the commands for magnitude pruning, I was facing issues with…
-
We are planning to introduce a new Ballerina connector for the latest Mistral REST API, generated using it's OpenAPI specification.
Related links:
- https://docs.mistral.ai/api/
- https://github.com/…
-
mistralai/Codestral-22B-v0.1 support
win4r updated
4 months ago
-
Mistral AI just dropped Pixtral, their 12b model with vision support.
- https://github.com/mistralai/mistral-common/releases/tag/v1.4.0
- https://www.reddit.com/r/LocalLLaMA/comments/1fe3x1z/mistr…
-
Create an API service that can be called to process the requests from the app.
We can then host this into a server.
The API shall accept the role and the token.
Instructions for deploying the …