-
Hi!
It's great to be able to work with some API like OpenAI, but some people, like student, doesn't necessarily have money to spend in these services. Mistral is one of the rare firms that offers, …
-
To test: send too many requests, see how it fails, and add retry logic.
Look at the together SUT adapter for existing retry logic.
-
ver. 1.75.2
```
User tag:\n[INST]
Assistant tag:[/INST]\n
```
Maybe the right thing to do
`User tag:\n\n[INST]`
Then the system prompt will have the form
```
[INST]User's Instructions[/…
-
Unsloth: Merging 4bit and LoRA weights to 16bit...
Unsloth: Will use up to 23.73 out of 50.99 RAM for saving.
100%|██████████| 32/32 [00:19 4 if True: model.push_to_hub_gguf("mINE", tokenizer, quant…
-
https://mistral.ai/news/codestral-mamba/
The latest model from Mistral utilizes the Mamba architecture (vs. Transformers) and targets code generation with strong performance on the leaderboards.
-
Hi,
I want to ask can I apply all of quantization method to mistral following the example of llama?
-
### Python -VV
```shell
(mistral) C:\Work\2024-10-04_mistral>python -VV
Python 3.12.6 (tags/v3.12.6:a4a2d2b, Sep 6 2024, 20:11:23) [MSC v.1940 64 bit (AMD64)]
```
### Pip Freeze
```shel…
-
Fails for both `Mistral-7B-Instruct-v0.2` and `intfloat/e5-mistral-7b-instruct `
Only fails with `tp_degree=1`, good for `2
-
mistral ai is powerful and cheap also it's open source , and noting the mission of agent-zero it would be pretty cool to have.
-
Hi there,
Thank you for the benchmark.
I have tried to this repo and get the inference of llava_hf model type and the checkpoint of llava-hf/llava-v1.6-mistral-7b-hf on infovqa, however, the inf…