-
I first want to mention that I do not want to support OpenAI or want to enable them in any way. That is particularly why I want Mistral to succeed. That being said, their API function has become somew…
-
Some models expose a JSON-mode API where they output correct JSON. There are some related APIs such as function calling (tool use). These are useful for programmatic use, as we need in an IDE like ema…
-
My understanding is that we have to build a fastAPI wrapper, and during intialized phase we call `client = mii.client("mistralai/Mistral-7B-v0.1")` and we implement a handler to call `client.generate`…
-
I'm trying to use the typescript client and I am getting the following error message:
```js
SDKValidationError: Response validation failed: [
{
"code": "invalid_type",
"expected": "bo…
-
### System Info
Using Docker server
```
model=mistralai/Mistral-7B-Instruct-v0.1
volume=$PWD/data
docker run --gpus '"device=3"' --shm-size 1g -p 8080:80 -v $volume:/data \
-e HUGGING_…
-
Hi there,
Are you considering adding an endpoint in your API in the future so that the client can fetch the total usage for the day? Because as you know, more and more, we're integrating LLMs into …
-
i am trying to run PrivateGPT for the first time. I have installed Llama and a service is running at the moment.
I have cloned the repo, install the poetry dependencies (`poetry install --extras "ui…
-
Having an LLM router implementation would be cool. Can take [RouteLLM](https://github.com/lm-sys/RouteLLM) implementation for example.
-
Could you include both the Hugging Face endpoint and the Hugging Cloud dedicated endpoint, please?
-
I am running my code on AWS Sagemaker notebooks and I have machine with 4 GPUs. Whenever I set the tensor_parallel_size>1 it shows me the following error.
NFO 12-13 13:07:31 llm_engine.py:72] Initi…