-
From req doc: 2d
P0: Users must be able to serve models from Hugging Face without having to do any additional conversions or configurations
Acceptance criteria:
- [ ] Test TGIS image with HF model
-…
-
### Model description
This model was released by Mistral [here](https://mistral.ai/news/mistral-nemo/), and is available on HuggingFace [here](https://huggingface.co/mistralai/Mistral-Nemo-Base-2407)…
-
### Motivation.
Currently, the OpenAI API server and AsyncLLMEngine share the same asyncio event loop. This means that the API server and the CPU components of the AsyncLLMEngine contend for the same…
-
This link contains heml chart with image anyscale/aviary:latest-tgi
This image is out-of-date. Even yaml shemas for models are wrong.
https://ray-project.github.io/aviary/kuberay/deploy-on-gke/
-
I tried adding a TOC / Video Timeline to the YouTube video but unable to.
Maybe @mauilion could post the stuff below for others to benefit
TOC for TGIK 077
```
00:53 - Welcome
03:37 -…
-
https://github.com/huggingface/text-generation-inference
Main features of TGI are quite awesome. It woud be nice to make it additional inference implementation.
-
### Priority
Undecided
### OS type
Ubuntu
### Hardware type
Xeon-SPR
### Installation method
- [X] Pull docker images from hub.docker.com
- [ ] Build docker images from source
### Deploy metho…
-
## Describe the bug
There is a synchronization issue at the launch of the Pod with the current images:
* the containers get all `Ready`:
```
flan-t5-small-gpu-predictor-00001-deployment-6768c5…
-
### Checked other resources
- [X] I added a very descriptive title to this issue.
- [X] I searched the LangChain documentation with the integrated search.
- [X] I used the GitHub search to find a…
-
While deploying VisualQnA on Gaudi from /GenAIExamples/VisualQnA/docker/gaudi, the script fails because opea/lvm-tgi:latest Image is not avaiable on docker repository to be pulled.
```
docker buil…