-
For example, if the deployment infrastructure is a Kubernetes cluster and the user has requested the use of GPUs or special purpose accelerators that do not exist, promptly return failure message.
Oc…
-
### Feature request
1. [Model] chinese-alpaca-2-7b
2. [Hardware] Gaudi2C
3. [Method] LoRA and FineTuning
4. [Related codes] examples/language_modeling
5. [Test Cmdlines]:
- **LoRA**:
```bas…
-
### Description
@peytondmurray, @akshay-anyscale @angelinalg and I had a discussion on how better to organize the Ray Serve examples page.
Instead of Beginner, Intermediate, Advanced in the curr…
-
-
### System Info
Compared with **v1.2.2-release** tgi-gaudi, sending the query to the **v2.0.0-release** tgi-server will hit the input_token_length + output_token_length assertion.
Especially, when…
-
### System Info
```shell
docker image:
vault.habana.ai/gaudi-docker/1.15.1/ubuntu22.04/habanalabs/pytorch-installer-2.2.0:latest
oh:
https://github.com/huggingface/optimum-habana/tree/bb6be54d11…
-
I'm trying to get the ChatQnA Gaudi Example to work and I'm running into a few issues.
**First**, in the `docker_compose.yaml` file, both the `tei_embedding_service` and the `tgi_service` have th…
-
### System Info
```shell
Optimum habana main branch, Synapse version 1.15, docker image 1.15.0:474 (RC)
```
### Information
- [X] The official example scripts
- [ ] My own modified scripts
### Ta…
-
at @onefact we have been using wasm, but this won't work for the encoder-only or encoder-decoder models i've built (e.g. http://arxiv.org/abs/1904.05342). that's because the wasm vm is for the cpu (ha…
-
Currently the base image is from,
```sh
vault.habana.ai/gaudi-docker/1.14.0/ubuntu22.04/habanalabs/pytorch-installer-2.1.1:latest
```
to support the new model Mixtral-8x7B and other variants there…