-
# Scenario
I used the following script to archive the model in .mar to use torchseve:
```
python tools/deployment/mmcls2torchserve.py \
../mdl-py-classification/model/mobile/mobilenet_v2_8xb32_…
-
### 🐛 Describe the bug
I followed the [HuggingFace example](https://github.com/pytorch/serve/tree/master/examples/Huggingface_Transformers) to deploy the GPT2 model, but a CORS error occurred when …
-
Hello,
If you look at this page, torchserve provides System metrics.
(https://pytorch.org/serve/metrics.html)
However, when torchserve is included in prometheus and looked up, System metrics are …
-
### 📚 The doc issue
can't serve the new model i want to serve with torchserve --start without the --ncs flag
What is it's purpose and what is it doing?
What exactly is the benefit of having it?
…
-
/kind bug
**What steps did you take and what happened:**
I'm using the `kserve.ModelServer` class to host dockerised service for a custom model subclassed from `kserve.Model`, as per the docs for …
-
### 🐛 Describe the bug
I am using 2 GPU.
Torchserve inference returns correct values only for predictions run on cuda:0.
### Error logs
```
x = "text to embed"
url = f"http://localhost:9080/pr…
-
### System Info
```
- `transformers` version: 4.22.0.dev0
- Platform: Linux-5.4.0-1072-aws-x86_64-with-debian-buster-sid
- Python version: 3.7.10
- Huggingface_hub version: 0.8.1
- PyTorch versi…
-
Hi,
I'm currently hosting a TorchServe server on ECS with inferentia instances.
I have a lot of models, but they are never running at the same time. Is there a way not to block one NeuronCore per …
-
I have read configurations here https://pytorch.org/serve/configuration.html
I am wondering do `number_of_netty_threads` and `netty_client_threads` have anything to do with performance (throughput)? …
-
### 🚀 The feature
As far as I know, there are no examples or documentation on serving Speech2Text models from Huggingface, such as Wav2Vec2. How could I enable serving with Wav2Vec2 Huggingface pre-t…