-
### 🚀 The feature
TorchServe supports streaming response for both HTTP and GRPC endpoint.
- [ ] #2186
- [ ] #2232
### Motivation, pitch
Usually the predication latency is high (eg. 5sec…
-
There is a noticeable performance difference when running StandAlone Torchserve and Kserve with TorchServe.
The difference in performance is 8:1 where Standalone Torchserve with just Kserve V2 Serv…
-
Current way of working with `padl_ext`
```
from padl_ext.torchserve.prepare import prepare
```
However, a potential problem with this would be if we use the same function names such as prepare…
-
### 🚀 Descirbe the improvement or the new tutorial
We could use an updated tutorial/walk through example on how to use TorchServe. The closest thing we have is the TorchServe Getting Started page loc…
-
I am using TorchServe to potentially serve a model from MMOCR (https://github.com/open-mmlab/mmocr), and I have several questions:
1. I tried to do inference on hundreds of images together using batc…
-
### Prerequisite
- [X] I have searched [Issues](https://github.com/open-mmlab/mmrotate/issues) and [Discussions](https://github.com/open-mmlab/mmrotate/discussions) but cannot get the expected help…
-
### 🚀 The feature
Update Jan 31 2024 - we should speed this up
PyTorch on MacOS x86 is being deprecated https://github.com/pytorch/pytorch/issues/114602
Officially support torchserve on M1-M2 m…
-
I am considering using [Torchserve](https://pytorch.org/serve/) to query an AlphaPose model. However, converting AlphaPose to TorchScript appears non-trivial.
I am curious if anyone has tried to d…
-
**What would you like to be added**:
Right now, llmaz is mostly designed for large language models, however, some users may need to support traditional models as a singleton solution, let's wai…
-
I have a yaml-file:
```
apiVersion: "serving.kubeflow.org/v1beta1"
kind: "InferenceService"
metadata:
name: "my-service"
spec:
predictor:
containers:
- image: pytorch/torchserve-k…