-
Kserve v0.9.0 release fixes log4j security vulnerability issue: https://github.com/kserve/kserve/releases/tag/v0.9.0
torchserve-kfs 0.6.0 docker image currently uses kserve v0.8.0
Can we please upg…
-
Frameworks: Optimizations to reduce memory used by models.
-
### Describe the feature
Currently boto3 sagemaker sdk supports requests only to main torchserve inference api via [invoke_endpoint](https://boto3.amazonaws.com/v1/documentation/api/latest/referenc…
blatr updated
2 years ago
-
Just realized about this assert in `BaseSegmentor`:
https://github.com/open-mmlab/mmsegmentation/blob/master/mmseg/models/segmentors/base.py#L85
I am trying to figure out why is this needed ther…
-
### Bug description
## Issue Summary:
A segmentation fault (core dumped) error occurs when importing `lightning` module.
### File: test.py
```python
import torch
import lightning as L
from …
-
[TorchServe](https://github.com/pytorch/serve) will soon support the KServe v2 inference protocol and can support additional types of PyTorch models that Triton does not currently support.
Create …
-
### 🐛 Describe the bug
Dev install doesn't work currently on M1 Mac (MacOS 12.6) because of the version of grpc and protoc do not work with M1 Mac.
In `serve/frontend/gradle.properties`:
```
grp…
-
Hello,
I'm trying to follow the [tutorial notebook](https://github.com/aws-neuron/aws-neuron-sdk/blob/master/src/examples/pytorch/bert_tutorial/tutorial_pretrained_bert.ipynb) (compiling a HF model…
-
### Description
I'm running an ECS cluster on `c6i.xlarge` and `inf1.xlarge` instances.
I noticed there's often a huge difference on the memory shown by ECS agent and `htop` or `free -m `. For e…
-
First, I created docker container by followed https://github.com/pytorch/serve/tree/master/docker#create-torchserve-docker-image, I leaves all configs default except remove `--rm` in `docker run ...` …