-
### Describe the issue
I have an OCR model with the following architecture ResNet-BiLSTM-CTC
OS environment:
+ cuda:11.6.2
+ python 3.7
+ onnxruntime-gpu==1.14.1
+ torch 1.10.0 cpu
cuda_pro…
-
**Link to the notebook**
In the code below I am clearly passing a different instance type where I want to deploy my trained moedl
```
finetuned_predictor = estimator.deploy(
instance_type='ml.…
-
[TF Lite Micro (link - supported platforms)](https://www.tensorflow.org/lite/microcontrollers#supported_platforms) makes local node ML inferencing possible, enabling powerful example applications like…
-
### System Info
I am currently mostly working with the ghcr.io/huggingface/text-embeddings-inference:cpu-1.2 Docker image on MacOS. Currently, I am only trying to find out which reranker models with …
-
### Elasticsearch Version
8.13
### Installed Plugins
_No response_
### Java Version
_bundled_
### OS Version
any
### Problem Description
Creating an inference endpoint with an id starting wit…
-
### Description
Improve validation and exception handling within the inference API.
Here are a few areas to get started
- When a text embedding service is created, during the creation process w…
-
**Describe the bug**
When running as a non-root user within a container, sagemaker-inference fails to start the multi-model-server. This works when all packages are installed as root, and the entry…
-
Will there be TEE based training service for training ML model over private features & non private features? It logically looks like an aggregation service that acts on reports to generate a summary r…
-
Notice: In order to resolve issues more efficiently, please raise issue following the template.
(注意:为了更加高效率解决您遇到的问题,请按照模板提问,补充细节)
## 🐛 Bug
### To Reproduce
Steps to reproduce the behavio…
-
### System Info
GPU Nvidia A10G
Cuda version 12.3
Driver version 535.183.01
TensorRT-LLM v0.8.0
Image nvcr.io/nvidia/tritonserver:24.02-trtllm-python-py3 (was used to build the tensorrt engine an…