-
- Serving multiple ML models using mlflow in a single VM [stackoverflow](https://stackoverflow.com/questions/70620074/serving-multiple-ml-models-using-mlflow-in-a-single-vm)
- How to get stable resul…
-
https://arxiv.org/abs/2109.11067
利用 A100 MIG 进行模型 serving 的探索
字节 MLSys 组
-
1) Ubuntu 16.04
2) Docker tensorflow/tensorflow 1.13.1 and tensorflow/serving:latest-gpu
3) NVIDIA TensorRT 5.0.2 (https://docs.nvidia.com/deeplearning/sdk/tensorrt-install-guide/index.html)
4) Ten…
-
Hello,
Firstly, thank you for this repo. When I try to connect the Ollama node to the Mistral7B model which is locally served by Ollama serve. I am getting this error again and again. Is using the …
-
API URL: https://biothings.ncats.io/multiomics_clinicaltrials_kp
Github URL: https://github.com/multiomicsKP/clinical_trials_kp
Git branch/commit: https://github.com/multiomicsKP/clinical_trials_kp/…
-
Is it possible to have spark serving in kubernetes
My Requirement:
1. Deploy sparkml lib model
2. Get distributed prediction
[AB#1153733](https://msdata.visualstudio.com/b9b2accc-2d1c-45b3-9d2…
-
While serving code_llama model and requesting `/generate_stream` with `stream: true`, `text_output` field in response does not contain any spaces (`" "`). Is this the expected behavior (i.e. users hav…
-
I discovered a performance issue that Tensorflow Serving has an unexplainable and significant network delay for tail latencies when facing higher loads of traffic.
My setup was a client and a Tenso…
-
**Is your feature request related to a problem? Please describe.**
We can create a `sagemaker.tensorflow.model.TensorFlowModel` for new versions of TensorFlow (e.g. 2.0, 2.1), but get a "container no…
-
> Inference model server implementation with gRPC interface, compatible with TensorFlow serving API and OpenVINO™ as the execution backend.
# Reference
- [ ] [Github repo](https://github.com/Int…