-
### Self Checks
- [X] This is only for bug report, if you would like to ask a question, please head to [Discussions](https://github.com/langgenius/dify/discussions/categories/general).
- [X] I have s…
-
Addon model-serving uses Seldon to serving models, and Seldon Core currently only supports the V1 Ambassador APIs: https://docs.seldon.io/projects/seldon-core/en/latest/ingress/ambassador.html
The …
-
/kind feature
**Describe the solution you'd like**
[A clear and concise description of what you want to happen.]
Multi-model serving e2e test only test downloading from gcs. We should update this…
-
Hi,
I have a onnx model that I am trying to serve with sleepsonthefloor/graphpipe-onnx:cpu docker image.
I am running it with this command:
`docker run -it -v "$PWD/models:/models/" -p 9000:9000 …
-
- --gpus should be a top level option when serving a model since you can start a server separately from most cmds.
- vllm-args should be exposed when running the serve command
- passing additonal a…
-
## Description
vLLM sampling parameters include a [richer set of values](https://github.com/vllm-project/vllm/blob/c9b45adeeb0e5b2f597d1687e0b8f24167602395/vllm/sampling_params.py), among which `lo…
-
ONNX export (e.g. with https://onnx.ai/sklearn-onnx/ ) would be very beneficial for deploying trained models to any environment and programming language. Do you have such export options considering ON…
-
With fairing we'd like to be able to predict locally within the notebook.
For example
```
model=Model()
model.predict(...)
```
We need to figure out the exact syntax and model signatures.
…
jlewi updated
4 years ago
-
Could you change your TensorFlow code samples as suggested in this Stack Overflow answer?
https://stackoverflow.com/questions/49660389/tensorflow-serving-ml-engine-online-prediction-json-file-forma…
-
Getting an error while calling model serving endpoint from the UI
Unrecognized content type parameters: format. IMPORTANT: The MLflow Model scoring protocol has changed in MLflow version 2.0. If yo…