search

triton-inference-server / pytriton

PyTriton is a Flask/FastAPI-like interface that simplifies Triton's deployment in Python environments.
https://triton-inference-server.github.io/pytriton/
Apache License 2.0
687 stars 45 forks source link

OUTPUT triton: list or tuple or any kind of Iterables #35

Closed dogky123 closed 9 months ago

dogky123 commented 10 months ago

Hello, i just want to ask if I can config triton return a list/tuple... instead of Tensor ? Please give me some docs if it is possible.

jkosek commented 9 months ago

The Triton API is designed to receive/return tensors. At the moment is not possible to return lists or tuples. The option we can suggest is to implement conversion of inputs/outputs between list/tuple and tensors inside the inference function. Would that work?

dogky123 commented 9 months ago

The Triton API is designed to receive/return tensors. At the moment is not possible to return lists or tuples. The option we can suggest is to implement conversion of inputs/outputs between list/tuple and tensors inside the inference function. Would that work?

It would take more cost to do

github-actions[bot] commented 9 months ago

This issue is stale because it has been open 21 days with no activity. Remove stale label or comment or this will be closed in 7 days.

github-actions[bot] commented 9 months ago

This issue was closed because it has been stalled for 7 days with no activity.