search

triton-inference-server / server

The Triton Inference Server provides an optimized cloud and edge inferencing solution.
https://docs.nvidia.com/deeplearning/triton-inference-server/user-guide/docs/index.html
BSD 3-Clause "New" or "Revised" License
8.27k stars 1.47k forks source link

python_backend dlpack: Bool type is not supported #5820

Closed cceyda closed 1 year ago

cceyda commented 1 year ago

Description Using python_backend I get "RuntimeError: Bool type is not supported by dlpack" with the below code. It is true that logits is of type bool. But dlpack seems to support bool at least on newer versions according to this: https://github.com/dmlc/dlpack/issues/75 & https://github.com/dmlc/dlpack/pull/114

Triton Information official ngc 23.04 container

To Reproduce

logits = pb_utils.get_output_tensor_by_name(inference_response, 'logits')
logits = logits.to_dlpack()

Expected behavior should work

oandreeva-nv commented 1 year ago

Bool type will be supported by dlpack in python backend starting with 23.05 release.