nv-morpheus / Morpheus

Morpheus SDK
Apache License 2.0
309 stars 119 forks source link

[FEA]: C++ impl of Triton stage should cast input tensors to row-major if needed #1688

Open dagardner-nv opened 2 months ago

dagardner-nv commented 2 months ago

Is this a new feature, an improvement, or a change to existing functionality?

Improvement

How would you describe the priority of this feature request

Medium

Please provide a clear description of problem this feature solves

Triton input tensors mush all be row-major, per kserve.

Currently the Python impl of the Triton stage performs this conversion implicitly as it makes use of tritonclient.InferInput.set_data_from_numpy to stage the data which ultimately calls numpy.ndarray.tobytes which defaults to row-major.

Describe your ideal solution

We should check for column-major input tensors and convert them to row-major

Additional context

No response

Code of Conduct