ValidateBytesInputs() check failed in Big Endian Machines

[Hemaprasannakc]

Description

I am leveraging Triton Inference Server r24.06 on Big Indian Machine. While sending the bytes data for inferencing I am getting the below error response though the number of elements given for inference is right. Note that the same inference request is working fine with r23.12 which I was consuming previously. Upon checking the difference between r23.12 vs r24.06 I see there are new validations introduced in infer_request.cc here.

I suspect this is something similar to byte order issue as mentioned here so the validation is getting failed.

Response: b’{“error”:“[request id: <id_unknown>] expected 35 string elements for inference input \‘IN0\‘, got 2"}’
Response status: 400

Triton Information r24.06

Are you using the Triton container or did you build it yourself? Built Triton Docker file for Big endian Machine .

To Reproduce Steps to reproduce the behavior.

• Step up Triton server with python backend on big endian machine.
• prepare the sample input bytes data as described here.
• Send the inferencing via CURL API.

Expected behavior Should return the output prediction without error.

It would be greatly beneficial if any comprehensive approach can be placed to address endianness challenges. Any insights or strategies to tackle these issues effectively would be highly valued.

[Hemaprasannakc]

@nnshah1 Could u please help me with above concern?

[nnshah1]

@Hemaprasannakc sorry for the delay - will need to investigate. Currently unfortunately while the support for bigendian was designed and implemented - we weren't able to merge because we didn't have an automated ci set up. since then higher priorities and several large changes require a heavy rebase for full big endian support - we don't have a timeline for that.

@harryskim for viz and guidance for overall BigEndian support.