search

triton-inference-server / server

The Triton Inference Server provides an optimized cloud and edge inferencing solution.
https://docs.nvidia.com/deeplearning/triton-inference-server/user-guide/docs/index.html
BSD 3-Clause "New" or "Revised" License
8.07k stars 1.45k forks source link

Incorrect data received on python backend from client. #7510

Open Chappie74 opened 1 month ago

Chappie74 commented 1 month ago

Description I am testing sending data received as output from one model as input to my python backend to post process (I will eventually do an ensemble later)

The problem I am having is that the data sent in, is not the data received. Below I've linked two images of me logging this data. The shape of the received data is correct; however, it seems that all the values are a repeat of -6.334826e-05 Any idea why this might be the case?

data sent image

data received image

Triton Information 24.07

Are you using the Triton container or did you build it yourself? Container FROM nvcr.io/nvidia/tritonserver:24.07-py3

To Reproduce the shape of the data is (1, 68, 64, 64) (on both ends)

Everything is correct on the client's end. 

inputs = tc.InferInput("INPUT_0", [1, 68, 64, 64], datatype="FP32")
inputs.set_data_from_numpy(results.as_numpy('output0'), binary_data=False) # tried both binary =False/True; output0 is from another model (printed data in first image)
results = client.infer(model_name="landmark_postprocess", inputs=[inputs], ) 
input0 = pb_utils.get_input_tensor_by_name(request, 'INPUT_0').as_numpy()
logger.log_info(str(input0))

Expected behavior I expect the data sent to be the data received.

Chappie74 commented 1 month ago

So, I left this off for about a day or two. Restarted my pc a couple times since then. Testing things back again today, I've noticed that the correct data is being passed now (without making any modification) Not sure what the issue might have been.