AttributeError: 'InferenceServerClient' object has no attribute 'InferInput'

[Galoev]

I have a Triton Server that runs in Docker. There I initialized the CLIP model. I wrote some simple code to try infer and get the output of this model. But I get the error AttributeError: 'InferenceServerClient' object has no attribute 'InferInput'

Here is my client code:

from transformers import CLIPProcessor
from PIL import Image
import tritonclient.http as httpclient

if __name__ == "__main__":

    triton_client = httpclient.InferenceServerClient(url="localhost:8003")

    # Example of tracing an image processing:
    processor = CLIPProcessor.from_pretrained("openai/clip-vit-base-patch32")
    image = Image.open("9997103.jpg").convert('RGB')

    inputs = processor(images=image, return_tensors="pt")['pixel_values']

    inputs = []
    outputs = []

    inputs.append(triton_client.InferInput("input__0", image.shape, "TYPE_FP32"))
    inputs[0].set_data_from_numpy(image)

    outputs.append(triton_client.InferRequestedOutput("output__0", binary_data=False))

    results = triton_client.infer(
        model_name='clip',
        inputs=inputs,
        outputs=outputs,
    )

    print(results.as_numpy("output__0"))

Here's the error I'm getting

Traceback (most recent call last):
  File "main.py", line 18, in <module>
    inputs.append(triton_client.InferInput("input__0", image.shape, "TYPE_FP32"))
AttributeError: 'InferenceServerClient' object has no attribute 'InferInput'

Help me please

[nnshah1]

I ran into something similar while creating our stable diffusion tutorial (which might be of interest to you as well):

InferInput will be a method of httpclient and not triton_client in your code above. Slightly confusing. Probably a slightly better name for httpclient is httputils.

https://github.com/triton-inference-server/tutorials/blob/a5252cd02e20700815baf08028c040b50983f214/Popular_Models_Guide/StableDiffusion/client.py#L39

[nnshah1]

In case you are trying the tutorial - please use this branch:

https://github.com/triton-inference-server/tutorials/tree/nnshah1-meetup-04-2024

There are a few updates that haven't hit main yet that cause issues (mainly around versioning)

[Galoev]

I ran into something similar while creating our stable diffusion tutorial (which might be of interest to you as well):

InferInput will be a method of httpclient and not triton_client in your code above. Slightly confusing. Probably a slightly better name for httpclient is httputils.

https://github.com/triton-inference-server/tutorials/blob/a5252cd02e20700815baf08028c040b50983f214/Popular_Models_Guide/StableDiffusion/client.py#L39

Thanks a lot!