search

allegroai / clearml-serving

ClearML - Model-Serving Orchestration and Repository Solution
https://clear.ml
Apache License 2.0
137 stars 40 forks source link

String not supported for Triotn #54

Open amirhmk opened 1 year ago

amirhmk commented 1 year ago

Have been working on model ensemble, continuing conversations from https://github.com/allegroai/clearml-serving/issues/53, just thought it may be a better idea to create new issues for different things that I find along the way. Essentially we want the output of the model be a S3 path where all the results are saved to as a JSON.

However it doesn't seem like clearml-serving is mapping object datatype properly? Triton does support strings..

The issue lies here I believe: np_to_triton_dtype. This currently maps an object to TYPE_BYTES to be written to the config.pbtxt file (which is not a valid type as per link above), whereas it should be TYPE_STRING.

ainoam commented 1 year ago

Always a good idea to separate issues @amirhmk, Thanks!

np_to_triton_dtype actually comes from triton itself.

Would you like to offer them a fix? It should be something like

    elif np_dtype == str:
        return "STRING"