This PR adds functionality for returning topk most relevant (with the highest scores) item ids from Triton IS, for NextItemPrediction task.
Current blocker:
The code designed to return top_k item ids (int 64 dtype), but model.output_schema returns next_item as float32 dtype, which creates an error from Triton.
Shall we change the code base in a way that model.output_schema matches with the expected output and output dtype from Triton? Or shall we return top_k item id scores, instead of item_ids?
Status update:
After modifying the model.output_schema, we can now return two outputs (item_scores, item_ids) from Triton.
Remaining tasks:
[x] be sure the dtype of categorical item-id in the model.output schema matches with the model.input_schema
This PR adds functionality for returning topk most relevant (with the highest scores) item ids from Triton IS, for NextItemPrediction task.
Current blocker:
The code designed to return top_k item ids (int 64 dtype), but model.output_schema returnsnext_item
as float32 dtype, which creates an error from Triton.Shall we change the code base in a way that model.output_schema matches with the expected output and output dtype from Triton? Or shall we return top_k item id scores, instead of item_ids?
Status update:
After modifying the model.output_schema, we can now return two outputs (item_scores, item_ids) from Triton.
Remaining tasks: