Print Intermediate Outputs for Each Step in a Chained Model Sequence

[renenxie]

/kind feature I would like to request a feature that allows intermediate outputs from each step in a sequence of models to be printed or returned, rather than only the final result , which can help user to trace relationship between model.

For example, in a workflow where a textblob-model is chained with a hugging-face-bert model, the output of the sentiment analysis could be passed as input to the BERT model for further processing. However, only the final result from the hugging-face-bert model is returned, making it hard to validate the intermediate sentiment analysis output.

Describe the solution you'd like Here’s an example of how you could modify your InferenceGraph to print the output of each model step:

apiVersion: "serving.kserve.io/v1alpha1"
kind: "InferenceGraph"
metadata:
  name: "chained-models"
  namespace: "default"
spec:
  nodes:
    root:
      routerType: Sequence
      steps:
        - serviceName: sentiment-model
          name: sentiment-model # step name
          printResponse: true # New field to print sentiment-model's output
        - serviceName: huggingface-bert-isot
          name: huggingface-bert-isot
          data: $response
          printResponse: true # New field to print huggingface-bert-isot's output

output like this

{
  "sentiment-model": {"prediction": "0.5"},
  "hugging-face-bert": {"prediction": "real"}
}

Anything else you would like to add: No

Links to the design documents: [Optional, start with the short-form RFC template to outline your ideas and get early feedback.] [Required, use the longer-form design doc template to specify and discuss your design in more detail]

[Shai1-Levi]

/assign