[Ingest pipelines] Improved UX for pipeline processor debugging

The pipeline processor hands it off to a sub-pipeline. But there is no way to jump into that pipeline and see what it is exactly doing in the UI. You just see a pipeline processor that either gets a checked, skip or failure icon.

Would be nice to click on the pipeline processor and it “opens” up the sub pipeline and you see it AND can modify it right from the one that is calling it.

e.g.

PUT _ingest/pipeline/demo-pipeline
{
  "processors": [
    {"set": {
      "field": "abc",
      "value": "demo"
    }},
      {"pipeline": {
        "name": "demo-subpipeline"
      }
    }
  ]
}

PUT _ingest/pipeline/demo-subpipeline
{
  "processors": [
    {"set": {
      "field": "subpipeline-says",
      "value": "hello"
    }}
  ]
}

POST _ingest/pipeline/demo-pipeline/_simulate?verbose
{
  "docs": [
    {
      "_source": {}
    }]
}

You see it in the verbose:

{
  "docs": [
    {
      "processor_results": [
        {
          "processor_type": "set",
          "status": "success",
          "doc": {
            "_index": "_index",
            "_version": "-3",
            "_id": "_id",
            "_source": {
              "abc": "demo"
            },
            "_ingest": {
              "pipeline": "demo-pipeline",
              "timestamp": "2024-04-18T07:02:21.924907587Z"
            }
          }
        },
        {
          "processor_type": "pipeline",
          "status": "success"
        },
        {
          "processor_type": "set",
          "status": "success",
          "doc": {
            "_index": "_index",
            "_version": "-3",
            "_id": "_id",
            "_source": {
              "subpipeline-says": "hello",
              "abc": "demo"
            },
            "_ingest": {
              "pipeline": "demo-subpipeline",
              "timestamp": "2024-04-18T07:02:21.924907587Z"
            }
          }
        }
      ]
    }
  ]
}

When checking the _ingest.pipeline, it would be nice to be able to expand it, modify it. Maybe one of the sub pipelines is failing, which is a pain to debug.

Because you need to do the following:

Run the main pipeline with your json example
Go to the failing pipeline processor in the main
Go to output, click on data in. COPY that.
Go to the pipeline that is failing.
Click on Add documents and paste your document there.

Rinse and repeat for everything you need.

elastic / kibana

[Ingest pipelines] Improved UX for pipeline processor debugging #186658