[BUG] Rename field processor does not work with object/nested fields

[LantaoJin]

Describe the bug

Searching object or nested fields with rename field processor will throw llegal_argument_exception

Related component

Search

To Reproduce

Repro:

# create an index with object field (nested field could reproduce too)
PUT test_object_field_index
# add some testing data
PUT test_nested_field_index/_doc/100
{ 
  "nationality" : "USA",
  "patients": [ 
    {"name" : "John Doe", "age" : 56, "smoker" : true},
    {"name" : "Mary Major", "age" : 85, "smoker" : false}
  ] 
}
# create pipelines with rename processor
PUT /_search/pipeline/rename_pipeline1
{
  "response_processors": [
    {
      "rename_field": {
        "field": "patients.name",
        "target_field": "pName"
      }
    }
  ]
}
PUT /_search/pipeline/rename_pipeline2
{
  "response_processors": [
    {
      "rename_field": {
        "field": "nationality",
        "target_field": "country"
      }
    }
  ]
}
# query fails with llegal_argument_exception
GET /test_object_field_index/_search?search_pipeline=rename_pipeline1
# but flat field rename works
GET /test_object_field_index/_search?search_pipeline=rename_pipeline2

Expected behavior

Return docs instead of failure

Additional Details

Plugins Please list all plugins currently enabled.

Screenshots If applicable, add screenshots to help explain your problem.

Host/Environment (please complete the following information):

• OS: Mac
• Version 2.14

Additional context Add any other context about the problem here.

[dblock]

@LantaoJin Want to turn this into a (failing) YAML REST test?

[mch2]

looks related to search pipelines - @msfroh mind taking a look?

[msfroh]

This is a perfect example where a search pipeline processor has diverged from the mirrored ingest pipeline processor, where https://github.com/opensearch-project/OpenSearch/issues/13587 will help us use a single implementation.

Right now, the search pipeline processor does not resolve dot-paths to look at object/nested fields, whereas the ingest processor does.

We could go for a short-term fix that improves the search pipeline processor.

Alternatively, we can go for the longer-term solution of implementing https://github.com/opensearch-project/OpenSearch/issues/13587, to ensure that the ingest and search pipelines have identical capabilities.