opensearch-project / OpenSearch

🔎 Open source distributed and RESTful search engine.
https://opensearch.org/docs/latest/opensearch/index/
Apache License 2.0
8.89k stars 1.63k forks source link

[Feature Request] Support mapping for object type and new settings for derived fields #13715

Open rishabhmaurya opened 2 weeks ago

rishabhmaurya commented 2 weeks ago

Is your feature request related to a problem? Please describe

Support mappings object type as described in https://github.com/opensearch-project/OpenSearch/issues/13143 Introduce following new settings applicable both to object and other types of derived fields -

"properties": {
   <field>: <type>
},
"source_indexed_field": <field_name>
"format": <date-format>
"ignore_malformed": <boolean>

source_indexed_field: This indexed field should be of type text and will be used to filter documents in 2 phased iterator approach before executing the expensive DerivedFieldQuery fetching from source, running script, deriving value, creating memory index. This is to improve the performance for object type. properties: fieldname and field type can be specified explicity to avoid inferring the type of sub fields within object type. format: date format to be used when type is inferred. Unrelated to object type and applicable to all types. ignore_malformed: ignores malformed documents for cases when field value cannot be derived or there is a type mismatch. If set to true, the query will ignore such documents and proceeds with rest of the matching documents.

{
  "mappings": {
    "properties": {
      "specs_json": { "type": "text" }
    },
    "derived": {
      "derived_specs": {
        "type": "object",
        "script": "emit(params._source[\"specs_json\"])",
        "format": "dd-MM-yyyy",
        "ignore_malformed": true,
        "properties": {
                "details.edition": "keyword",
                "details.pages": "long"
        },
        "source_indexed_field": "specs_json"
      }
    }
  }
}

Describe the solution you'd like

Support mapping with following definition for derived fields -

{
  "mappings": {
    "properties": {
      "specs_json": { "type": "text" }
    },
    "derived": {
      "derived_specs": {
        "type": "object",
        "script": "emit(params._source[\"specs_json\"])",
        "format": "dd-MM-yyyy",
        "ignore_malformed": true,
        "properties": {
                "details.edition": "keyword",
                "details.pages": "long"
        },
        "source_indexed_field": "specs_json"
      }
    }
  }
}

Related component

Search:Query Capabilities

Describe alternatives you've considered

No response

Additional context

No response