fix etext highlight

[eroux]

The current way OpenSearch highlights etext fields (and other fields too I guess) is to highlight all the tokens in the result that are in the query, which means that if the query contains "pa", it will highlight all the "pa" in the etext result, making it very noisy. It should make its best to only highlight tokens matching the query.

Two possible strategies are:

• updating the query with different highlight parameters
• do the highlighting in the API, but I don't think that's possible in Python (I have some code to do it in Java using the Lucene analyzers though)

[eroux]

this query seems to be working:

{
  "query": {
    "bool": {
      "should": [
        {
          "has_child": {
            "type": "etext",
            "query": {
              "nested": {
                "path": "chunks",
                "query": {
                  "match_phrase": {
                    "chunks.text_bo": "བྱ་ངང་བ་སེར་བོ་མཚོ་"
                  }
                },
                "inner_hits": {
                  "highlight": {
                    "fields": {
                      "chunks.text_bo": {
                        "highlight_query": {
                          "match_phrase": {
                            "chunks.text_bo": "བྱ་ངང་བ་སེར་བོ་མཚོ་"
                          }
                        }
                      }
                    }
                  }
                }
              }
            },
            "inner_hits": {
              "_source": {
                "includes": ["id"]
              },
              "highlight": {
                "fields": {
                  "chunks.text_bo": {
                    "highlight_query": {
                      "match_phrase": {
                        "chunks.text_bo": "བྱ་ངང་བ་སེར་བོ་མཚོ་"
                      }
                    }
                  }
                }
              }
            }
          }
        }
      ]
    }
  }
}

[berger-n]

just found out that the highlighted text is sometimes at the edge of the returned fragment: link