nupurjaiswal
commented
3 weeks ago Hi, I tried the below approaches recommended by AWS support:
-
Tried flag “lenient: true” with synonym graph filter and word delimiter filter Error: "Token filter [custom_word_delimiter] cannot be used to parse synonyms"
-
Tried multiplexer filter as below:
"multiplexer_filter":{ "type":"multiplexer" "filters":["custom_word_delimiter","custom_hunspell_stemmer","custom_synonym_graph_filter"]
I was able to create the index, but while indexing the document I got the below error
“raise HTTP_EXCEPTIONS.get(status_code, TransportError)(
opensearchpy.exceptions.RequestError: RequestError(400, 'illegal_argument_exception', 'Increment must be zero or greater: -1')”which is already mentioned under: https://discuss.elastic.co/t/word-delimiter-graph-token-synonym-graph-token/278942
- Tried multiplexer and added “lenient: true” in synonym graph filter as below:
"multiplexer_filter":{ "type":"multiplexer" "filters":["custom_word_delimiter","custom_hunspell_stemmer","custom_synonym_graph_filter"] "custom_synonym_graph_filter":{ "type": "synonym_graph", "synonyms_path": "analyzers/F3495229", "lenient":true }I was able to create index and finish indexing, but when I tried querying I wasn’t getting the current result
Result:
POST /my-index-12/_analyze
{
"analyzer": "text_en_index",
"text": "note-book"
}
{
"tokens": [
{
"token": "note book",
"start_offset": 0,
"end_offset": 9,
"type": "word",
"position": 0
}
]
}Expected result:
{
"tokens": [
{
"token": "notebook",
"start_offset": 0,
"end_offset": 9,
"type": "word",
"position": 0,
"positionLength": 2
},
{
"token": "note",
"start_offset": 0,
"end_offset": 4,
"type": "word",
"position": 0
},
{
"token": "book",
"start_offset": 5,
"end_offset": 9,
"type": "word",
"position": 1
}
]
}
Please let me know if you have any other recommendation.
Thanks, Nupur
msfroh
prudhvigodithi
dblock
Describe the bug
I am migrating from Solr to OpenSearch and need to maintain the same analyzer behavior that I had in Solr, where the word_delimiter_graph filter is applied before the synonym expansion.
In Solr, this order worked without issues, and I used FlattenGraphFilterFactory to handle token graph flattening after synonym processing. I need to maintain this specific order to keep consistent search behavior during the migration. Any guidance or suggestions would be greatly appreciated.
Solr schema:
OpenSearch mapping, where I am facing the error for the analyzer "text_en_index" and "text_en_query":
In OpenSearch, when I try to apply the word_delimiter_graph filter before the synonym_graph filter (as required by my use case), I receive the following error:
Question:
1) Is there a known limitation in OpenSearch that prevents the word_delimiter_graph filter from being applied before the synonym_graph filter? 2) Is there any recommended workaround or configuration that would allow me to maintain the same filter order while avoiding the token graph parsing error?
Related component
Other
To Reproduce
N/A
Expected behavior
N/A
Additional Details
Plugins Please list all plugins currently enabled.
Screenshots If applicable, add screenshots to help explain your problem.
Host/Environment (please complete the following information):
Additional context Add any other context about the problem here.