logstash-plugins / logstash-filter-elasticsearch

Elasticsearch filter for Logstash
Apache License 2.0
21 stars 84 forks source link

Backslashes in the query string not supported? #55

Open Dom-nik opened 7 years ago

Dom-nik commented 7 years ago

filter { elasticsearch { hosts => ["10.x.y.z:9200"] index => "mappings" query_template => "/etc/logstash/conf.d/mapping_test.dsl"

query => '{ "query": { "term": { "AssetType": "%{AssetType}"} } }'

          fields => {"AssetTypeGrouping" => "AssetTypeGroupingMapped"}
          enable_sort => false
 }

}

output { elasticsearch { hosts => ["10.x.y.z:9200"] index => "mappings_test" } }

- Sample Data:
My input data contains a field like this: `"AssetType":"\\Demo\\Something"`
My `mappings` index contains documents like this:
```json
{
        "AssetType": "\\Demo\\Something",
        "AssetTypeGrouping": "marketing assets"
}

:error=>#<LogStash::Json::ParserError: Unrecognized character escape 'D' [...]

NOTE: It helped (worked as expected) when I added the following step in the query parsing in the plugin code:

query_tmp = event.sprintf(@query_dsl).gsub!('\\', '\\\\\\')

So it seems that the query parser doesn't understand backslashes in the query string. Is it the solution or am I doing something completely stupid?

webmat commented 6 years ago

Sorry about the delay here :-(

Did the fix in ElasticSearch help with the issue on the Logstash side?