search

Norconex / committer-elasticsearch

Implementation of Norconex Committer for Elasticsearch.
https://opensource.norconex.com/committers/elasticsearch/
Apache License 2.0
11 stars 6 forks source link

Ingest Pipeline #15

Closed jmrichardson closed 7 years ago

jmrichardson commented 7 years ago

Hi,

First, thank you for this great software. I am just getting started using it, and really like how it works. Is there a way to specify a pipeline on index? See: https://www.elastic.co/guide/en/elasticsearch/reference/current/ingest.html

Thanks in advance

essiembre commented 7 years ago

Thanks for the kind words. Assuming you are using a Collector, document manipulation is performed by the Importer module. The importer module can be configured from your Collector configuration.

The importer will provide you ways to filter, modify, decorate, split, or otherwise manipulate documents before they are sent to Elasticsearch.

If you want to do it from within Elasticsearch ingestion process, you can use the Importer module on its own and wire it somehow, but given it is already part of the HTTP and Filesystem Collectors, you may as well use it there.

Does that answer?

jmrichardson commented 7 years ago

Yes, this will work instead of the pipeline. However, you may want to consider for those use cases where the processors or plugins available may have functionality not present in the importer. Just a thought.