search

searchisko / elasticsearch-river-remote

Universal remote system indexing River Plugin for Elasticsearch
Apache License 2.0
12 stars 5 forks source link

Rivers are deprecated #67

Open lukas-vlcek opened 9 years ago

lukas-vlcek commented 9 years ago

https://github.com/elastic/elasticsearch/issues/10345

velias commented 9 years ago

Sad to see this. Seems we finally have to develop own native indexers runtime in Searchisko.

unibrew commented 9 years ago

It's just a quick idea without thinking it through very much but why not reuse logstash? The only thing really missing from it is input plugin that would do the REST incremental calls, which shouldn't be very hard to write. We probably could leverage the existing tcp input plugin https://github.com/elastic/logstash/blob/v1.4.2/lib/logstash/inputs/tcp.rb. Preprocessing with calling ES when needed and then saving the result into ES is all there. Especially that filters available out of the box in logstash are far more feature rich than our structured-content-tools.

lukas-vlcek commented 9 years ago

Rivers going away in ES 2.0 (https://github.com/elastic/elasticsearch/pull/11568)