Problems upgrading from elasticsearch 0.90.5 to 0.90.10 with elasticsearch-analysis-stempel

[jasonthomas]

When trying to upgrade from elasticsearch 0.90.5 with elasticsearch-analysis-stempel 1.7.0 to elasticsearch 0.90.10 with elasticsearch-analysis-stempel 1.9.0 on a multi node cluster I receive the following error:

org.elasticsearch.indices.IndexCreationException: [apps] failed to create index [1]

java version info: java -version java version "1.7.0_51" Java(TM) SE Runtime Environment (build 1.7.0_51-b13) Java HotSpot(TM) 64-Bit Server VM (build 24.51-b03, mixed mode

Let me know if I can provide you with any additional information.

[1] https://gist.github.com/jasonthomas/516f17371d32f64d2565

[solarce]

+1, we're seeing this on other clusters we're trying to upgrade to 0.90.10 within Mozilla now too

[dadoonet]

Is this happening when you have an existing index created with 0.90.5? Or is this happening when you create a new index in 0.90.10?

What is your icu plugin version BTW?

[solarce]

For us, we're trying to upgrade from 0.20.5, as we're having issues with our 0.20.5 clusters having members go into GC when hitting memory limits and becoming unresponsive.

It seems to happen because we have existing indexes created with the stempel plugin, they fail to initialize and the error in https://gist.github.com/jasonthomas/516f17371d32f64d2565 appears in the log numerous times.

https://bugzilla.mozilla.org/show_bug.cgi?id=960172#c10 is tracking this work

The cluster in the bug above doesn't have the ICU plugin

$ sudo rpm -qa | grep ela [12:03:01] elasticsearch-plugin-site-bigdesk-2012072700-114752cc5b.noarch elasticsearch-plugin-analysis-stempel-1.2.0-1.el6.noarch elasticsearch-0.20.5-2.el6.x86_64 elasticsearch-plugin-site-paramedic-2012072302-fbfd358f7f.noarch elasticsearch-plugin-site-head-2012072400-abdf1ef083.noarch

bburton@elasticsearch1: /usr/share/java/elasticsearch/plugins $ ls -lah [12:03:25] total 24K drwxr-xr-x 6 root root 4.0K Mar 11 2013 . drwxr-xr-x 5 root root 4.0K Mar 11 2013 .. drwxr-xr-x 2 root root 4.0K Jan 30 16:24 analysis-stempel drwxr-xr-x 3 root root 4.0K Oct 30 2012 bigdesk drwxr-xr-x 3 root root 4.0K Oct 30 2012 head drwxr-xr-x 3 root root 4.0K Oct 30 2012 paramedic

[solarce]

Just wanted to note, I'm idling in #elasticsearch if you want to talk in more real time

[kimchy]

is there anything in the logs correlating to the index creation failure?

[cyl-moz]

The only errors I see are that, after the initial "failed to create index" errors occurs, are messages like:

[2014-01-30 14:54:32,937][DEBUG][action.search.type       ] [elasticsearch1_dev] All shards failed for phase: [query]

[dadoonet]

I can reproduce it:

wget https://download.elasticsearch.org/elasticsearch/elasticsearch/elasticsearch-0.90.5.tar.gz; tar xzf elasticsearch-0.90.5.tar.gz; cd elasticsearch-0.90.5
bin/plugin -install elasticsearch/elasticsearch-analysis-stempel/1.7.0
bin/elasticsearch -f

curl -XPUT localhost:9200/test

wget https://download.elasticsearch.org/elasticsearch/elasticsearch/elasticsearch-0.90.11.tar.gz; tar xzf elasticsearch-0.90.11.tar.gz; cd elasticsearch-0.90.11
bin/plugin -install elasticsearch/elasticsearch-analysis-stempel/1.9.0
cp -r elasticsearch-0.90.5/data elasticsearch-0.90.11
bin/elasticsearch -f

It gives the error mentioned before.

This issue comes from: https://github.com/elasticsearch/elasticsearch/blob/master/src/main/java/org/elasticsearch/index/analysis/PreBuiltAnalyzerProviderFactory.java#L44

[dadoonet]

I opened issue #5030 to track it. It could happen with other analysis plugins. For example Kuromoji is concerned as well.

Note that we will release very soon patched analysis plugins to fix that without forcing to upgrade elasticsearch.

Closing this issue for now.

Thanks for reporting and for all details you all provided!

[dadoonet]

Heya!

Could you please check if this new version works for you?

bin/plugin -install elasticsearch/elasticsearch-analysis-stempel/1.10.0

Thanks!

[cyl-moz]

The new version seems to be working fine. (Upgraded without issue and there have been no complaints for the past two weeks.) Thanks for the quick response.