bennylope / elasticstack

:card_index: Configurable indexing and other extras for Haystack (with ElasticSearch biases)
BSD 2-Clause "Simplified" License
127 stars 31 forks source link

Newer versions of elasticsearch. #4

Closed trikosuave closed 8 years ago

trikosuave commented 10 years ago

I understand elasticstack has only been tested with es 0.19. I am curious if there are plans to make it work with newer versions. I want to use the ngram_analyzer which works nicely with 0.19, If i upgrade es to any thing much newer, elasticsearch basically ignores the type of search specified. I am assuming this is something between haystack and elasticsearch, but it doesn't make sense. New versions of elasticsearch have queries that we would like to take advantage of elsewhere in our site.

Thanks!

bennylope commented 10 years ago

I can't promise but an immediate resolution but I'd be keen to find one - what version of Haystack are you using?

trikosuave commented 10 years ago

Thanks! django-haystack==2.0.0 elasticstack 1.0.0

I found this but I'd really like to use elasticstack because it is cleaner: http://stackoverflow.com/questions/18908131/edgengramfield-min-and-max-letters-in-django-haystack

I tried the above solution before I found elasticstack but couldnt get it to work.

trikosuave commented 10 years ago

My test: i have a product named AAAAA. I'd like to find that by searching for 'aaa' which works with 0.19. Here are my settings with 1.0.0:

curl -XGET 'http://localhost:9200/_settings' | python -mjson.tool % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 115 693 115 693 0 0 102k 0 --:--:-- --:--:-- --:--:-- 0 { "haystack": { "settings": { "index": { "analysis": { "analyzer": { "edgengram_analyzer": { "filter": [ "haystack_edgengram" ], "tokenizer": "lowercase", "type": "custom" }, "ngram_analyzer": { "filter": [ "haystack_ngram" ], "tokenizer": "lowercase", "type": "custom" } }, "filter": { "haystack_edgengram": { "max_gram": "15", "min_gram": "2", "type": "edgeNGram" }, "haystack_ngram": { "max_gram": "15", "min_gram": "3", "type": "nGram" } }, "tokenizer": { "haystack_edgengram_tokenizer": { "max_gram": "15", "min_gram": "2", "side": "front", "type": "edgeNGram" }, "haystack_ngram_tokenizer": { "max_gram": "15", "min_gram": "3", "type": "nGram" } } }, "number_of_replicas": "1", "number_of_shards": "5", "uuid": "***", "version": { "created": "1000099" } } } } }

trikosuave commented 10 years ago

here are my 0.19 settings:

curl -XGET 'http://localhost:9200/_settings' | python -mjson.tool { "haystack": { "settings": { "index.analysis.analyzer.edgengram_analyzer.filter.0": "haystack_edgengram", "index.analysis.analyzer.edgengram_analyzer.tokenizer": "lowercase", "index.analysis.analyzer.edgengram_analyzer.type": "custom", "index.analysis.analyzer.ngram_analyzer.filter.0": "haystack_ngram", "index.analysis.analyzer.ngram_analyzer.tokenizer": "lowercase", "index.analysis.analyzer.ngram_analyzer.type": "custom", "index.analysis.filter.haystack_edgengram.max_gram": "15", "index.analysis.filter.haystack_edgengram.min_gram": "2", "index.analysis.filter.haystack_edgengram.type": "edgeNGram", "index.analysis.filter.haystack_ngram.max_gram": "15", "index.analysis.filter.haystack_ngram.min_gram": "3", "index.analysis.filter.haystack_ngram.type": "nGram", "index.analysis.tokenizer.haystack_edgengram_tokenizer.max_gram": "15", "index.analysis.tokenizer.haystack_edgengram_tokenizer.min_gram": "2", "index.analysis.tokenizer.haystack_edgengram_tokenizer.side": "front", "index.analysis.tokenizer.haystack_edgengram_tokenizer.type": "edgeNGram", "index.analysis.tokenizer.haystack_ngram_tokenizer.max_gram": "15", "index.analysis.tokenizer.haystack_ngram_tokenizer.min_gram": "3", "index.analysis.tokenizer.haystack_ngram_tokenizer.type": "nGram", "index.number_of_replicas": "1", "index.number_of_shards": "5", "index.version.created": "191299" } } }

skakri commented 10 years ago

Just came across the same issue. Elasticsearch 1.0 support (+ other issues) is in development (they switched from pyelasticsearch to elasticsearch), everything seems to be fine with elasticstack (I was worried only about ngrams, maybe there are still other issues).

pip uninstall django-haystack pyelasticsearch
pip install elasticsearch
pip install -e git://github.com/toastdriven/django-haystack.git@3c9a5305598318673f70febb30ab98bf43e079c4#egg=django-haystack-dev
./manage.py rebuild_index --noinput
trikosuave commented 10 years ago

Thanks, the ngram part works fine. However, the geo_point searches are no longer working. Before the _mapping would show string and not geo_point, now it shows geo_point but doesn't find any search results when it should. I would imagine this isn't an elasticstack issue. I just need to figure out what changed in haystack/elasticsearch. Thanks again!