RFC: Investingate Search Options

[cchristelis]

RFC:

Problem:

The site is very search centric and as such choosing the best search solution is important.

Proposed Solution:

A very common search app is haystack (http://haystacksearch.org/). This in conjunction with Solar may be the correct solution for this site.

Expected Outcome:

• Briefly review alternatives and if there is a much better solution we should consider this.
• Investigate haystack and solar to define the system requirements and estimate the time and effort to implement this.
• Provide a implementation breakdown.

[cchristelis]

Perhaps it is worth investigating this elastic search docker as an alternative: https://github.com/dockerfile/elasticsearch

[gubuntu]

Consider django sphinx as well. We use it in t4a.

Also postgresql full text search. On 16 Apr 2015 1:53 PM, "cchristelis" notifications@github.com wrote:

Perhaps it is worth investigating this elastic search docker as an alternative: https://github.com/dockerfile/elasticsearch

Reply to this email directly or view it on GitHub https://github.com/kartoza/feti/issues/10#issuecomment-93716220.

[lucernae]

Summary

The idea is to use haystack as a wrapper to the search engine. We will communicate (and define) our search via haystack to the search engine. Haystack support some search engine backend. Haystack itself needs a backend to work. We will probably implement this backend service as a separate docker image and configure it to access our database.

Solr

• Haystack suggest Solr 3.5+
• Solr needs Java
• If we want to use docker, consider --> https://registry.hub.docker.com/u/dockerimages/docker-solr/ and https://registry.hub.docker.com/u/dockerimages/docker-solr-systemd/
• Needs pysolr binding
• Solr have their own schema.xml . We need to generate this schema using haystack, everytime we update our models and indexes, and give it to solr
• support spatial search
• support full text

ElasticSearch

• Haystack suggest ES 0.9+
• ES needs Java
• If we want to use docker, consider https://github.com/dockerfile/elasticsearch
• Needs elasticsearch-py binding
• support spatial search
• support polygon search
• support full text

Whoosh

• Haystack suggest Whoosh 1.3.1+
• Pure python
• It says, good for small scale live deployment, and easy to setup --> but I think it means not ready for production
• doesn't support spatial search

So, I think we will use ElasticSearch backend, maybe it is less complicated to setup and manage (compared to Solr), and it provides more functionality. If we want a quick solution, we just have to use haystack instead of implementing a wrapper of postgres's full text.

Implementation plan

• [x] We will need a docker image of ElasticSearch and set it up as a search engine of our database.
• [x] Install Haystack on Django, and set the connection URL to our ElasticSearch backend
• [x] Implement Django models and Haystack SearchIndex so Haystack can understand how do we index the data
• [ ] Implement SearchView and/or SearchForm, because I think we will need more than just a simple one search box. We will implement the search criteria using SearchForm, and have it connected with SearchQuerySet API to return results.
• [ ] We can apply additional sorting method or filtering geographic information using Spatial Search, which is said very similar to GeoDjango.

[lucernae]

The investigation part is pretty much over...