Evaluate external search indexes for Form Runner search

[ebruchez]

Currently, search is done at the database level. We have 2 ways to do search:

• with XQuery for eXist
• with SQL for relational databases

Because the data is stored in XML, search with relational databases is not easy, especially with dbs which don't support good XML indexes. So search with MySQL, for example, is bad.

One solution to improve on this was to consider storing all data in relational tables (#1571), so that XML is no longer an issue. This has drawbacks:

• this is a big change
• publishing a form requires creating tables, which some setups might not allow
• this ends up creating many tables in the database

Another idea is to keep the XML and universal tables, handle search with a separate search front-end like Solr, Elasticsearch (note that both are based on Apache Lucene!) and, if needed for analytics etc. create separate relational tables in write-only mode (#1069).

One concern is the difficulty of installation. It is an extra moving part. But this might be manageable:

• there would be one Solr instance per Orbeon Forms server, whether in the same web app or not
• for failover or load balancing, instances would be continuously synchronized as needed

[ebruchez]

+1 from customer today for Elasticsearch