Deprecate embedded ElasticSearch usage

[karussell]

ElasticSearch does not really support embedded mode anymore, see discussion here.

We have to migrate to an external server. We already support the external server so for now we just have to change the documentation and deprecate the embedded usage.

[karussell]

See #299 as alternative way

[ddmng]

Hi! About:

We already support the external server so for now we just have to change the documentation and deprecate the embedded usage

could you suggest me how how to start it so that it connects to an external elastic server?

Thanks.

[ddmng]

OK, found -transport-addresses :+1:

[karussell]

(questions please to the forum next time :) )

[lonvia]

I'm a bit torn on this one. I believe that photon comes as a single executable jars is one of it strong points. Download data file, download jar and you have a ready-to-use geocoder. No complicated database setups, fighting with docker containers or library dependency hell. So if there is a way to keep the bundled version, I'd be happy to maintain that. Maybe there is a way to bundle it but run it in a separate child process.

Independently whether it works or not, we should start offering a photon jar without a packaged ES for those that run their own server already. It is highly likely that people prefer to do that in production.

[otbutz]

ElasticSearch is a beast and we do a disservice to users when we try to hide its complexity. And we would also end up with two separate installation models which gives further surface to bugs.

IMHO a separate ES installation is the way to go forward as it's also more suitable for production usecases. But we could still ship some kind of bootstrap script which sets up an ES installation with a downloaded data file.

[otbutz]

Future versions of Elasticsearch will move from the Apache 2.0 license to dual licensing under Server Side Public License (SSPL) and the Elastic License: https://www.elastic.co/de/blog/licensing-change

This does not affect API clients though: https://www.elastic.co/de/pricing/faq/licensing#i'm-using-elasticsearch-via-apis-how-does-this-change-affect-me

[lonvia]

It does deliver a death blow to the bundled version, though, because we can't relicense Photon under SSPL without major pains.

[otbutz]

Looks like Elastic decided this one for us...

[reuschling]

It would also be a chance to switch to a current Elasticsearch version. Currently they are at 7.10.2. The solution as an Elasticsearch plugin would result into following installation process:

1. Install Elasticseach (unzip the downloaded ES file or use the ES version from your OS)
2. Invoke bin/elasticsearch-plugin install file:///path/to/photonPlugin.zip or bin/elasticsearch-plugin install https://some.domain/path/to/photonPlugin.zip
3. Deploy data file

Easy enough from my opinion - but these three points could further be bundled into one installation script of course.

[otbutz]

This could work, at least from the licensing perspective: https://www.elastic.co/de/pricing/faq/licensing#im-building-plugins-for-elasticsearch-or-kibana-how-does-this-change-affect-me

For the avoidance of doubt, building a plugin to be used in Elasticsearch or Kibana does not constitute a derivative work, and will not have any impact on how you license the source code of your plugin.

[lonvia]

Given Elastic's general behaviour, I'd be rather careful with anything that bundles Photon closer with ES than necessary.

[reuschling]

Thats a valid point. Though, the difference is not so big, if you doesn't use plain json-http calls for ES communication. Otherwise, I see currently two posibiities:

1. No bundling at all:
   • Use ES Rest high level client, you are in need to take care of Api changes from ES
   • Give communication parameters (ip, index, etc) to Photon for each installation manually, e.g. inside a config file or at startup
   • Two seperate processes (don't have to be a disadvantage) and installation directories
   • You are in need to start an own http server for the photon Rest Api, photon creates new http calls to ES
2. Deliver as a plugin (to be honest, I do so for my software):
   • Implement the ES plugin interface and create a handler class. There is a chance that they change this Api in future also, but it's only a few lines of code.
   • The client object comes from the interface - there will be the same need to take care of Api changes as in the Rest high level client
   • Create a small plugin description textfile (just classname, plugin version, optionally targeted ES versions)
   • Create a small standard java security manager textfile for hard disk permissions, entry example: 'permission java.io.FilePermission "./data/dynaq/retrievalmetrics/-","read";'
   • One process, one installation directory (ES directory), no need to give communication parameters
   • The Rest api runs in ES, so you will have just an enhanced ES Api as endpoint - no second one. The plugin client object delivers the calls to ES as optimal as possible

I doesn't know if it is possible because of the new license, but technically it would be possible to deliver an elasticsearch zip file with preinstalled photon plugin. In this case you would have the same bundled one-file situation as before. At least this is common in the context of docker images.

[otbutz]

I'd favor a standalone solution. This allows users to run photon with standard elastic search instances like prebuilt containers, hosted solutions, etc

[lonvia]

The ease of installation of Photon is one of its unique selling features. Whatever the future of Photon it needs to include an equivalent of the current embedded mode.

I'm finally closing this issue as not planned.