No Support for Wildcarded Indexes

[finnessyj]

A great feature to be added would be the ability to wildcard indexes in elastic search. For example, if you're using time-based indexes for twitter data, and you've got twitter-2017.10, twitter-2017.11, twitter-2017.12, and twitter-2017.01, all of which have the same fields, it would be great to add a single Vector Data Source that is twitter-* (like Kibana). Elastic Search should handle that query, it would just need to be passed through and verified. Obviously, the hard part is ensuring that the fields match, and managing fields being added and removed from index to index. But...kibana can do it, and it would be fantastic if elasticgeo could do it too.

[sjudeng]

Previous versions supported a separate store configuration parameter to define wilcard indices (see search_indices in 2.11.2 store configuration documentation). This was removed after the merge of #47 (support for index aliases). Would the current support for index aliases accomodate your use case?

[finnessyj]

That looks like a good possibility. Let me try it out, and I'll report back.

[finnessyj]

Well, it worked...until tomorrow. If you have day indexes, you'll have to re-alias every day. From Elastic Search's documentation:

In this case, the alias is a point-in-time alias that will group all current indices that match, it will not automatically update as new indices that match this pattern are added/removed.

That means, if the index isn't created at the time of the alias, it won't be included, whereas a wildcard on the query will be updated at the time of the query.

[sjudeng]

Thanks for the explanation. I'll revert the commit (https://github.com/ngageoint/elasticgeo/commit/d0660ed50ca222748443904e42892f5740af8e9a) that removed search_indices as part of the next release.

[sjudeng]

Instead of restoring search_indices I added support for wildcards in the index name. Once GeoServer 2.12.2 is released I'll publish a new plugin release with these updates. In the meantime you can build and test off master if you're interested.

[finnessyj]

Awesome! Thanks! Looking forward to it.

On Tue, Jan 23, 2018 at 6:54 PM sjudeng notifications@github.com wrote:

Instead of restoring search_indices I added support for wildcards in the index name. Once GeoServer 2.12.2 is released I'll publish a new plugin release with these updates. In the meantime you can build and test off master if you're interested.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/ngageoint/elasticgeo/issues/58#issuecomment-359994389, or mute the thread https://github.com/notifications/unsubscribe-auth/AcC6A7UmvadKE5p-nMfEq5MicN9O3_fAks5tNo08gaJpZM4RWowI .

[sjudeng]

The 2.12.2-RC1 release candidate that includes this update is now published.