search

nxus / searcher

MIT License
0 stars 0 forks source link

Doc `model` field - may conflict with source document field #19

Open loppear opened 5 years ago

loppear commented 5 years ago

As @davidkellerman noted in #18 we currently add a model field to the documents that is intended to allow multiple model classes to be stored in a single ES index - added on indexing, and forced as part of the query when searching (search results must be of a single model type today?).

Given what we've learned about ES mapping definitions (yes you can put any docs in, but if they have differing inferred types for a given field they'll fail), it seems pretty unlikely we actually would support heterogenous models in the same index rather than using separate ES indexes.

Therefore, it probably makes sense to remove this model field and query filter entirely, just have to review current uses to make sure queries and searchable aren't actually depending on it somewhere.

loppear commented 5 years ago

Looking at this a bit more, GFP definitely currently relies on multiple models stored in the same index (probably mostly for the inconvenience of nxusrc for multiple connections with related/overlapping definitions, e.g. most of these models have a complex address field that would be duplicated naively in separate index mapping definitions) and rely on unique fields with unique mapping needs having different names. That project's custom query building includes the {filter: [{term: {model}]} to restrict appropriately (but also reminds me that uses today could search across model types by leaving that out of a custom query).

So this issue is probably a big refactor of client code, whether we moved to remove this field or give it a safer internal name like __modelKey - suggest we just document that you don't want a field named model in your document for today.

davidkellerman commented 5 years ago

When I configured searcher for GFP, I thought from the documentation you were supposed to create a single ES index for an app, and map all your Waterline models into it. (It did seem rather awkward.)IIRC, there isn't an example in the docs of setting up multiple ES indexes.

That said, the searcher configuration adds a huge blob of JSON to .nxusrc, and splitting it into separate indexes would have made it worse. And raised the issue of sharing analysis definitions across indexes.