search

loopbackio / loopback-connector-elastic-search

Strongloop Loopback connector for Elasticsearch
MIT License
79 stars 56 forks source link

Add the `include` filter #28

Open pulkitsinghal opened 9 years ago

pulkitsinghal commented 9 years ago

Notes to learn from mongodb's implementation:

  1. https://github.com/strongloop/loopback-connector-mongodb/blob/367bc033a546130b9f834d17a9d4f769aaefef92/lib/mongodb.js#L730
  2. https://github.com/strongloop/loopback-connector-mongodb/blob/2a90951c482508bdc06d04e5a5b87e8e54f1ae6f/test/imported.test.js#L9
  3. loopback-datasource-juggler/test/include.test.js

In the meantime a workaround is to use the native filter instead of include to write the query using ES's DSL instead of loopback. Ofcourse you need to have already setup parent-child mappings in the datasource correctly no matter what.

pulkitsinghal commented 9 years ago

cc @anana54, @rabee3 and @zelphir - this issue may be of interest to you when its finished

RabeaWahab commented 9 years ago

thanks @pulkitsinghal, will be checking it.

pulkitsinghal commented 9 years ago

I'm thinking that this also ties into the finishing #24 first ... should I make the include filter smart enough to look at the mappings and then form queries differently to support both nested and parent/child approach?

Should I do the work in increments so that until posts for children #24 (in parent/child) aren't done ... only nested mappings are supported for includes.

Honestly I am not a fan of nested and like the parent/child approach more but someone who scales differently in production might disagree. So I would like to write the best generic implementation possible. Will have to give this some more thought.

aquid commented 7 years ago

@pulkitsinghal Please close this issue if you think the latest release serves our purpose. I know you want a better solution for include rather than using loopback's fallback way. Please refer to the commit above and do the needful.

pulkitsinghal commented 7 years ago

@aquid - How does the include implementation handle or behave in extremely large datasets? Can you or Bhushan share some numbers? If not that then maybe at least a high level prediction and reasoning if it will do well or poorly when 1k or 10k or 100k or 1m results are processed as part of a matching include query?