loopbackio / loopback-connector-elastic-search

Strongloop Loopback connector for Elasticsearch
MIT License
78 stars 56 forks source link

Optimize index and mapping creation #53

Open pulkitsinghal opened 7 years ago

pulkitsinghal commented 7 years ago

We must make sure that the following optimization is possible for most if not all versions of ES.

        /* 
         *       1. create a data structure from `self.settings.mappings`
         *          that has all mappings grouped under a unique index name
         *       {
         *         shakespeare : {
         *           index: "shakespeare",
         *           body: {
         *             mappings: {
         *               modelNameOrExplicitTypeName_A: {
         *                 properties: {...}
         *               },
         *               modelNameOrExplicitTypeName_B: {
         *                 properties: {...}
         *               }
         *             }
         *           }
         *         },
         *         juju : {
         *           index: "juju",
         *           body: {
         *             mappings: {
         *               modelNameOrExplicitTypeName_C: {
         *                 properties: {...}
         *               },
         *               modelNameOrExplicitTypeName_D: {
         *                 properties: {...}
         *               }
         *             }
         *           }
         *         }
         *       }
         *
         *       2. Then for each entry, optimize mapping and indexing to happen in one atomic request to ES
         */

References:

  1. ES 0.90
  2. ES v1.x
  3. ES v2.x
  4. GitHub code search indices.create mappings extension:js
    1. https://github.com/MennaDarwish/DSP/blob/e6156f3ea41294daeefc73b9a00fb92d232213ba/lib/elasticsearch/indices_initializer.js#L11