loopbackio / loopback-connector-elastic-search

Strongloop Loopback connector for Elasticsearch
MIT License
79 stars 55 forks source link

How to run boot scripts when Es connector is not initialised #66

Open aquid opened 7 years ago

aquid commented 7 years ago

If we try to run a boot script now without our connector being initialised we get back an error saying Index doesn't exist.

here's the error message

Unhandled rejection [index_not_found_exception] no such index, with { resource.type=index_or_alias resource.id=blubox-elastic index=blubox-elastic } :: {"path":"/blubox-elastic/Bluboxer/_search","query":{"size":1,"from":0},"body":"{\"sort\":[\"_uid\"],\"query\":{\"bool\":{\"must\":[{\"match\":{\"email\":\"super@blubox.in\"}}]}}}","statusCode":404,"response":"{\"error\":{\"root_cause\":[{\"type\":\"index_not_found_exception\",\"reason\":\"no such index\",\"resource.type\":\"index_or_alias\",\"resource.id\":\"blubox-elastic\",\"index\":\"blubox-elastic\"}],\"type\":\"index_not_found_exception\",\"reason\":\"no such index\",\"resource.type\":\"index_or_alias\",\"resource.id\":\"blubox-elastic\",\"index\":\"blubox-elastic\"},\"status\":404}"}
  at respond (/Users/shahwarcoder/Blubox/node_modules/elasticsearch/src/lib/transport.js:289:15)
  at checkRespForFailure (/Users/shahwarcoder/Blubox/node_modules/elasticsearch/src/lib/transport.js:248:7)
  at HttpConnector.<anonymous> (/Users/shahwarcoder/Blubox/node_modules/elasticsearch/src/lib/connectors/http.js:164:7)
  at IncomingMessage.wrapper (/Users/shahwarcoder/Blubox/node_modules/lodash/index.js:3095:19)
  at emitNone (events.js:72:20)
  at IncomingMessage.emit (events.js:166:7)
  at endReadableNT (_stream_readable.js:913:12)
  at nextTickCallbackWith2Args (node.js:442:9)
  at process._tickDomainCallback (node.js:397:17)
  at process.fallback (/Users/shahwarcoder/Blubox/node_modules/async-listener/index.js:482:15)

cc @pulkitsinghal

pulkitsinghal commented 7 years ago

@aquid - we received the following answer in the chat room:

Heath Morrison @doublemarked Jan 10 16:16
@pulkitsinghal this should not be necessary
Datasource initialization occurs before boot scripts are executed.
The datasource initialize method has a callback,
which should not be called until the datasource is fully initialized.
(afaik)
I see in your connector that you are indeed calling this callback only on connect, which makes me wonder what @aquid means when he says "without our connector being initialised” in strongloop-community/loopback-connector-elastic-search#66
wolfgang-s commented 7 years ago

@aquid I use the es connector like so as a boot script: (almost on first position, since it starts with "e" ;)):



module.exports = function (app, cb) {
  console.info('Boot waiting for ElasticSearch initialization...');
  var connected = false;

  // As soon as the datasource has been connected, move on.
  app.datasources.db.once('connected', function () {
    console.info('ElasticSearch connected, boot continuing');
    connected = true;
    cb();
  });

  // I need this, otherwise, if no ES is available, it will hang forever. PM2 then autorestarts.
  setTimeout(function(){
    if(!connected) {
      cb(new Error("retrying elasticsearch connection"));
    }
  }, 5000);
};```

Only "problem", the es connector does NOT retry infinitely, so at some point, the application just hangs, that's why I "restart" by throwing an error and pm2 takes over.
aquid commented 7 years ago

@wolfgang-s So the above piece of code goes before each boot scripts or just one boot scripts which run before all other and checks if elastic is up and running.

Can't we do a refresh configuration for index too? Is it possible?

wolfgang-s commented 7 years ago

This is a normal loopback boot module. Runs once before the rest of the boot modules.

I'm using PM2 which restarts the node app on failure (when it takes more than 5 seconds).

Actually almost the same as here: https://github.com/strongloop-community/loopback-connector-elastic-search/issues/7