search

loopbackio / loopback-connector-elastic-search

Strongloop Loopback connector for Elasticsearch
MIT License
78 stars 56 forks source link

Consecutive POST/PUT then GET results in empty response #72

Closed kamal0808 closed 7 years ago

kamal0808 commented 7 years ago

When a Create or Update is done, it takes some time to save or update in ES index. Due to this, fetching the same document in a consecutive request results in "no document found".

I got this issue when I was trying to login. It may be a possible duplicate of #60 , but I think the reason mentioned in that issue is different. I get no errors in ES logs, whereas #60 has errors in logs.

When calling the login API, an access token is created by loopback which is returned to the client. If we try to call an API just after that requires the access token, it will result in "Access Token" not found. For example,

User.login(...).$promise
.then(function(response){
    return User.profile(response).$promise;
})
.then(function(...){
...
})
.catch(function(err){
   console.log(err);  
});

The above code will result in error if User.profile() requires the Access Token created by User.login(). But it works if we set a timeout of around 2s in between.

Is it because the Access Token created need to be indexed first before it is fetched, which takes time?

wolfgang-s commented 7 years ago

UPDATED! You need to use refresh: true on ES index/update/create, so it wait's until the document is ready for searching. This should ALWAYS be used:

  elasticClient.update({ // and index or create operations
    index: 'indexName',
    type: 'typeName',
    id: 'asdf',
    refresh: true, // this is REALLY important, but of course slows down the query
    retryOnConflict: 10, // i have this as well for update opertations, just in case there are a lot of update requests on the same id and a version conflict happens
    body: {
      doc: {
...
      },
    },
  });
pulkitsinghal commented 7 years ago

@wolfgang-s - couldn't find an example of the code above in your fork's master branch, did you mean for me to check there?

wolfgang-s commented 7 years ago

Yes that is true, i changed it to make configureable via config files. That's why you did not find it. This is not perfect. I need to find a better solution ... maybe i have time tomorrow!

wolfgang-s commented 7 years ago

I just pushed my changes: 2416ef8

I removed one setTimeout from a test and it still succeeds (with refresh=true). Only problem is I'm using ES 5.0 so I have some failing tests (different API).

But everything works like a charm ;)

pulkitsinghal commented 7 years ago

@wolfgang-s Since you've done all the hard work, how about submitting a pull request? That way we can give your proper acknowledgement! cc @aquid

I am not against ES5~ing but might need some time & help from you to point out where we need improvement.

wolfgang-s commented 7 years ago

I will submit a PR with the refresh: true changes only (discarding my 5.0 changes) next week.

aquid commented 7 years ago

@wolfgang-s Any updates on the PR?

wolfgang-s commented 7 years ago

Done! Sorry for the delay ... I'm working with ES 5.0 so I had to install docker and stuff for the tests to get started with 1/2 ... here it is:

https://github.com/strongloop-community/loopback-connector-elastic-search/pull/77

aquid commented 7 years ago

@wolfgang-s No problem, already added comments in your PR.