ElasticSearch: not the same sort on the server and on the client

[talalav]

I have the same problem. My elasticsearch query is: { "query": { "match": { "title": "xxxx yyyy } }, "sort": { "_score": { "order": "desc" } } } The sort is not the same. I'm using the 2.0.9 version of your package.

[matteodem]

How did you configure your EasySearch instance?

[talalav]

@TopicIndex = new EasySearch.Index(
  fields : ['title']
  collection : Topics
  limit: 20
  engine: new EasySearch.ElasticSearch 
    query: (searchObject) -> {match: { title: { query: searchObject.title } }}  
    sort: (searchObject) -> { _score: { order: "desc" } }
  )

This is CoffeeScript code.

[talalav]

Hello Matteo! I wrote my settings. How do you find them? The issue is still labelled as waiting for something :)

[matteodem]

I don't really have the time now to investigate in that issue, feel free to have a closer look at the source code yourself in the mean time.

[micant]

I'm having the same issue, I'll post a solution if I find one.

[micant]

This looks like it might be the problem - https://jira.mongodb.org/browse/SERVER-14083

We got around it buy passing in a meteor sort specifier in collection.find().

[micant]

@talalav feel free to PM me if you'd like to see our workaround

[matteodem]

Yeah that's the problem. That's rather bad

[moonrockfamily]

I worked around this issue by hacking the engine.js on line 179. https://github.com/EduShareOntario/jobCollectionService/blob/306d73e2a4a2199e2bdbb812de7c726611ff61df/packages/easysearch_elasticsearch/lib/engine.js

from:

        }, { limit: options.search.limit });

to:

        }, { limit: options.search.limit, sort: options.search.sort || {} });

[matteodem]

Is this a valid fix to this bug? Feel free to create a PR if so

[talalav]

This doesn't work for me.

[moonrockfamily]

That's too bad you didn't have success with my hack.

Is it possible options.search.sort is undefined?

[talalav]

@TopicIndex = new EasySearch.Index(
  fields : ['title']
  collection : Topics
  limit: 20
  engine: new EasySearch.ElasticSearch 
    query: (searchObject) -> {match: { title: { query: searchObject.title, fuzziness: 2 } }}  
    sort: (searchObject) -> { _score: { order: "desc" } }
  )

[moonrockfamily]

I had another look at the hack, and it seems to be flawed as you have experienced. On line 153 body.sort = this.callConfigMethod('sort', searchDefinition, options); the result of the call to the sort config method is stored in body.sort, but then not used on line 179 when performing the Mongo collection find.

cursor = options.index.collection.find({
  $or: _.map(ids, (_id) => {
    return { _id };
  })
}, { limit: options.search.limit, sort: options.search.sort || {} });

Try changing line 179 to use body.sort instead of options.search.sort:

cursor = options.index.collection.find({
  $or: _.map(ids, (_id) => {
    return { _id };
  })
}, { limit: options.search.limit, sort: body.sort || {} });

[talalav]

Indicates an error in the server console: Error: Exception while polling query {"collectionName":"topics","selector":{"$or":[{"_id":"jZWN56vgzLhT7dZuC"},{"_id":"5Mj8DrG7kXbFNFmD4"},{"_id":"9PoE8FYKwb3BAcvoY"},{"_id":"sG6bdqX38dFv5eJTj"},{"_id":"Hmbm8LfoHuvRd73ZF"},{"_id":"A3vrFireMREqEp3aa"}]},"options":{"transform":null,"limit":10,"sort":{"_score":{"order":"desc"}}}}: bad sort specification

[moonrockfamily]

It's coming back to me now, I was down this path before... _score is an ElasticSearch provided value, not a property of the collection documents. Preserving the order will require more work as I described in my commit message.

- todo: replace hack augmentation using the cursor 'observe' to alter the docs to have a __sortPosition attribute equal to the position the document id is within the original elasticsearch hits.hits result.

[steven-tib]

Hi, any progress on this issue? I think I'm facing the same problem. The results are displayed randomly on the client however when testing the query/sort directly on the elastic server its correctly sorted. @moonrockfamily @talalav have you manage to sort your data properly since last November? thanks for your help

[moonrockfamily]

I didn't have a need to strictly respect the elastic search order and hence post sorting by another document attribute was sufficient. The small change I made to the engine just enabled the passing through of a sort config restricted to document attributes.

Extending the observe callbacks to add the sort position is the solution I think would allow sorting the collection consistently with the elastic search results.

[talalav]

Hi! I switched to MongoDB text search.

23 янв 2017 г. 6:59 PM пользователь "Steven Maurin" < notifications@github.com> написал:

Hi, any progress on this issue? I think I'm facing the same problem. The results are displayed randomly on the client however when testing the query/sort directly on the elastic server its correctly sorted. @moonrockfamily https://github.com/moonrockfamily @talalav https://github.com/talalav have you manage to sort your data properly since last November? thanks for your help

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/matteodem/meteor-easy-search/issues/478#issuecomment-274527662, or mute the thread https://github.com/notifications/unsubscribe-auth/AF5JyOoDRLRCXJfN86genah-YFlRlPGwks5rVM5HgaJpZM4H8QRA .

[steven-tib]

Ok thanks for your replies, I'm switching also to mongodb text search for now and see when I'll reach the limits of this solution, then I might go back to Elasticsearch.

[matteodem]

I don't have a lot of time to provide a great elasticsearch solution right now, please feel free to create a PR if you want to help out. I can guide you if there's any problems.

[steven-tib]

Thank you @matteodem, on my side I will just go with mongo text search as it seems to work for what I need for my MVP, I might come back later to elasticsearch so whenever you have more time just let us know, I can help to run some tests when you update it. thanks again for this package.

[matteodem]

@steven-tib please check out if this bug is fixed with the latest elasticsearch release. I'll re-open if that's not the case

[steven-tib]

@matteodem thanks for the update. I'm not yet back to elasticsearch and still using mongo text search. probably in the next month I will come back to ES. I will then let you know if this bug is still occurring.