Meteor: Multiword searching with single/multiple fields using easy-search

[Rahbaran]

Hi, I love your package. I wonder if multiword search is possible without elastic search?

I'm not very good with MongoDB and haven't used elastic search, yet. And the easy search package has always done the job :)

However, now I have the following case:

The user can search through publications in a bibliography. An entry looks like this:

{ "_id" : "S5LpsFKBXzffde5Bc", "authors" : [ { "lastName" : "Stieger", "firstName" : "D." }, { "lastName" : "Matzler", "firstName" : "K." }, { "lastName" : "Chatterjee", "firstName" : "S." }, { "lastName" : "Ladstaetter-Fussenegger", "firstName" : "F." } ], "title" : "Democratizing Strategy", "outlet" : "Published Paper", "journal" : "California Management Review, 54 (4)", "year" : "2012", "abstract" : "Crowdsourcing is typically associated with the incorporation of company-external stakeholders such as customers in the value creating process. This article proposes a framework for a company-internal application of crowdsourcing methods. It presents a set of five goals companies can pursue employing internal crowdsourcing. The practical approach of an Austrian medium-sized technology company is described in detail, including insights on software design and appropriate procedures.", "type" : "pp" }

The Search initialisation:

PublicationsIndex = new EasySearch.Index({
  collection: Publications,
  fields: ['authors.firstName', 'authors.lastName', 'title', 'outlet', 'year', 'abstract'],
  engine: new EasySearch.Minimongo()
}); 

Here's the search initialisation (at the bottom): https://github.com/OpenStratNet/OSN/blob/master/lib/collections.js And the helper (line 17-39): https://github.com/OpenStratNet/OSN/blob/master/client/templates/members/bibliographyMembers.js

How is it possible to enter multiple words from different fields?

1st example: How can I type in "Chatterjee Crowdsourcing" (the first word from field lastName, the latter from abstract) and get a result? // search multiple words in multiple fields

2nd example: How can I type in "typically crowdsourcing" (both words from abstract but not in the saved order) and get a result? // search multiple words in a single fields in a different order

Note: The code works with a single word. Any help highly appreciated. Btw, I also posted this on Stack Overflow a few days ago: http://stackoverflow.com/questions/37394136/meteor-multiword-searching-with-single-multiple-fields-using-easy-search

[macmacs]

Yeah, just implemented EasySearch and stumbled on exactly the same Problem. Is there a way? Would be GREAT!

[matteodem]

This is typically achieved by using a MongoTextIndex engine or ElasticSearch which are both more mature approaches to doing search. Everything else is hacky and uses things such as splitting up the search string and doing some specialized mongo query.

[tim-phillips]

When using MongoTextIndex, partial word stemming does not work like it does using MongoDB engine. Is there a way to get this to work or would I have to use ElasticSearch?

MongoTextIndex docs page for reference.

[actooalizame]

I'm into the same issue.

I need to search for words in a field independently of it's order. Ex.: 'John Fitzgerald Kennedy' must be searchable by typing 'John Kennedy' or 'Kennedy John'. The MongoTextIndex option doesn't do this correctly, at all.

Is using elastic-search the only way of achieving this functionality? I found the documentation on https://atmospherejs.com/easysearch/elasticsearch pretty insufficient.

Hope you manage to improve this awesome package by enabling this search option.

[matteodem]

The documentation for the elasticsearch package is so short, since Elasticsearch itself provides a really good documentation: https://www.elastic.co/guide/index.html

[tim-phillips]

Is using elastic-search the only way of achieving this functionality?

[matteodem]

I'm not really sure why using MongoTextIndex doesn't help, can you show me your configuration?

[tim-phillips]

Apologies for being unclear. Multi-word search does work with MongoTextIndex, however, there are two problems I've found with it:

1. Partial word stemming is lost (e.g. typing 'ware' will not find records with the word 'warehouse')
2. Typing two words into the search box finds everything with those two words in it, instead of having each word filter the results

For example, searching 'mouse keyboard' finds all entries with the word 'mouse' and then also all entries with the word 'keyboard', so if there are 12 records with the word 'mouse', 8 records with the word 'keyboard' and 4 entries with both words, then all 24 records will be returned. I would expect only the 4 records with both words to be returned.

export const AssetsIndex = new EasySearch.Index({
  collection: Assets,
  fields: ['barcode', 'modelName', 'modelNumber', 'categoryName', 'manufacturerName', 'locationName'],
  defaultSearchOptions: { limit: 20 },
  engine: new EasySearch.MongoTextIndex({
    fields() {
      return {
        barcode: 1,
        modelName: 1,
        modelNumber: 1,
        manufacturerName: 1,
        locationName: 1,
        checkedOut: 1,
        createdAt: 1
      };
    },
    sort() {
      return { createdAt: -1 };
    }
  })
});

[tim-phillips]

@matteodem Any idea on how to get partial word search and multiple word search working together?

[ouya99]

@babenzele

we have the search issue.. how did you solve to get a "mouse" AND "keyboard" search functionality? I have used the mongo db full text search, which gives us a AND search but the stemming is not what we want either.. i want a full text search like regex does.

[tim-phillips]

@java99 I'm using Meteor Griddle now to solve my requirement of searchable tables.

[ouya99]

@babenzele me too :) . what do you use with griddle.. Regex or full-text search on mongo?

[tim-phillips]

@java99 I use whatever it does by default

[matteodem]

I've written up a little something in the recipe docs regarding complex use cases like you stated above.

TLDR: consider switching to ElasticSearch (or other specialized search engines) 😄