karmi / retire

A rich Ruby API and DSL for the Elasticsearch search engine
http://karmi.github.com/retire/
MIT License
1.87k stars 533 forks source link

Index field on Mongoid embedded documents #814

Open General9 opened 11 years ago

General9 commented 11 years ago

How do I index a specific field on an embedded document using the tire gem for ElasticSearch's syntax?

I have tried the following for my Question model which embeds many Answers and I would like only the description field on the Answer model to be indexed.

Searching for text that should match a stored answer's description is returning no results. I am using Mongoid as my MongoDB driver by the way.

 mapping do
    indexes :id, :index => :no
    indexes :created_at, :index => :no
    indexes :updated_at, :index => :no
    indexes :title, :index => :analyzed
    indexes :description, :index => :analyzed
    indexes :page_views, :index => :no
    indexes :asker, :index => :no
    indexes :tags, :index => :analyzed
    indexes :answers do
      indexes :id, :index => :no
      indexes :description, :index => :analyzed
      indexes :answerer, :index => :no
      indexes :created_at, :index => :no
      indexes :updated_at, :index => :no
    end
    indexes :comments, :index => :no
    indexes :votes, :index => :no
    indexes :up_count, :index => :no
    indexes :down_count, :index => :no
    indexes :vote_score, :index => :no
  end
karmi commented 11 years ago

I see you have indexes(:answers) { indexes :description }, that should be it. What is the output of @question.to_indexed_json.


Side note: I don't understand all those index: "no" configs in your mapping.

General9 commented 11 years ago

@question.to_indexed_json is returning the following:

{
  "asker": {
    "user_id": "1",
    "display_name": "info"
  },
  "created_at": "2013-07-27T11:52:48.451+02:00",
  "description": "This is adescription",
  "down_count": 0,
  "page_views": 0,
  "tags": [
    "tag1",
    "tag2"
  ],
  "title": "This is a title 1?",
  "up_count": 0,
  "updated_at": "2013-07-27T11:52:48.451+02:00",
  "vote_score": 0,
  "votes": []
}

There is nothing about answers in there. Not too sure what I am missing?


And on the Side Note: I was trying to specify that I don't want those fields on the model indexed at all. Will it work if I just leave the fields out from the mapping?

karmi commented 11 years ago

Please see Elasticsearch, Tire, and Nested queries / associations with ActiveRecord - Stack Overflow. You might have to manually create the JSON by implementing the to_indexed_json method.


Please see the documentation, your assumption is incorrect.

General9 commented 11 years ago

Using the StackOverflow example I implemented the to_indexed_json method as follows:

def to_indexed_json to_json( include: { answers: { only: [:description] } } ) end

I did not use any of the touch stuff as that only works with mongoid relations on (belongs_to) in my case the Answer document is embedded_in Question. Running @question.answers.metadata.touchable? returns false.

My @question.to_indexed_json now includes answers but running a search for the text "answer" as an example still returns nothing.

@question.to_indexed_json output below:

"{\"_id\":\"51f3a10d1363185d86000002\",\"answers\":[{\"description\":\"This i s an answer.\"}],\"asker\":{\"user_id\":\"1\",\"display_name\":\"info\"},\"creat ed_at\":\"2013-07-27T12:29:33.204+02:00\",\"description\":\"This is a descriptio n\",\"down_count\":0,\"page_views\":2,\"tags\":[\"tag1\",\"tag2\"],\"title\":\"T his is a title 1?\",\"up_count\":0,\"updated_at\":\"2013-07-27T12:29:33.204+02:0 0\",\"vote_score\":0,\"votes\":[]}"

karmi commented 11 years ago

Have you re-created the index with the new mapping? Have you reindexed the data? There are Rake tasks for those.

General9 commented 11 years ago

Yes I have re-created the index. I have a rake task that purges my database, deletes the index and re-creates the index on seeding the database.

If I go back to exactly what I am trying to do maybe you can help me. It seems like it should be a simpler task than it is actually turning out to be.

So if I have a Question model of this form:

class QuestionDetail::Question include Mongoid::Document include Mongoid::Timestamps include Tire::Model::Search include Tire::Model::Callbacks

index_name("#{Tire::Model::Search.index_prefix}questions")

field :title field :description field :page_views, type: Integer, default: 0 field :asker, type: QuestionDetail::User

taggable :tags

embeds_many :answers, class_name: 'QuestionDetail::Answer' end

and Answer model of this form:

class QuestionDetail::Answer include Mongoid::Document include Mongoid::Timestamps

field :description field :answerer, type: QuestionDetail::User

embedded_in :question, class_name: 'QuestionDetail::Question' end

I am trying to use elastic search to be able to search for questions only on the following fields - (title, description, tags) from the Question model and (description) from the Answer model. The rest of the fields should not be searchable or should not return any results, for example if a user types in 2 in the search box and there is a question with 2 votes that should not return any results because votes should not be searchable.

So the question is, how do I define the mapping for that using the tire gem? My initial (1st) question shows how I was attempting to achieve this.

General9 commented 11 years ago

Ok so I have been battling with this the whole day and this is how far I have got.

My mappings:

mapping do indexes :created_at, :type => 'date', :index => :not_analyzed indexes :vote_score, :type => 'integer', :index => :not_analyzed indexes :title indexes :description indexes :tags indexes :answers do indexes :description end end

My to_indexed_json method:

def to_indexed_json { vote_score: vote_score, created_at: created_at, title: title, description: description, tags: tags, answers: answers.map{|answer| answer.description} }.to_json end

My Search query:

def self.search(term='', order_by, page: 1) tire.search(page: page, per_page: PAGE_SIZE, load: true) do query { term.present? ? string(term) : all } sort { by case order_by when LAST_POSTED then {created_at: 'desc'} else {vote_score: 'desc', created_at: 'desc'} end } end end

The only issue I am battling with now is how do I make vote_score and created_at field not searchable but still manage to use them for sorting when I'm searching.

I tried indexes :created_at, :type => 'date', :index => :no but that did not work.

General9 commented 11 years ago

I ended up going with the approach of only matching on the fields I want and that worked. This matches on multiple fields.

tire.search(page: page, per_page: PAGE_SIZE, load: true) do query { term.present? ? (match [:title, :description, :tags, :answers], term) : all } sort { by case order_by when LAST_POSTED then {created_at: 'desc'} else {vote_score: 'desc', created_at: 'desc'} end } end