Multiple mappings and indices per single ActiveModel/Record class?

[concept47]

Lets say I want to create two separate indexes on something like BlogPosts, so that I can do a quick search using one index (for autocomplete purposes for example) then use the other index for full blown search querying.

Is that something I can do with Tire?

So something like this (forgive me if its a little primitive):

class Post < ActiveRecord::Base
    include Tire::Model::Search
    include Tire::Model::Callbacks

    mapping do
        index_name 'autocomplete'
        indexes :title, :analyzer => 'my_ngram_analyzer'
    end

    mapping do 
        index_name 'main'
        indexes :title
        indexes :description
        indexes :author
        indexes :published_on
    end
end

Where the callbacks know to add and remove new posts from the appropriate indexes.

[concept47]

So I took a look at the source code and apparently this isn't possible in the current state. I'm wondering if anyone else would be interested in seeing this particular feature in Tire.

[karmi]

Hi,

answered your question at StackOverflow.

It does not make much sense to me that the DSL would behave like this... It would add a lot of overhead then updating the index, since the class would have to go over all inidices, and fire all the updates.

Would there be another use case for the behaviour you've outlined?

[ike-bloomfire]

It does not make much sense to me that the DSL would behave like this... It would add a lot of overhead then updating the index, since the class would have to go over all inidices, and fire all the updates

Right, but thats something you'd be aware of if you wanted to do it this way. I also don't see a reason for more than 2 or 3 indexes on a model.

I just think that this feature makes Tire a lot more flexible, and gives a user the opportunity to think of architecting their elasticsearch indexes in smarter more sophisticated ways.

Would there be another use case for the behaviour you've outlined?

For example, what if I have about 30 (massive exaggeration, I know) different models that I want to be able to search across all at once. With tire I'd have to setup 30 different indexes for each one, when I could just put everything in one index to conserve resources and speed up queries.

The one index-to-one model setup, absolutely works and I have no quarrel with it, I just think this functionality could help make Tire even more powerful than it already is

[karmi]

I just think that this feature makes Tire a lot more flexible, and gives a user the opportunity to think of architecting their elasticsearch indexes in smarter more sophisticated ways.

Absolutely. The problem, though, is not coding the support for a feature like this, but making sure it's understandable, there is enough documentation and examples for various use cases, things like that...

For example, what if I have about 30 (massive exaggeration, I know) different models that I want to be able to search across all at once. With tire I'd have to setup 30 different indexes for each one (...)

That is already quite possible with Tire:

require 'logger'
require 'active_record'
require 'tire'
require 'oj'

ActiveRecord::Base.logger = Logger.new(STDERR)
ActiveRecord::Base.establish_connection( adapter: 'sqlite3', database: ":memory:" )

ActiveRecord::Schema.define(version: 1) do
  create_table :books do |t|
    t.string   :title
    t.timestamps
  end
  create_table :magazines do |t|
    t.string   :title
    t.timestamps
  end
end

Tire.index('my-library').delete

class Book < ActiveRecord::Base
  include Tire::Model::Search
  include Tire::Model::Callbacks

  index_name 'my-library'
  mapping { indexes :title, analyzer: 'snowball' }
end

class Magazine < ActiveRecord::Base
  include Tire::Model::Search
  include Tire::Model::Callbacks

  index_name 'my-library'
  mapping { indexes :title, analyzer: 'snowball' }
end

Book.create     title: "My Book..."
Magazine.create title: "My Magazine..."

Book.tire.index.refresh
Magazine.tire.index.refresh

puts '-'*80
puts 'Books:     ' + Book.search { query { all } }.results.map(&:title).inspect
puts 'Magazines: ' + Book.search { query { all } }.results.map(&:title).inspect
puts 'Library:   ' + Tire.search('my-library') { query { all } }.results.map(&:title).inspect

[ike-bloomfire]

My argument was not that you couldn't do it, but that it was inefficient.

[karmi]

Now I'm confused. You said:

With tire I'd have to setup 30 different indexes for each one, when I could just put everything in one index to conserve resources and speed up queries.

The code I posted demonstrates that you can, in fact “put everything in one index” -- did you try it?

(It's possible to use one index for multiple, separate models. It's not possible to index one model into multiple, separate indices.)

[ike-bloomfire]

The code I posted demonstrates that you can, in fact “put everything in one index” -- did you try it?

Yes. I'm aware of this, its how I got halfway around this issue.

That was total brain freeze on my part, not sure why I even made that particular argument to begin with.

[karmi]

No worries :) There's no discussion that being able to "split" the model indexing into multiple indices would be a very nice feature (eg. because of the autocompletion thing, etc), but I'm really worried about meta-stuff such as explaining it properly, making sure it does not have "surprises", etc. Let's keep it open as a reminder.

[ike-bloomfire]

No problem, I can be on the hook for writing up documentation and helping out with this feature if you ever decide to add it in.

[matthewford]

@karmi is this always the case? when I tried this I was seeing that second mapping loaded did not include the analysers

[karmi]

@matthewford You mean for the Book and Magazine example? Yeah, the first one wins -- the index should be created in advance outside of the class definition, which is often the case for complicated setups. I should update the example.

[matthewford]

Thanks, yes this caught me out too. I've now moved the index creation into a rake task.

[karmi]

@matthewford Better still, create it as a eg. a class/module method, and call it from the Rake task. This way, you can re-use it in tests setups, etc.

[demersus]

I actually have a use case for multiple indexes & mappings per model. Our website is an online marketplace. We have local classified listings, and online store products. We have multiple models which fit the "local" listing description, each with their own index. (Classified, Vehicle, RealEstate, etc...) Now, I would also like to index some of them in a separate index for items that can be purchased online and shipped. (Store products) A store product can be both a local classified, and a shippable product. Not all store products will be listed as a local classified, and not all local classifieds will be purchasable online and shipped.

It all depends on what the searcher wants. Do they want to find items in their vicinity, or do they mind having it shipped?

I guess this is a weird edge case.... But being able to manage multiple indexes, and mappings per model would be great!

[karmi]

@demersus Hmm, the way I see it, the "store product" would either be a different Ruby class, which would be composed from a Classified or Vehicle, and have some other info on top of that. That class could perfectly live in a specific index.

Or you could filter your search results based on a specific property of the Classified/Vehicle/etc class. Either way, I don't see this as a good match for "multiple indexes & mappings per model" -- quite the contrary?

[karmi]

Closing the issue, let's reopen or create new one in the future when there's some code sketch.