olivernn / lunr.js

A bit like Solr, but much smaller and not as bright
http://lunrjs.com
MIT License
8.96k stars 548 forks source link

Add support for storing/retrieving field values #88

Closed pentode closed 10 years ago

pentode commented 10 years ago

I've made a (successful!) first pass at adding support for storing/retrieving field values. I added a fieldStore property to lunr.Index and populate that when the stored property of the field is true, e.g.,

var index = lunr(function () {
  this.ref('id');
  this.field('title',{boost:20, store: true});
  this.field('subtitle',{boost:10, store: true});
  this.field('bio',{boost:5});
  this.field('toc',{boost:5});
  this.field('desc',{boost:4});
});

and the result from lunr.Index.search(...) is

[ { ref: '4371',
    score: 0.02380936053390949,
    fields: 
     { title: '...',
       subtitle: '...' } },
  { ref: '395',
    score: 0.002723523438116153,
    fields: { title: '...' } } ]

To keep the serialize/deserialize functionality, I had to tweak the lunr.Store.load method so it isn't directly coupled to lunr.SortedSet, because the value of lunr.Store entries is now an object instead of a SortedSet. Maybe I shouldn't have used lunr.Store for this new fieldStore property, though?

If this is a feature that you'd like to have added, I'll clean up my work and make sure all tests are passing...

olivernn commented 10 years ago

I think storing documents in the manner you describe is probably out of scope for lunr. It seems to me that this kind of functionality is quite use-case specific and easy enough to do around lunr, if I understand the feature request properly.

I would have thought that you would have the actual documents being indexed available outside of lunr anyway. You could then retrieve these documents using the document refs that are being returned from the search.

Let me know if I have misunderstood what you're proposing, but I don't think this is something that I want to add to lunr.

pentode commented 10 years ago

I understand your points. I'm building an index (of 5000+ documents) server-side and syncing periodically to the mobile device. I'm also distributing 5000+ json files which contain the raw data for display purposes. I really like being able to show meaningful search result lists without loading the json data for each matched record. I can certainly serialize/deserialize the stored field data outside of lunr.

kevinastone commented 10 years ago

:+1: I have an offline index step, so storing document metadata in the index would be beneficial.

hoogw commented 5 years ago

In 2019 I have same need !!!!

the return result only show ref id. I actually want the real full json item associate with that ref id!!!

Otherwise, how I am going to use the returned ref id to retrieve the real full json element?

raghur commented 4 years ago

@pentode I know it's ages ago - but by any chance can you throw up your modification as a gist or something?

Uzay-G commented 3 years ago

This would be pretty useful. Maybe it could be set as an option?