Document for Node.JS

[billymoon]

This works fine on node.js on the server side too. If there was a little documentation explaining how, I think it could be useful to some people.

[olivernn]

Yeah, it should work fine on node.js. What steps did you follow to get it working there? I guess you had to add a module.exports = lunr, was there anything else?

[billymoon]

Well, actually I did not add anything at all. I did not convert it into a module, but just ran it with nodejs, so copying and pasting your example code into a new file, and running with node.

I can make it into a module if you like, and provide a short explanation. I would imagine you could add a paragraph to your docs then.

On Mar 4, 2013, at 12:29 PM, Oliver Nightingale notifications@github.com wrote:

Yeah, it should work fine on node.js. What steps did you follow to get it working there? I guess you had to add a module.exports = lunr, was there anything else?

— Reply to this email directly or view it on GitHub.

[olivernn]

Ok, glad it worked well.

I think the main focus of lunr should be on the browser (there are many options for search on the server). Having said that a lunr-server version would be quite useful, especially when combined with lunr in the browser. I think that there would be a bit of work involved in adapting lunr to work well on the server, for example persisting the index somehow. It is definitely something that I would be interested in looking into though.

[billymoon]

Well, I find it interesting. I think it lowers the barrier of entry for adding full text search on the server side, which can be fairly complicated to set up. Also, there are NodeJS only hosting services springing up, with no access to the filesystem, let alone ability to install server side software etc… It would be great to have this as a nodejs only solution.

Re: persisting storage

I imagine it like this… on loading the server, check if index has been created, if not, then create it, serialise as JSON, and store it as a string (probably in a database, or as a file). Then still at server load time, load the index from a string, parse the JSON, and store as an object in memory (simply assign to a variable). On each request to the server, use the index stored in memory - no re-building required (unless new content is added that needs indexing).

So long as the index size does not take up too much memory, I think that it would work fine.

I don't think it would take too much work really, I think you have done the hard part :)

Also, the nice thing about putting it on nodejs, is there is a growing community of developers that is absent (to my knowledge) from front-end javascript development.

On Mar 4, 2013, at 1:31 PM, Oliver Nightingale notifications@github.com wrote:

Ok, glad it worked well.

I think the main focus of lunr should be on the browser (there are many options for search on the server). Having said that a lunr-server version would be quite useful, especially when combined with lunr in the browser. I think that there would be a bit of work involved in adapting lunr to work well on the server, for example persisting the index somehow. It is definitely something that I would be interested in looking into though.

— Reply to this email directly or view it on GitHub.

[olivernn]

For now I'll put a note in the readme with some instructions on how to get this working in node.js as is.

I'll try and put together some ideas for a server version of lunr too, although it probably won't be immediately. I'll leave this issue open to collect any discussion related to a server version of lunr, feel free to add any other ideas as to what would be required.

[mcollina]

I believe a package for lunr on npm might be very useful for a lot of different reasons. For "small" problems a solution like yours might be a silver bullet on the server. And it might be the basis for a 'lunr-server'.

Let people handle persistance in their own way. Maybe you might just provide a way to boost the startup time by serializing the index over JSON, but that's not required.

[billymoon]

I couldn't agree more.

Re: persistance Yes, it should be up to the user, and more a job of documenting than including in the library. I think a couple of examples of how to handle persistence, and the significance in terms of performance vs resource overhead, would be enough for people to do what they want with it.

There are no recursive loops in the index are there? That might prevent default JSON serialisation functions, and would require a custom function to handle it. Other than that, I don't see it as necessary to include in the library itself.

I am imagining the current script is encapsulated as a module, returning an object with a few methods exposed (according to common practice in nodejs).

module.export = { createIndex: function(data){ /* function to generate the index, takes a data object to generate the index from, returns the index in a wrapper with some methods attached to it */ } }

Then from the nodejs app…

var lunr = require('lunar') var myIndex = lunr.createIndex({ /* data object to index */ }) var results = myIndex.search("string to search for")

Something like that anyway. How do you think that looks?

[olivernn]

Interesting, so I guess if lunr implemented a dump and load method which would output some JSON, then if required users could choose whatever storage suited there needs best?

It would be cool to allow people to maintain the index server side, and then send only the relevant subset of the index to the client.

It does look like this might be useful for a number of use-cases, I'll take a look at what is involved. I guess there is still scope for a lunr-server though, something that encapsulates the persistence etc, but that can probably wait for now.

[mcollina]

Also, being able to manipulate the index server side might be very nice, i.e. publishing this on npm.

[billymoon]

If the module looks like this...

var lunr = (function(){
  var lunr = function(a){var b=new lunr.Ind // the rest of the minified lunr code...
  return lunr;
}()); if(typeof module != "undefined"){ module.exports = lunr };

With this line for node only

var lunr = require('./lunr')

Then this test code should work in both browser or node

var index = lunr(function(){
  this.field('title', {boost: 10})
  this.field('body')
  this.ref('id')
})

index.add({
  id: 1,
  title: 'Food',
  body: 'Foo foo foo!'
})

index.add({
  id: 2,
  title: 'Bar',
  body: 'Bar bar foo bar!'
})

console.log(index.search('foo'))

[alessioalex]

This would be just awesome for Node also, and as you guys said above, the main thing here would be to allow for independent storage. So if you have a more complex use case, just swap the MemoryStore for MongoDB / Redis / etc The main package should just store stuff into memory, and other stores can be independent packages (like the session stores for Connect).

[olivernn]

I've just pushed version 0.2.1 which includes support for running in node. This is also available on npm, you should be able to install with npm install lunr and then use it from node with var lunr = require('lunr').

Let me know how you get along and I can add some more docs for this in the readme.

[billymoon]

Works fine for me, that's great!

[olivernn]

Issue #11 is about the serialise functionality, which, when combined with the npm module, will make generating the index on the server possible. Any discussion on how this should work etc would be welcome.

[olivernn]

Serialise has now been added as part of version 0.3.0