Async API?

[ianwremmel]

I'm working on using IndexedDB as a persistent backend for a search index. After discovering lunr, I was going to simply use the trimmer, stemmer, and stop word filter, but then realized I was starting to write something that was algorithmically similar to lunr.Index. Unfortunately, IndexedDB is a purely async api so that won't work without rewriting most of the library.

Has there been any consideration for adding an async API to lunr?

[olivernn]

Sounds like an interesting project! Very early versions of lunr were built on top of IndexedDB. At first the index was kept in IndexedDB but I found the performance was pretty terrible, I then moved to having the index in memory and snapshotting the state to IndexedDB before ditching IndexedDB altogether. The IndexedDB API at the time was really ugly, has it got any better?

I digress. What do you mean by "an async API"? What benefit would it bring. Lunr does no IO at all, it is entirely CPU bound and will block the main thread whilst it is searching/indexing. Now you could be running lunr in a WebWorker to keep it off the UI thread, at which point you could get some benefit from an async API, but that would be a wrapper around the WebWorker, not lunr specifically.

Anyway, let me know how you get along with your IndexedDB project, I'd be interested to see what you come up with.

[ianwremmel]

The IndexeDB API is still pretty terrible, but given the amount of data I'm working with, it's the only option.

I'm hoping to be able to opensource the library I'm working, but there's a lot of bureaucracy to figure out first.

Basically, my thinking is that if all lunr methods either accept nodebacks or return promises, it would be much easier to hook it into alternate stores (be it indexeddb, web workers, or something else exposed via something like node-webkit).

[olivernn]

This still sounds like something that should be done by wrapping lunr, rather than making changes to lunr directly.