reyesr / fullproof

javascript fulltext search engine library
http://reyesr.github.com/fullproof/
Apache License 2.0
667 stars 46 forks source link

Can I use fullproof as a database for my application? #18

Open lmatteis opened 11 years ago

lmatteis commented 11 years ago

I'm writing an offline HTML5 app. People go to my website, and it loads an interface to manage their own personal data. No other server requests are made other than the initial loading of the page.

This data can be quite large, as the data might grow without my control. However I'd like my users to be able to safely backup their data, say to one or several JSON files to their local computer. So they can decide to maybe safely store their data somewhere else. Is this possible through fullproof? Also I'd want to re-import this data, but this is possible thanks to the examples shown on the homepage.

Any insight if this might work for my scenario? Also, last question, can I search for specific field-names? For example in Lucene you can do name:luca and it will only search for stuff in the field name.

Thanks and great job!

reyesr commented 11 years ago

Hi Luca, I'm not sure there's much value in backup'ing the indexes of a search engine in the browser. What really matters is the data, ie the documents of the user, which can and should be saved locally. When loading the data back in your application, you can just re-create the search engine database by injecting the documents. For the backup/saving in itself, fullproof currently does not store documents, it just indexes them, which means there's no way to generate the original data from what the data used and stored by fullproof.

Regarding the fields, you just have to consider a field as a specific index. You can have as many indexes as you want in fullproof, so you can add another one where you only inject the data found in some field. There's no limit to the number of fields possibly managed by fullproof, however there probably lacks some higher level interface to manage field-base documents and automatically create the engine (a scoring engine in this example) and its index + manage the search and sort the result according to field boost (aka field scores).

lmatteis commented 11 years ago

So your advice would be to use say Web SQL as the database part, and fullproof simply for the searching. So to backup/restore data, I would simply do that through Web SQL?

My worry is that Web SQL seems to be a deprecated browser storage mechanism. Any ideas what to use when browsers won't support it anymore? Is IndexedDB a valuable alternative? Is it fast enough, and can it withhold large amounts of data?

reyesr commented 11 years ago

Hi Luca, I 'd really recommand going with IndexedDB, unless localStorage is enough. Although the support for IndexedDB may lack on old browsers, if you can require a minimum version for popular browsers, I'd go for it. You can run benchmark on your own browser to compare the performance: http://reyesr.github.com/html5-storage-benchmark/

lmatteis commented 11 years ago

Hi Rodrigo, actually my only browser would be webkit (chromium) as I'm developing a desktop app using AppJS. So indexedDB is supported, but my worry was regarding the space it can hold. I've seen limits around 50mb which would be very little for my needs! Any ideas? Also I assume fullproof works with indexedDb right?