Internal add queue - Githubissues

fergiemcdowall / norch

A search server that can be installed with npm

655 stars 45 forks source link

Internal add queue #106

Closed fergiemcdowall closed 8 years ago

fergiemcdowall commented 8 years ago

At the moment if many users add documents at the same time the system will not calibrate properly. What is needed is an internal queue/batch system

eklem commented 8 years ago

:+1:

megastef commented 8 years ago

This might be of high interest for this project: Skiff from @pgte - Implementation of the Raft consensus algorithm in Node.js https://www.npmjs.com/package/skiff
http://blog.yld.io/2014/10/27/skiff-raft-consensus-in-node-js/

fergiemcdowall commented 8 years ago

Hi @megastef and thanks for the link- interesting stuff. If I understand it correctly raft consensus allows key value ranges to be distributed among several nodes in a consistent manner? How could raft consensus be used in the context of norch (and/or search-index)?

fergiemcdowall commented 8 years ago

Concurrent indexing available in latest release