search

fergiemcdowall / norch

A search server that can be installed with npm
655 stars 45 forks source link
search search-algorithm search-engine searchbar

readme

NPM version NPM downloads MIT License Build Status Join the chat at https://gitter.im/fergiemcdowall/search-index

      ___           ___           ___           ___           ___      
     /\__\         /\  \         /\  \         /\  \         /\__\     
    /::|  |       /::\  \       /::\  \       /::\  \       /:/  /     
   /:|:|  |      /:/\:\  \     /:/\:\  \     /:/\:\  \     /:/__/      
  /:/|:|  |__   /:/  \:\  \   /::\~\:\  \   /:/  \:\  \   /::\  \ ___  
 /:/ |:| /\__\ /:/__/ \:\__\ /:/\:\ \:\__\ /:/__/ \:\__\ /:/\:\  /\__\ 
 \/__|:|/:/  / \:\  \ /:/  / \/_|::\/:/  / \:\  \  \/__/ \/__\:\/:/  / 
     |:/:/  /   \:\  /:/  /     |:|::/  /   \:\  \            \::/  /  
     |::/  /     \:\/:/  /      |:|\/__/     \:\  \           /:/  /   
     /:/  /       \::/  /       |:|  |        \:\__\         /:/  /    
     \/__/         \/__/         \|__|         \/__/         \/__/

Deploy

npm install norch and then start with norch

or programatically:

require('norch')(options, function(err, norch) {
  // Norch server started on http://localhost:3030 (or the specified host/port)
})

Put stuff in

curl -X POST -d @myData.json http://localhost:3030/add (where myData.json is a newline separated file of JSON objects)

Search for hits (uses search-index's API)

http://localhost:3030/search?q={"query":[{"AND":{"*":["usa"]}}]}

(http://localhost:3030/search returns everything)

Make autosuggest

http://localhost:3030/matcher?q=usa

Export, import, and replicate an index

# create a snapshot on the server (available under /latestSnapshot)
curl -X POST http://localhost:3030/snapshot
# get latest snapshot
curl -X GET http://anotherIndex:3030/latestSnapshot > export.json
# replicate an export file into a new index on another server
curl -X POST -d @export.json http://someOtherServer:3030/import

API

Endpoint Method Response Typical Use Case
/add POST status code Add documents to the index
/availableFields GET stream Discover the name of fields which can be searched in
/buckets GET stream Aggregate documents on ranges of metadata
/categorize GET stream Aggregate documents on single metadata values
/concurrentAdd POST status code For when more than one source is adding documents to the index at the same time
/createSnapshot POST status code Create a snapshot of the index
/delete DELETE status code Remove documents from index
/docCount GET object Counts total document in index
/flush DELETE status code Remove all documents from index
/get GET stream Get documents by ID
/import POST file Import/merge an existing index into this one
/latestSnapshot GET file Download the latest index snapshot
/listSnapshots GET file See list of snapshots
/match GET stream Match by linguistic similarity- autosuggest, autocomplete
/search GET stream Search in the index
/totalHits GET object Show number of hits that a given query returns

The Norch API docs are here. Norch is essentially an http wrapper around search-index.

About Norch

Norch.js is an experimental search engine built with Node.js and search-index featuring, Full text search, Stopword removal, aggregation Matching (Autosuggest), Phrase search, Fielded search, Field weighting, Relevance weighting (tf-idf), Paging (offset and resultset length)

Logging

On Linux and OSX. Install bunyan, tail the log-file and pipe to bunyan.

Install bunyan:

npm install -g bunyan

Tail log-file:

tail -f log-info.log |bunyan

Mailing list: norchjs@googlegroups.com - subscribe by sending an email to norchjs+subscribe@googlegroups.com

License

MIT, Copyright (c) 2013-16 Fergus McDowall