universal-automata / liblevenshtein

Various utilities regarding Levenshtein transducers.
https://github.com/universal-automata/liblevenshtein
MIT License
67 stars 13 forks source link

liblevenshtein

Join the chat at https://gitter.im/universal-automata/liblevenshtein

A library for generating Finite State Transducers based on Levenshtein Automata.

Language Build Status Release Repository
Java Build Status Maven Central liblevenshtein-java
CoffeeScript Build Status npm version liblevenshtein-coffeescript
C++ Work-in-progress Work-in-progress liblevenshtein-cpp

Levenshtein transducers accept a query term and return all terms in a dictionary that are within n spelling errors away from it. They constitute a highly-efficient (space and time) class of spelling correctors that work very well when you do not require context while making suggestions. Forget about performing a linear scan over your dictionary to find all terms that are sufficiently-close to the user's query, using a quadratic implementation of the Levenshtein distance or Damerau-Levenshtein distance, these babies find all the terms from your dictionary in linear time on the length of the query term (not on the size of the dictionary, on the length of the query term).

If you need context, then take the candidates generated by the transducer as a starting place, and plug them into whatever model you're using for context (such as by selecting the sequence of terms that have the greatest probability of appearing together).

For a quick demonstration, please visit the Github Page, here. There's also a command-line interface, liblevenshtein-java-cli. Please see its README.md for acquisition and usage information.

The library is currently written in Java, CoffeeScript, JavaScript, and C++. I will be porting it to other languages, soon. If you have a specific language you would like to see it in, or package-management system you would like it deployed to, let me know.

This library is based largely on the work of Stoyan Mihov, Klaus Schulz, and Petar Nikolaev Mitankin: "[Fast String Correction with Levenshtein-Automata](http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.16.652 "Klaus Schulz and Stoyan Mihov (2002)")". For more details, please see the wiki.