wolfgarbe / SymSpell

SymSpell: 1 million times faster spelling correction & fuzzy search through Symmetric Delete spelling correction algorithm
https://seekstorm.com/blog/1000x-spelling-correction/
MIT License
3.12k stars 284 forks source link

[Question] Is the library thread safe? #90

Closed Dzoge closed 4 years ago

Dzoge commented 4 years ago

Hello,

Can I use Lookup method of the singleton SymSpell class object instance across multiple threads?

softwx commented 4 years ago

You may want to wait for Wolf for a definitive answer, since it's been a while since I worked on this project, but here are my thoughts. The place where there is a danger for multiple threads accessing a single instance is in the dictionary construction. Only one thread should construct the dictionary. Once the dictionary has been built, and is no longer modified, I'm pretty sure there's no internal state that's changed within the Lookup method that would cause conflicts when multiple threads utilize that method.

wolfgarbe commented 4 years ago

softwx is right. If single SymSpell instance is created, then Lookups can be consumed in parallel by multiple threads (e.g. parallel queries from different users to a web search engine or a data base).

Lookup is thread safe, as long as there are no LoadDictionary, CreateDictionary, and CreateDictionaryEntry in parallel.

But if you want to incrementally update your dictionary with CreateDictionaryEntry, then both Lookup and CreateDictionaryEntry have to be synchronized by locks.

While you could use a simple Lock(), a ReaderWriterLockSlim ensures better performance, because it still allows multiple Lookups in parallel:

ReaderWriterLockSlim threadLock = new ReaderWriterLockSlim();

threadLock.EnterReadLock();
try
{
      symspell.Lookup(term,verbosity);
}
finally
{
      threadLock.ExitReadLock();
}

threadLock.EnterWriteLock();
try
{
    symspell.CreateDictionaryEntry(term,frequency);
}
finally
{
    threadLock.ExitWriteLock();
}