PhonologicalCorpusTools / CorpusTools

Phonological CorpusTools
http://phonologicalcorpustools.github.io/CorpusTools/
GNU General Public License v3.0
111 stars 16 forks source link

Optimizing bigram checking #773

Open ecoates-bc opened 3 years ago

ecoates-bc commented 3 years ago

The bigram selector widget in Mutual Information and Transitional Probability checks that selected bigrams appear in the corpus. It takes longer when the corpus is large, and if many bigrams are being selected at the same time, it can take up to a minute to check. The code for searching for bigrams in the corpus should be refactored to run faster.