search

aboSamoor / polyglot

Multilingual text (NLP) processing toolkit
http://polyglot-nlp.com
Other
2.31k stars 337 forks source link

question about the speed #220

Open septfish opened 4 years ago

septfish commented 4 years ago

I want to increase the detection speed. I used multi-threading, but found that the speed is slower. How to increase the speed? Or how to properly parallelize the detection text

alexgarel commented 4 years ago

Did you try to use multiprocessing ?

In python thread are generally a bad idea, because of the GIL (Global Interpreter Lock). You're best bet is generally with multi process.