GlobalMaksimum / sadedegel

A General Purpose NLP library for Turkish
http://sadedegel.ai
MIT License
92 stars 15 forks source link

Parallelization of Vectorization #183

Open dafajon opened 3 years ago

dafajon commented 3 years ago

Vectorization is now wrapped in a sklearn compatible transformer for pipelining. Next step ahead would be paralleization over multiple CPUs to speed up the processing. Right now single core is a major bottleneck before training and inference while working with large datasets.