search

mchaput / whoosh

Pure-Python full-text search library
Other
569 stars 69 forks source link

huge performance difference between python 2.7 and python 3.7 #9

Open yunyun27 opened 3 years ago

yunyun27 commented 3 years ago

Two virtual environments, one python 2.7, the other python 3.7 Both have same code and same data, using django-haystack with whoosh 2.7.4 The index folder is about 170M under python 2.7, about 220M under python 3.7 (same data but 50M difference in size)

But the real huge difference lies in the search speed: A basic simple search is quite fast under python 2.7 (about 0.1 sec), but really slow under python 3.7 (5~10 sec, about 100 times slower).

The following calls take the bulk of the time under python 3.7: index.searcher() index.doc_count() parser.parse(query_string) collector.prepare(self, q, context)

theiviaxx commented 3 years ago

have you tried removing haystack from the equation? im fighting this same issue and seeing if maybe haystack isnt playing nice.