taktix / Usenetr

4 stars 0 forks source link

parsing with subject filters is rediculously slow #10

Open taktix opened 14 years ago

taktix commented 14 years ago

this is likely due to the fact that a single file can be thousands of posts and there is so much db access happening. a possible solution is to do the post consolidation at crawl time, keeping the files in memory