This patch solves problems in corpus import script with hughe memory consuming in packet about 18k documents. After this optimalization loop of one documents consume about 150 bytes. It should allow to import packages to about 5 mln documents at once.
This patch solves problems in corpus import script with hughe memory consuming in packet about 18k documents. After this optimalization loop of one documents consume about 150 bytes. It should allow to import packages to about 5 mln documents at once.