search

ulaval-rs / trombone

GNU General Public License v3.0
0 stars 0 forks source link

Determine Trombone Limits #2

Closed gacou54 closed 3 years ago

gacou54 commented 3 years ago

The limits seem to be related to memory consumption. For example, the corpus.DocumentContexts tool of 5000 documents takes about 4.5 GB, while for 10,000 documents it becomes impossible on my machine. An option to store temporary files in the file system may be a solution.

gacou54 commented 3 years ago

The storage = file option appears to limit the memory usage, using approximately 4.5 GB for 10,000 and 20,000 documents.